Spot any improvements ??

[OldChap]

With the amount of time I am not getting buckets coinciding with the time I am here with the rig I find it difficult to optimise so I am using my "standard" settings.

anyone see any possible improvements?

http://www.lakecityquietpills.com/photo/multihost/images/24913281237002151917.png (http://www.lakecityquietpills.com/photo/multihost/)

http://www.lakecityquietpills.com/photo/multihost/images/86578019191772390129.png (http://www.lakecityquietpills.com/photo/multihost/)

http://www.lakecityquietpills.com/photo/multihost/images/37509284483412581316.png (http://www.lakecityquietpills.com/photo/multihost/)

http://www.lakecityquietpills.com/photo/multihost/images/16121345796029355679.png (http://www.lakecityquietpills.com/photo/multihost/)

http://www.lakecityquietpills.com/photo/multihost/images/07638428679480347942.png (http://www.lakecityquietpills.com/photo/multihost/)

Thinking of going back to a dedicated rig but the downtime due to lack of buckets is putting me off.

http://www.lakecityquietpills.com/photo/multihost/images/90874097679837616075.png (http://www.lakecityquietpills.com/photo/multihost/)

[hixie]

I've got some rather ugly crawl stats and had to remove 12 nodes. I'll try increasing the workers and see if that helps saturate my line. Does everyone have priorities set to normal?

Wish there was some link we could go to that gave us a countdown to how many buckets were left, and estimated time til empty at current crawl rate. Would make it easier to plan for maintenance. BTW, what happened to "we'll soon have a infinite number of buckets, i dare you to crawl them all!"

[OldChap]

When you asked the question, it made me remember the reason why I set it that way....

I found that when running one or more of Refic's nodes in linux, the scheduler in the kernel does a better job than the one in MJ. It seems to use less cpu time.

This is probably due to it still using mono. Think of the whole thing being a windows program forced to run on linux maybe.

[hixie]

I have a single node running on a windows machine, and that node averages about 10Mb/s. The crawl stats are much nicer than my linux nodes. Getting a high amount of timed out errors, gonna have to check it out when i get home tonight.

[hixie]

Shut down another node last night, still don't know what is causing the high DNS and timeout errors.

[OldChap]

Just notice I got some buckets:

http://www.lakecityquietpills.com/photo/multihost/images/71832000525804741982.png (http://www.lakecityquietpills.com/photo/multihost/)

[hixie]

I'm down to 8 nodes now, CPU utilization went from overloaded to average 15% for some reason. Crawl stats is still ugly compared to my control.

And how the hell do you attach photos? doesnt seem to work

[OldChap]

I use a host site for photos. click on my image and look at the address.

[hixie]

The CPU load looks much better now, the only difference was 8 nodes versus 9.

http://www.lakecityquietpills.com/photo/multihost/images/87157835991960738722.png (http://www.lakecityquietpills.com/photo/multihost/)

Node #8, DNS errors are down now (although still high), but timeouts are still high. With all 8 nodes running, i got a max of 3.3M down today. :shrug:

http://www.lakecityquietpills.com/photo/multihost/images/28346079378410211368.png (http://www.lakecityquietpills.com/photo/multihost/)

And for comparison, here is the stats for my control node.

http://www.lakecityquietpills.com/photo/multihost/images/91543793501223245341.png (http://www.lakecityquietpills.com/photo/multihost/)

[OldChap]

How does the host allocate resources to the vm? can you control?

What is running on the host while the vm is running?

[hixie]

Something interesting to note, all 8 nodes were being starved of work, when suddenly fresh buckets came in. You can see the 3 peaks on the left hand side, most likely from the archiving.
So CPU load part is definitely solved.

http://www.lakecityquietpills.com/photo/multihost/images/03319175413231073353.png (http://www.lakecityquietpills.com/photo/multihost/)


How does the host allocate resources to the vm? can you control?

What is running on the host while the vm is running?

ESXi allocates resources as you set them, it's even possible to over-commit more resources then you actually have, and dynamically take back resources other VMs have not used. Resource section of ESXi is pretty comprehensive.
The host runs, FreeNas, Linux Mint (MJ12), Untangled (router) and VCenter sever (doesn't use any resources).

[OldChap]

cpu seems not utilised much. What physical NIC is used?

Have you run namebench to find best dns server?

[hixie]

The onboard NIC is a dual port Intel 82576 chipset, i found 2 pcie Intel nics with the same chipset for a good price, and those are on the way to be tested.

I have DNSbench, and my ISP had the top score, then opendns and finally google. Originally i had my ISP dns server as the main one to use, but after a while i got huge dns errors (80+%) so i removed my ISP dns server from dnsmasq's list, dnsmasq now primary goes to opendns before falling back to google. Error rate is much better then it was, but numbers are still relatively high, and much higher then my control.

[hixie]

Update: Seem to have located and solved the problem. I by chance plugged another router into the modem for temporary wifi, and noticed there was plenty of speed and bandwidth available.
So concluded that Untangled must have been causing some sort of limit, so i setup another VM on ESXi to test pfsense on it, all test seems to indicate that untangled was indeed limiting somehow.

However, i seem to have lost connectivity to the nj12 node webserver for some reason. any ideas?


EDIT: Webserver just decided it'll like to start working after 30mins. Let's hope everything goes well!

[OldChap]

:D :up:

I need to test some ecc reg ram that should run in one of my rigs... it is on the qvl

If so, that frees up the udimms currently fitted for another rig to run just MJ. I hope then to see where my limits are on 120/12

[hixie]

I've noticed my scores haven't changed at all. So i've looked at the traffic graphs, pfsense is saying i get on average about 20megabits per seconds, which curiously, if converted to megabytes is 2.5megabytes per second. Which is exactly what i was getting with untangled. Which explains the exact same score? Atleast DNS errors and timeouts have definitely improved.

Back to troubleshooting again.

[OldChap]

At one point I asked about routers on MJ Forum and was told that an ordinary 250MB type should be fine but the reality of the problem at the time was ......

1 node 1750KB/sec
2 nodes 300KB/sec
3 nodes 400KB/sec

....etc I never did get to the bottom of that as I started running just one node and concentrating on WCG

[hixie]

I guess i should start posting on the mj12 forum?