Planned Sitework Tuesday 13th Jan 2009, Starting performance tuning testing Options

[Imhotep]

Greetings.

Starting at 07:00 EST (12:00 GMT) on Tuesday, 13th Jan I'll be doing some tweaking on the server configuration to optimize speeds and page loads. I don't anticipate going more than 2 hours on it. During that period you might notice short intermittant "page cannot be displayed" or "internal server error" pages on the forums. The www site should only be minimally affected. Outages should be no longer than a minute or so as I load the new configuration and test whether it works or not. If it does then I run the benchmarking program against it and if it doesn't make things faster than revert the change. During benchmarking you should not notice any anomolies.

I

[Guest_ohsteve_*]

Peter, great job. is the tweaking done thru hardware or soft ware? Hardware i can understand, software leaves me baffled most times. Oh and (jokingly) what is anomolies, are they molies that don't have any ID? undercover molies? when they turn thier backs on you, do you see molie asses? Sorry couldn't resist.
Steve

[Imhotep]

Peter, great job. is the tweaking done thru hardware or soft ware? Hardware i can understand, software leaves me baffled most times.

Almost missed this part, sorry.

All the tweaking at this point is done through software. The underlying operating system is Linux (a Unix flavor). You can set up how many processes that are running are dedicated to serving web pages (more and things serve faster, less and it's slower, too many and you eat up too much RAM and it gets slower again), optimizing the database (SQL) for how big a RAM chunk it has to work with, how often it flushes changed data out to disk, etc etc etc (again with the too much/too few balance) that kind of thing. It's really just experimenting with each hardware configuration (number of processors, number of cores on those cpu's, how much RAM is in the system, how fast the hard drives are for handling the writes), getting Java accellerators configured and working, etc etc etc. There is a lot of black magic, guesses, and crossed fingers going on when actually doing the tweaks, but once in place you generally don't have to change them very often (if at all). Sometimes when a forum's database gets really large (1GB+) you need to up system RAM by 2-4x the size of the database so that the entire thing can run off memory rather than constant swapping out to disk (the busier the forum is the bigger a problem that becomes) it's not so much the size of the underlying databases, but the number of simultanious online users. More users online at any given time, more random queries to different parts of the databases happen which creates more swapping because the system needs to read something that it doesn't have in memory right now.. but oh, darn, I just wrote that piece to disk and need it so let's write this piece.. shucks, I wanted that... ... ... so you dump as much RAM into the system as you can so it's only ever writing changed data back to the hard drives and running as much as possible out of RAM. This server can handle up to 128GB of RAM on the motherboard, our current database is less than 300MB. I don't think we'll have to worry about this for a while - I think the bandwidth we have available will be a limiting factor long before server capacity or speed.