We're doing some upgrading and maintenance today

[AZhitman]

...so if you see an occasional outage, don't panic.

We're (once again) splitting out and expanding. Last night and today, we've been separating the Apache and MySQL traffic, moving the MySQL database onto its own server, and migrating data, which takes away from the existing server's resources.

Hang tight - These things just mean we're growing quicker than planned - Good work!

Thanks all.

Greg, Bex and the Management.

[Dattebayo]

Whatever, there is only Gen Chat. The rest is inconsequential.

[AppleBonker]

Thanks for the update!

[Zydeco]

Whatever, there is only Gen Chat. The rest is inconsequential.

True story.

[93coupe]

I never understood "hang tight." I think that if things were tight, they wouldn't be hanging, you know?

Anyway, I'm excited for newness. However, I do severely miss my NICOnostalgic.

[AZhitman]

I never understood "hang tight." I think that if things were tight, they wouldn't be hanging, you know?

It means "to persevere tenaciously".

Hang tough, hang tight, hang loose... all derivations.

[AppleBonker]

I never understood "hang tight." I think that if things were tight, they wouldn't be hanging, you know?

What about hanging, but the boxer-briefs are holding it tight against your leg?

[93coupe]

Then I wouldn't truly consider that a hanging. I would think that is more of a press.

[PEZi]

assbabies!

[sx moneypit]

Thanks Greg and crew!Keep up the good work!

[Xdisaster240sX]

I like...zombies

[breadbox]

Upgrades! Nico is getting a mega man arm... or a Metroid arm, for all you fems.

[Xdisaster240sX]

we need some bling

[heliochrome85]

i just came over here to say how much h0m0 there is in this thread.

i approve.
carry on.

[Jesda]

I never understood "hang tight." I think that if things were tight, they wouldn't be hanging, you know?

It means "to persevere tenaciously".

Hang tough, hang tight, hang loose... all derivations.

I like to "hang brain"

[Xdisaster240sX]

you named it Brain?

[Jesda]

And his protruding neighbor is, sadly, named "Pinky"

[nissangirl74]

And his protruding neighbor is, sadly, named "Pinky"

[Papi Chulo]

And his protruding neighbor is, sadly, named "Pinky"

[PoorManQ45]

I replied on Bex facebook status about this.

Have you guys looked into virtualizing the server that you already have? If I recall correctly you said it had a i7 920, 4gb of memory, and some random harddrive...

This system should be powerful enough to handle a massive database. What are you running for a raid controller? If not a hardware controller, have you looked into one? They can be had pretty cheaply and will offload a huge amount of processing power from the CPU.

Also, setting up a pair of virtualized servers will allow you to allocate resources as needed for each database server.

[AZhitman]

In one hell of an odd coincidence, the servers that host all our ad banners, served via OpenX, have taken a colossal dump:

http://www.dreamhoststatus.com/2010/10/ ... customers/

When it rains....

[AZhitman]

I replied on Bex facebook status about this.

Have you guys looked into virtualizing the server that you already have? If I recall correctly you said it had a i7 920, 4gb of memory, and some random harddrive...

This system should be powerful enough to handle a massive database. What are you running for a raid controller? If not a hardware controller, have you looked into one? They can be had pretty cheaply and will offload a huge amount of processing power from the CPU.

Also, setting up a pair of virtualized servers will allow you to allocate resources as needed for each database server.

Let me see, how to say this nicely...

You don't recall correctly, because we're running 12GB of RAM - However, the size of the database doesn't matter, it's the number of people connecting and the amount of resources those connections take that are the issue, especially if they hang. With mysql and apache sharing memory, there can be an issue with # of open connections. Certain functions tax the dbase more than others, and we were occasionally exceeding the MaxClients setting.

Virtualization sucks donkey d!ck and has its own issues. It's not an option for us for several reasons... We're working with what have have to work with, which is a great setup, it's just a matter of getting things tuned to handle what amounts to thousands of people/bots connecting at the same time sometimes.

The work we've been doing for the past 24 hours involves splitting out onto two* machines - moving Apache to a somewhat smaller server, and keeping MySQL on the bad boy, which will stay at 12GB RAM, dual 1.5Tb HD, and have capacity for 5,000GB/month traffic (not that this will matter, since it's just the dbase).

* Actually, three. Ads and images serve from a totally different server elsewhere.

[AZhitman]

OK, how's it looking for everyone now?

[PoorManQ45]

Let me see, how to say this nicely...

You don't recall correctly, because we're running 12GB of RAM - However, the size of the database doesn't matter, it's the number of people connecting and the amount of resources those connections take that are the issue, especially if they hang. With mysql and apache sharing memory, there can be an issue with # of open connections. Certain functions tax the dbase more than others, and we were occasionally exceeding the MaxClients setting.

Virtualization sucks donkey d!ck and has its own issues. It's not an option for us for several reasons... We're working with what have have to work with, which is a great setup, it's just a matter of getting things tuned to handle what amounts to thousands of people/bots connecting at the same time sometimes.

The work we've been doing for the past 24 hours involves splitting out onto two* machines - moving Apache to a somewhat smaller server, and keeping MySQL on the bad boy, which will stay at 12GB RAM, dual 1.5Tb HD, and have capacity for 5,000GB/month traffic (not that this will matter, since it's just the dbase).

* Actually, three. Ads and images serve from a totally different server elsewhere.

Hold up a minute, you might want to go back and review you information a bit there. Virtualization, with proper equipment, can reduce resource interference between the two databases and is usually cheaper then running separate servers.

There will be no "sharing of memory". Each database will be given its own chunk of resources and that's it. They will not affect each other.

I believe part of your issue is the harddrives you're using. I haven't seen any 1.5TB SAS drives, so that indicates that you're using consumer based drives in this server. With the amount of I/O that you're talking you need to step up to SAS drives. 10k would work, 15k being even better. The controllers can handle more throughput, the seek time is lower, the I/O per second is higher and the longevity is actually higher. Hopefully, at the very least, the two databases were on seperate drives.

I properly configured virtualized server would provide each server its own processing cores, memory, and harddrive array. Doing this will prevent either database from effecting the other.

[admin]

Hold up a minute, you might want to go back and review you information a bit there. Virtualization, with proper equipment, can reduce resource interference between the two databases and is usually cheaper then running separate servers.

There will be no "sharing of memory". Each database will be given its own chunk of resources and that's it. They will not affect each other.

I believe part of your issue is the harddrives you're using. I haven't seen any 1.5TB SAS drives, so that indicates that you're using consumer based drives in this server. With the amount of I/O that you're talking you need to step up to SAS drives. 10k would work, 15k being even better. The controllers can handle more throughput, the seek time is lower, the I/O per second is higher and the longevity is actually higher. Hopefully, at the very least, the two databases were on seperate drives.

I properly configured virtualized server would provide each server its own processing cores, memory, and harddrive array. Doing this will prevent either database from effecting the other.

Just a word of advice...know what the f*** you're talking about BEFORE you spout off

Virtualization was not going to solve this issue. The hard drive I/O is not the issue. I don't know where the f*** you came up with "two databases" 'cuz there's only one for this site. The problem is purely an issue of hitting the maximum number of connections available for Apache AND MYSQL. Part of this is due to the type of table being used for the posts on the forums. We have been testing different ideas to make that work better too, but there's a trade off. Either we read from the tables quickly or we write to the tables efficiently, but neither MyISAM and INNODB seems to have been a perfect solution. Moving to 2 servers was the next logical step. If we played with virtualization, it would just f*** us even quicker because Apache alone has been needing 8-10GB of RAM to keep things going. With all the various settings and mods we've added to the forums to keep people happy, it's going to take some resources like that. This site gets more traffic than your mom's bed. As many as 200k page views a day across the server starts pushing the limits of a single server regardless of the setup. So chill man...obviously whatever servers you run for whatever you do work fine with virtualization. I've run hundreds of sites on dozens of servers with dozens of different configurations (including using virtualization which caused issues of its own, so it's not some magic bullet).

[nissangirl74]

I have been informed that some people's posts aren't showing up. If this happens to you, let me know what thread you posted in so I can talk with the "guys-that-are- smarter-than-me".

[PoorManQ45]

Virtualization was not going to solve this issue. The hard drive I/O is not the issue. I don't know where the f*** you came up with "two databases" 'cuz there's only one for this site.

Sorry, for some reason I was referring to Apache as a database. My fubar...

Would you care to explain why virtualization wouldn't work. It is the same EXACT thing as having two systems. I'm used to ESX and Hyper-V for virtualizing servers. When working in this envoironment you are dedicating exactly what you need for each system. So, I'm not sure why you're saying this wouldn't resolve the issue.

I'm used to dealing with systems with multiple 5xxx and 7xxx series processors though. So maybe that's where I'm missing the mark...

Your hardware should be able to handle this site just fine.

And yes, speeding up the Harddrive I/O would greatly improve site performance. I am a fan of offloading as many processing cycles from the CPU as possible. A hardware controller does this nicely.

Is the only issue you're having with the number of connections? Are you actually maxing out any of the hardware?

[RCA]

Just a word of advice...know what the f*** you're talking about BEFORE you spout off

Virtualization was not going to solve this issue. The hard drive I/O is not the issue. I don't know where the f*** you came up with "two databases" 'cuz there's only one for this site. The problem is purely an issue of hitting the maximum number of connections available for Apache AND MYSQL. Part of this is due to the type of table being used for the posts on the forums. We have been testing different ideas to make that work better too, but there's a trade off. Either we read from the tables quickly or we write to the tables efficiently, but neither MyISAM and INNODB seems to have been a perfect solution. Moving to 2 servers was the next logical step. If we played with virtualization, it would just f*** us even quicker because Apache alone has been needing 8-10GB of RAM to keep things going. With all the various settings and mods we've added to the forums to keep people happy, it's going to take some resources like that. This site gets more traffic than your mom's bed. As many as 200k page views a day across the server starts pushing the limits of a single server regardless of the setup. So chill man...obviously whatever servers you run for whatever you do work fine with virtualization. I've run hundreds of sites on dozens of servers with dozens of different configurations (including using virtualization which caused issues of its own, so it's not some magic bullet).

Nice try Greg...

[admin]

We're constantly maxing out the RAM (12GB...which according to the datacenter is the max for that motherboard) with just running Apache and random other processes, not counting MySQL. If we have several people use egosearch or edit posts at the same time, tables get locked because of how the database is set up, which creates a delay since MySQL gets next to no resources, then it starts using swap and things slow down, processes build up, connections max out and then the site dies. To some extent, the issue is with how phpbb is set up, but since we're basically doing this for free, it's kind of hard to get motivated to rewrite some of the core of phpbb to make it scale properly.

[AZhitman]

Nice try Greg...

Pardon?

Actually, I see that Brien has met our webd00d... I don't have the heart to tell him that "admin" is a well-respected and highly sought-after (and well-compensated) expert on SEO, e-commerce and website construction.

I'm actually a bit surprised that he'd take the time to indulge our resident know-it-all ni discussion - Then again, we're lucky to have his help.

RCA, I'm flattered that you'd think I posted that, but I'm not that smart. If I were, I'd be pulling down a 7-figure salary like him.