no more downtime ?

green123

looks like no more downtime between 3 and half 3
about time they sorted this problem
well done admins

Stargal

Wow that's actually pretty cool that there's no more half hour break.

upgrade

admiralofthefleet

i heard it has been switched to 3 pm

green123

yeah
well it was a bit ridiculous anyway
so its about time they sorted it
lets hope they dont have it at a different time

Thrill

But........how will we know when it's time to go to bed?

Holy sh*t, is that the time? I'm hitting the sack.

Well it's screwed up this party
http://www.boards.ie/vbulletin/showthread.php?t=2055036501

Karoma

Moved from AH with an expiring redirect... 3.6!

KTRIC

green123 wrote: »

yeah
well it was a bit ridiculous anyway
so its about time they sorted it
lets hope they dont have it at a different time

It has to occour either way, it doesn't make any difference to the backups when they are done. The mods picked a time when there would be less traffic.

@03

I believe that either the admins forgot to set the backup script, or they decided to do the first couple of backups during day time just in case there are any issues.

Just imagine the aggro if it went down @03:00 and didn't come back!

green123

KTRIC wrote: »

It has to occour either way, it doesn't make any difference to the backups when they are done. The mods picked a time when there would be less traffic.

surely its possible to do backups without bringing the whole site down every single day ?

KTRIC

green123 wrote: »

surely its possible to do backups without bringing the whole site down every single day ?

Simple answer is "no". You can't back up files that are in use.

seamus

Karoma wrote: »

Moved from AH with an expiring redirect... 3.6!

Expiring redirect? So when the redirect expires it zaps back to AH? Cool!

regi

I didn't set the backup script right. Don't worry, you'll get the usual go-to-bed reminder, but I might tweak the time a bit!

green123

everyday again ?
surely it should be possible not to have to do this everyday ?

regi

We do like the idea of daily backups.

Actually, I do plan to build a standby database server that we continually replicate to, but I also like a 100% dump of the DB too.

seamus

green123 wrote: »

everyday again ?
surely it should be possible not to have to do this everyday ?

Absolutely. They could not do it, ever. But then the next time something goes bang, we lose a few hundred thousand posts

The database gets backed up every day so that we have a copy of all of the posts that were made that day. If you don't do it every day, you risk losing more information.

There are a few things that can actuall be done which would offer alternatives;

1. Set up the old server as a mirror. What this does is have the second server watching and copying everything the live server does. A bit like a kid copying his mate's test in school. The great thing about this is that the mirror is always only a few minutes behind the live system. The bad thing about this is that it's not a "backup" solution. It requires regular monitoring and administration to ensure that it's working properly. It's really only for scenarios where you require high-availability and a hot-swappable database server. Anyone using this method on any database, still maintains regular backups apart from the replication cluster.
The other great thing about this is, provided you know the mirror is working, then you can do your dumps from the mirror server, leaving the live server free to go on working.

2. Use differential backups instead of full backups. This kind of scenario usually means that a full backup (all data is backed up) is performed once a week, say on a Sunday. Then every evening after that, only the changed data is backed up. This means that with the exception of the Sunday night backup, each night's backup should be pretty quick. I'm not sure if MySQL provides this though.
The main problem with this is the time it takes to do a restore. If the database dies on Friday afternoon (for example), you need to restore the Sunday night backup, then the monday night one, then the tuesday one, and so on. You'd have to do about seven individual restores to get you back to a working situation. If you do a full backup nightly, then you only need to do two (database & TLog) to get you back to a working state. With admin time at a premium, this would be a pain.

These are only the two most obvious I can see. I'm sure King Dan knows some more funky trickery to perform superfast live backups.

Gordon

seamus wrote: »

Expiring redirect? So when the redirect expires it zaps back to AH? Cool!

Cool, expiring redirects! I think that means that the little redirect shortcut notice on the original forum disappears after some time.

These are only the two most obvious I can see. I'm sure King Dan knows some more funky trickery to perform superfast live backups.

It looks like green123 knows much more about backups than you seamus. Maybe green123 can fill the admins in on how exactly to do the backup, or why exactly a backup isn't needed as his knowledge on this matter seems obscenely informed.

seamus

Gordon wrote: »

Cool, expiring redirects! I think that means that the little redirect shortcut notice on the original forum disappears after some time.

Ah, that makes more sense

I prefer that

green123

Gordon wrote: »

It looks like green123 knows much more about backups than you seamus. Maybe green123 can fill the admins in on how exactly to do the backup, or why exactly a backup isn't needed as his knowledge on this matter seems obscenely informed.

no need to be such a smart ****

i was just saying that surely there must be other ways to do backups without bringing the whole site down everyday

and as seamus has pointed out, there are other ways

Gordon

green123 wrote: »

no need to be such a smart ****

i was just saying that surely there must be other ways to do backups without bringing the whole site down everyday

and as seamus has pointed out, there are other ways

Ah, give us a kiss.
x

daveirl

This post has been deleted.

AlmightyCushion

I presume the super new server will cut down on the time the back up takes. Also what is the big deal? Does the 30 minute backup really inconvenience you that much green123? It's only 30 minutes at 3am. Over on the cuckoos nest they have even made a game out of it.

Sully

30mins is nothing in the middle of the morning, would ya go away outta it. Take a break, or go for a stroll or visit some other site for that 30mins!

CuLT

green123 wrote: »

no need to be such a smart ****

i was just saying that surely there must be other ways to do backups without bringing the whole site down everyday

and as seamus has pointed out, there are other ways

"day" is a mighty strong word there.

RoundyMooney

For me, even as an insomniac, the 3am backup was never an issue, and is certainly infinitely better than the alternative possible loss of data.

Blowfish

It's quite simple, 3am is nap time, before a fresh round of browsing/posting at 3.30.

It's only half an hour green123, and the good news is, the other boards sites don't go down, so you can browse them until it's over.

randylonghorn

daveirl wrote: »

This post has been deleted.

I see where you're coming from, Dave, but tbh I wouldn't even argue that ...

oleras

regi wrote: »

We do like the idea of daily backups.

Actually, I do plan to build a standby database server that we continually replicate to, but I also like a 100% dump of the DB too.

Can the site be made read only while the backup happens ?

oleras

daveirl wrote: »

This post has been deleted.

Would you point out where the OP is demanding anything ?

This is feedback, he was giving feedback on the sites performance, or lack of between 3-3.30am.

daveirl

This post has been deleted.

MrPudding

seamus wrote: »

Absolutely. They could not do it, ever. But then the next time something goes bang, we lose a few hundred thousand posts

The database gets backed up every day so that we have a copy of all of the posts that were made that day. If you don't do it every day, you risk losing more information.

There are a few things that can actuall be done which would offer alternatives;

1. Set up the old server as a mirror. What this does is have the second server watching and copying everything the live server does. A bit like a kid copying his mate's test in school. The great thing about this is that the mirror is always only a few minutes behind the live system. The bad thing about this is that it's not a "backup" solution. It requires regular monitoring and administration to ensure that it's working properly. It's really only for scenarios where you require high-availability and a hot-swappable database server. Anyone using this method on any database, still maintains regular backups apart from the replication cluster.
The other great thing about this is, provided you know the mirror is working, then you can do your dumps from the mirror server, leaving the live server free to go on working.

2. Use differential backups instead of full backups. This kind of scenario usually means that a full backup (all data is backed up) is performed once a week, say on a Sunday. Then every evening after that, only the changed data is backed up. This means that with the exception of the Sunday night backup, each night's backup should be pretty quick. I'm not sure if MySQL provides this though.
The main problem with this is the time it takes to do a restore. If the database dies on Friday afternoon (for example), you need to restore the Sunday night backup, then the monday night one, then the tuesday one, and so on. You'd have to do about seven individual restores to get you back to a working situation. If you do a full backup nightly, then you only need to do two (database & TLog) to get you back to a working state. With admin time at a premium, this would be a pain.

These are only the two most obvious I can see. I'm sure King Dan knows some more funky trickery to perform superfast live backups.

The problem, well a problem, with mirroring and one of the reasons you still need a backup is that depending on the method of mirroring you will mirror corruption as well as data. So if you start corrupting your R1 your R2 can quickly follow.

It might be possible to use a "synthetic" full backup. Start with a full backup and run incrementals as often as you want. Then, every week or every day take the last full and any subsequent incrementals and roll them into a spanky new full backup.

Personally I would try to run all the backups against the mirror as suggested and leave the production server alone.

MrP

Duiske

oleras wrote: »

Can the site be made read only while the backup happens ?

Good question. I noticed this was done for a time during the recent upgrade.

regi

regi wrote: »

I didn't set the backup script right. Don't worry, you'll get the usual go-to-bed reminder, but I might tweak the time a bit!

What time do you plan to set it for, the cuckoos have been freaked out over the weekend

http://boards.ie/vbulletin/showthread.php?t=2055036501

seamus

Duiske_Lad wrote: »

Good question. I noticed this was done for a time during the recent upgrade.

I think this is an option which is set in vBulletin - a box which is ticked.

Theoretically as part of the backup job, you could create a script which ticked this box, then kicked off the backup job, then unticked the box.

But if the box got ticked and then something went "weh", boards.ie is left in read-only mod until an admin has a chance to log in.

At least with the backup job, you know boards is still going to work afterwards.

DeVore

"Surely".... the most dangerous word in IT.

DeV.

stevenmu

seamus wrote: »

But if the box got ticked and then something went "weh", boards.ie is left in read-only mod until an admin has a chance to log in.

At least with the backup job, you know boards is still going to work afterwards.

Wouldn't the alternative to this be something going "weh" and the whole site staying down ? Just guessing but I'd have thought that anything which would prevent the script bringing the site out of read-only after a backup, would also prevent it coming back on line if it was down for a the backup.

Shiminay

Lads and ladies, you're forgetting something *very* important...

The admins and some of the other people involved in the "physical" side of boards (i.e. the hardware setup and hosting all that good stuff) are professionals at this very job. Boards is one of many websites maintained by these professionals and if that level of service is good enough for the people who pay their bloody hard earned wages, then it should certainly be good enough for a community who for the most part thrives on the service they've given us.

There are many many technical users of boards who also specialise in these services and it's the kind of thing where everyone has an idea of what might tweak things here or there, but there isn't always an exact science to the technology behind these things (it's not an area I've done any real work in so I wouldn't even have a clue what to suggest).

Rest assured, we're in the best possible hands - I mean who could worry when people like Asok have their finger on the proverbial big red button... :eek:

seamus

stevenmu wrote: »

Wouldn't the alternative to this be something going "weh" and the whole site staying down ? Just guessing but I'd have thought that anything which would prevent the script bringing the site out of read-only after a backup, would also prevent it coming back on line if it was down for a the backup.

I don't know what the script does at the moment
Making boards read-only would involved firing an SQL query at the database server before the backup, and then firing the opposite query at the database afterwards.
I'm assuming that the current script stops the apache service before the backup, and starts it again after.

It might be just me, but I put more faith in the latter command working 99.999% of the time. I would almost expect the former to fail once every ten times.

It probably is just me.

po0k

Kharn wrote: »

Rest assured, we're in the best possible hands - I mean who could worry when people like Asok have their finger on the proverbial big red button... :eek:

The Children

green123

i haven't noticed any downtime for backups recently.
so what's the story now ?
have you finally came around to my way of thinking ?

Kinetic^

All the mods and admin sat around a giant round table just to discuss how we could make your experience on boards a lot more pleasant. It was down to a mod going around to your house and giving you a massage during this time or stopping the downtime. With a result of 794 to 1 (Terry),it was decided that the scheduled downtime each morning would be a thing of the past. Bravo.

Terry

I give good massages. I just wanted to help.

orestes

I haven't slept since the backup went away

Overheal

seamus wrote: »

1. Set up the old server as a mirror. What this does is have the second server watching and copying everything the live server does. A bit like a kid copying his mate's test in school. The great thing about this is that the mirror is always only a few minutes behind the live system. The bad thing about this is that it's not a "backup" solution. It requires regular monitoring and administration to ensure that it's working properly. It's really only for scenarios where you require high-availability and a hot-swappable database server. Anyone using this method on any database, still maintains regular backups apart from the replication cluster.
The other great thing about this is, provided you know the mirror is working, then you can do your dumps from the mirror server, leaving the live server free to go on working.

2. Use differential backups instead of full backups. This kind of scenario usually means that a full backup (all data is backed up) is performed once a week, say on a Sunday. Then every evening after that, only the changed data is backed up. This means that with the exception of the Sunday night backup, each night's backup should be pretty quick. I'm not sure if MySQL provides this though.
The main problem with this is the time it takes to do a restore. If the database dies on Friday afternoon (for example), you need to restore the Sunday night backup, then the monday night one, then the tuesday one, and so on. You'd have to do about seven individual restores to get you back to a working situation. If you do a full backup nightly, then you only need to do two (database & TLog) to get you back to a working state. With admin time at a premium, this would be a pain.

These are only the two most obvious I can see. I'm sure King Dan knows some more funky trickery to perform superfast live backups.

I'm a server noob, relying on google.... but what about using the old server as a redundant failover node?

http://en.wikipedia.org/wiki/Two-node_cluster

cHaTbOx

orestes wrote: »

I haven't slept since the backup went away

I know it has fooked me up somewhat.

Gordon

zombie thread

DeVore

green123 wrote: »

i haven't noticed any downtime for backups recently.
so what's the story now ?
have you finally came around to my way of thinking ?

how many Green123's does it take to change a light bulb?

One. He holds the light bulb up, and the world revolves around him.

Ross brought a second replicated database server online and did majic. One effect is a removal of the 4am offliness. Its been that way for a several weeks.

DeV.