For (I think) only the third time in its history, Pocket-Monkey was down for more than a day. It went down Saturday morning (UK time) and it came back up just a couple of minutes ago; a total of (roughly) 61 hours. I'm very sorry for the interruption to service!
So what was wrong? Well, the actual problem wasn't all that big. There was an issue with the database server for which I was already testing a potential solution over on the test site, and before I could deploy that solution to the live site the database server went down. It was easy enough to fix, and then I could deploy the update that I had in the wings ready to deploy to work on the speed issues (and deal with the inevitable issues arising because of just how much data the live site has at this point), but unfortunately it was bad timing: Saturday morning when the site went down, I was running out the door to do critical weekend work for a client that we've been building up to all year, and being even an hour late was Not An Option. Since then, I've barely had time to sleep. But I have, now, been able to spend the time to fix the server, deploy the improvement, convert the necessary data, check that all is working, and bring the site back up. And so we're back!
I'll be keeping a close eye on the database server. I have plans in place to improve how we use our servers over the next couple of months, and at the same time I'm working with our hardware vendor to beef up the hardware as well. It's looking like Pocket-Monkey's entering a bit of a growth phase; we fell at the first jump, but I think we'll be able to handle it going forward. Still a couple of bumps to come, I think, in the next couple of weeks, but we're headed the right way.
Timeouts will be off for the next couple of days, and when they come back on all tourney grace time will be reinstated, etc.
On the speed issues: Obviously I'll be watching that pretty closely, too. The improvement I deployed should, in theory, improve the speed of the site -- but results like that on testing servers and such are one thing, the live site with live users is something else entirely. :-)
Thanks, and again, sorry for the outage. -- T.J. Crowder First Primate Pocket-Monkey.com
Thanks so much for your hard work! I hope you catch up on your rest. A couple of days of outage are no problem when you look at the quality of service here!
Re: Big Site Outage
Message #3118 Replies: 0
posted by pint of beer (john shields) on 07/24 at 17:19
AAAAAAAAAAAAAAAGGGGGGGGGGHHHHHHHHHH, just kept my sanity, welcome back TJ, i have had to talk to my family since saturday LOL. lets hope it is fixed for good with all the new people swelling the ranks of members for some of us older members
its been very frustrating not being able to get on this site. now that i understand, and believe me when i say, i know nothing about computers, so i cannot relate to what you are dealing with. but i do know when something goes wrong with something i do know about, i give 120% of my time to fix and make sure that is corrected and it is perfect. so i can relate in that aspect. i give you all my respect. this site is the better site of all three sites i play on. thanks so much.
Re: Big Site Outage
Message #3122 Replies: 0
posted by RedBaron (Will C. Crowder) on 07/24 at 17:40
Great news, TJ, thanks!! With luck, the new deployment will deal with the issues. Man, what timing...Now follow Goofball's advice and get some sleep! :-)
Good work TJ... and like many of the others, I too was suffering pocket monkey withdrawel.... My family found out that I do really hear them (sometimes)!! The hardest part for me was the hold on the tournaments... I am doing better than I expected in one of them and am anxious to find out the outcome of round one..... Now you go get some rest and we will catch you on the flip side... ricki
No problem T.J., we'll support you anyhow. You have a great site running and we all be helping to get you in new trouble with it by getting more and more players over here :P
Hope the client is satisfied, though when you put as much effort in that job as you put in this site, we can be sure he is.
Well done T.J. You deserve a medal. Gave my wife an opportunity to remind me it's our Ruby on the 30th and her 60th in August. Drat, off to the shops I suppose!!!!
Thank you T.J.!!!! Now really you must eat and sleep or you will be no good for anybody let alone yourself! :-))
It was especially nice to be able to read updates on what you were expecting/hoping that the site would be back up and I really appreciated that!! It brought back horrible memories of another site I play at and their messages then were not updated so it was left upto your own imagination and I unfortunately have a very creative one! LOL
Great work getting the server back up. Thanks for all the hard work you put in.
Re: Big Site Outage
Message #3135 Replies: 1
posted by jinjas (Virginia Houseman) on 07/24 at 20:11
Great to have you back TJ.. was 'almost' lost without this site.. must have checked dozens of times in anticipation.. thanks so much for all your hard work..
You got a great site, so missing a couple days of playing to make our playing better, is very little to give. Thanks for all the hard work, take it easy now that most of it is over, have a drink on us and things should straighten out with time!
posted by thorin4th (Jon Stuntz) on 07/24 at 22:15
Gee, I'm glad you're back! I was so desperate I almost started some new games at IYT. Now that I've found PM, the only game I play at IYT is the Scrabble-like game. Great job as always. Thanks.
I just want to be one of many who says THANK YOU for all you did here! Sure i missed the site alot But i know it isn't easy running a site and keeping everything as it must be So with this said , you shouldn't apologize for a JOB WELL DONE! Your site is one of the VERY BEST i have had the pleasure of playing on! THANK YOU T.J. YOU ARE OUR POCKET MONKEY HERO! :-) Most Sincerely, Rebee
thanks for all the work you are doing to keep the site up and running. i'm sure us players weren't the only ones sufferring from insanity with the site down. take a breather TJ and get some rest. :-)
Thanks for all you do TJ. This site is my favourite of the four I play at, and I suffered terrible withdrawal symptoms while it was down. Keep up the good work.
We missed the games but you did a good job getting back on as soon as possible. I believe we all realize there IS life out there that sometimes requires sacrifice - yours and ours. You 'done good'.
PARTICULARY TJ, THANKS FOR THE GREAT SITE THAT YOU MAINTAIN, it's a pleasure playing here, no fighting, no cussing, no putting people down like a site I just left. So you keep your site just the way it is and I love it!
CONGRATULATIONS on havign such a great site!!!!!!!!!!!!!!
Wow. Just, wow. Thank you all for the incredibly positive comments! It's really terrific to know that people are enjoying playing here.
As for this being a friendly site, I thank you for that. You folks set the tone, and you keep it really positive and friendly. And that's what I want Pocket-Monkey to be -- a friendly little place to hang out and play games. So far, I think that's what it is, and that's down to the community here. And so again, thank you all for making it so good to hang out here! -- T.J. Crowder First Primate Pocket-Monkey.com
Forum
software by
Crowder Software Pocket-Monkey and the Pocket-Monkey logo are trademarks of T.J. Crowder and Jock Murphy. All other trademarks are the property of their respective owners.
