Div Posted June 14, 2016 Share Posted June 14, 2016 Last week was not great obviously so I wanted to explain a bit about what happened and what is going to happen next! What Happened Last Week The forum upgrade itself was a big one. I've let the software fall behind over the past couple of years and so we're now having to make quite a big jump from where we are to the latest version of the forum that we use. I tested this for a few months on a separate server and ran through the process probably about a dozen times in test before deciding to take the plunge and do it for real. The upgrade itself was fine, it went without a hitch. The problem we had was that after the upgrade is completed it starts to run a series of "maintenance tasks" one of which is to rebuild all the posts in a new format for the new software. These tasks run with low priority, they take a long time to complete, but the aim of this is that we were supposed to be able to open the forum whilst they were running. In test I was able to do that without a problem, but on live with a couple of thousand people hitting the forum, it just ground to a halt and everything more or less failed. I took all the users off but the jobs were still running slowly. I knew from testing that they would take around a week to complete which is why I then decided to run the cut down "wasteland" version of the forum for a few days whilst the upgrade ran on a different server. Towards the end of the week we got to the very last maintenance job and I had high hopes that we were through but the last job kept failing and falling over before eventually I just gave up and rolled back to the original server and the original forum and that's where we are today. The cause of the slowness is really all down to the posts table. We have 8.6 million posts in the database, and whilst in commercial database terms that is not a lot, for this software it appears to be too much for it to cope with when running through the upgrade. I've tred throwing shedloads of processing power and memory at it but it just doesn't seem to be able to handle it. I've looked at other forum applications but the big enterprise ones are too expensive to be feasible and the other ones, well they look shit to be honest. Having spent months on this, literally, and having spent a load of money too, I'm not for giving up! There are functional, security, and usability reasons for wanting to upgrade. The new software performs much better on mobile especially and with 60% of our traffic coming via mobile it's important to me that we keep pushing on. What Happens Next Really the only way forward is to prune the forum. We're carrying millions of posts we don't need. By reducing the amount of data in the posts table I think we can improve the speed of the upgrade to the point where it becomes viable. I reckon we can get down from 8 million posts to nearer 2 million without really affecting anything noticeable. To do that I'm going to start a number of prune jobs over the next week or so. Apart from P&B gold which won't be touched the task will be told to delete any topic that has had no posts in it for the last 2 years. Any topic with active posts in the last 2 years will be untouched, and of course Gold will be untouched. At the end of that we'll have a smaller post table, and I will do some more tests, and then we'll schedule the upgrade again. So there you go! Link to comment Share on other sites More sharing options...
killienick Posted June 14, 2016 Share Posted June 14, 2016 Sounds like a plan. I would also suggest that you just bin the politics forum too - it deserves to be purged. Link to comment Share on other sites More sharing options...
Swarley Posted June 14, 2016 Share Posted June 14, 2016 TLDR..... All I'm seeing are excuses..... We're all rooting for you If I were you I'd blitz those word and photo association threads, plus the random images/gifs threads in the Nightshift. That would free up some space? Link to comment Share on other sites More sharing options...
Div Posted June 14, 2016 Author Share Posted June 14, 2016 Sounds like a plan. I would also suggest that you just bin the politics forum too - it deserves to be purged. I think keeping stuff like that in one place is the right thing to do. Means if you aren't interested in politics or you think that the section is shit, then it's easy to avoid. Link to comment Share on other sites More sharing options...
Div Posted June 14, 2016 Author Share Posted June 14, 2016 TLDR..... All I'm seeing are excuses..... We're all rooting for you If I were you I'd blitz those word and photo association threads, plus the random images/gifs threads in the Nightshift. That would free up some space? Space isn't an issue, it's just volume of individual posts that is the issue. I honestly think I could have got away with pruning 5 million posts and nobody would have noticed, but I like to be up front with what I'm doing so everyone can bollock me when it goes wrong. Link to comment Share on other sites More sharing options...
Ned Nederlander Posted June 14, 2016 Share Posted June 14, 2016 Div, it would hardly be the end of the World but I've previously bumped what I believe is the oldest remaining post on the forum and went to do it again just then but it's listed as having been archived and I'm unable to bump it. I've not seen anything listed as archived on P&B before!? Link Link to comment Share on other sites More sharing options...
The Master Posted June 14, 2016 Share Posted June 14, 2016 Div, it would hardly be the end of the World but I've previously bumped what I believe is the oldest remaining post on the forum and went to do it again just then but it's listed as having been archived and I'm unable to bump it. I've not seen anything listed as archived on P&B before!? Link Damnit...there goes my plan to congratulate Div on coming up with a brilliant solution, before proceeding to bump every thread that's >2 years old... Link to comment Share on other sites More sharing options...
Div Posted June 14, 2016 Author Share Posted June 14, 2016 Div, it would hardly be the end of the World but I've previously bumped what I believe is the oldest remaining post on the forum and went to do it again just then but it's listed as having been archived and I'm unable to bump it. I've not seen anything listed as archived on P&B before!? Link Yes mate, archiving the old posts was my original plan after last week but it doesn't really resolve the problem so I've binned archving in favour of pruning. ps; I have queued that particular topic for "unarchiving" so if you want to preserve it you can post in it so it escapes the cull! Link to comment Share on other sites More sharing options...
Ned Nederlander Posted June 14, 2016 Share Posted June 14, 2016 Yes mate, archiving the old posts was my original plan after last week but it doesn't really resolve the problem so I've binned archving in favour of pruning. ps; I have queued that particular topic for "unarchiving" so if you want to preserve it you can post in it so it escapes the cull! Link to comment Share on other sites More sharing options...
Mark Connolly Posted June 14, 2016 Share Posted June 14, 2016 Last week was not great obviously so I wanted to explain a bit about what happened and what is going to happen next! What Happened Last Week The forum upgrade itself was a big one. I've let the software fall behind over the past couple of years and so we're now having to make quite a big jump from where we are to the latest version of the forum that we use. Became a media darling, and forgot where he came from. Fucking sickening. Everything else seemed reasonable though. Link to comment Share on other sites More sharing options...
1320Lichtie Posted June 14, 2016 Share Posted June 14, 2016 Just get rid of the Word Association thread. L23 and Booitsme must've went round in circles hunners of times already anyway. The BRALT as well. Link to comment Share on other sites More sharing options...
Romeo Posted June 14, 2016 Share Posted June 14, 2016 Don't delete any of my posts, they are all quality and deserve to be kept. Bin everyone else's though. Link to comment Share on other sites More sharing options...
Jan Vojáček Posted June 14, 2016 Share Posted June 14, 2016 Just get rid of the Word Association thread. L23 and Booitsme must've went round in circles hunners of times already anyway. The BRALT as well. Came in here to post that. You could clear the 'Things you want to share with P&B' thread too. I use it a fair bit, but I doubt anyone wants to flick back a few pages in it, so it would be no great loss. Look how helpful everyone's being with their suggestions of what threads to clear. I'll need to bring this back down a notch. You could get rid of all the old Rangers' threads because they're dead so won't read them. Link to comment Share on other sites More sharing options...
Swarley Posted June 14, 2016 Share Posted June 14, 2016 Came in here to post that. You could clear the 'Things you want to share with P&B' thread too. I use it a fair bit, but I doubt anyone wants to flick back a few pages in it, so it would be no great loss. Look how helpful everyone's being with their suggestions of what threads to clear. I'll need to bring this back down a notch. You could get rid of all the old Rangers' threads because they're dead so won't read them. That's a fair shout, same goes for RTBC, PTTGOYN and pish folk write on Facebook. Link to comment Share on other sites More sharing options...
Div Posted June 14, 2016 Author Share Posted June 14, 2016 That's a million posts gone already Link to comment Share on other sites More sharing options...
ayrmad Posted June 14, 2016 Share Posted June 14, 2016 ffs, the lying pedantic post tweakers will get away with murder now. Link to comment Share on other sites More sharing options...
WILLIEA Posted June 14, 2016 Share Posted June 14, 2016 That's a million posts gone already Good luck and Godspeed! Link to comment Share on other sites More sharing options...
Johnny van Axeldongen Posted June 14, 2016 Share Posted June 14, 2016 P&B will turn into a digital Marseilles if this doesn't work! Link to comment Share on other sites More sharing options...
ICTChris Posted June 14, 2016 Share Posted June 14, 2016 Huge swathes of P&B history will be gone. Cock teasing bitch, GLOVES, Akabusi sex tales, badgers in the drain. All gone. I'm in favour though. Destroy history. Only look forward. Link to comment Share on other sites More sharing options...
Div Posted June 14, 2016 Author Share Posted June 14, 2016 2.6 million gone now and I've not even gone near Gen Nonsense Link to comment Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.