Sciencemadness Discussion Board
Not logged in [Login - Register]
Go To Bottom

Printable Version  
 Pages:  1  
Author: Subject: call for participation: modernizing SM forum software
Polverone
Now celebrating 14 years of madness
*********




Posts: 3121
Registered: 19-5-2002
Location: The Sunny Pacific Northwest
Member Is Online


[*] posted on 26-1-2012 at 14:24
call for participation: modernizing SM forum software


Changing software


The XMB software that this forum runs on is a little quirky and showing its age. Sciencemadness should eventually migrate to other forum software.

Important forum features:
-Free (I could pay for vBulletin, but then I couldn't prepare working forum backups in virtual machine images)
-Runs under Linux
-Supports user access controls and different moderator roles/powers
-Substantial history of use
-Substantial developer community and ongoing development
-Substantial user base

The option I'm leaning toward is Simple Machines Forum: http://www.simplemachines.org/

If anyone thinks that this is a poor choice or has alternative choices to endorse, please let me know in this thread.

Migrating historical data


There was a tool developed to migrate from XMB 1.9.8 to SMF. See this thread (perhaps skip near the end, it's long): http://www.simplemachines.org/community/index.php?topic=1407...

It doesn't work with XMB 1.9.11, which we are running. There is a user-contributed tool that sort-of/mostly converts XMB 1.9.11, but it still has some defects. I will want conversion to be very close to perfect before I can switch from XMB to some other forum system.

How you can help


I write software in my day job. At the current rate, working by myself in spare time, I will be prepared to do the XMB migration in about 40 years. I need technically-inclined members, especially those with programming and/or database experience, to investigate the XMB to SMF conversion and figure out what additional work needs to be done. Then we can start building tools to do that work. You can download a complete, ready-to-run virtual environment with Linux, XMB, MySQL, PHP, and a sanitized copy of the SM forum data here: https://www.sciencemadness.org/whisper/viewthread.php?tid=12...

It should also be possible to install SMF and the existing conversion tools in this virtualized server image.

Beyond simple data conversion we will need some sort of redirect scheme so that old thread/post URLs get transparently redirected to the corresponding data in the new SMF forum. This could be something as simple as a large .htaccess file derived from the contents of the thread table in the XMB database.

I'm sure that you can identify additional issues that may need to be addressed after a big change like this. Please post in this thread if you are interested and possibly have the time/skills to contribute. I am hoping to bring together members with the most appropriate skills to work as a small team planning the migration and building the tools for it. The conversion tools, at least those that might benefit others, will also be released as free software after they have done their job here. It might benefit another site or two looking to move away from XMB.

[Edited on 1-26-2012 by Polverone]




PGP Key and corresponding e-mail address
View user's profile Visit user's homepage View All Posts By User
quicksilver
Super Moderator
*******




Posts: 1820
Registered: 7-9-2005
Location: Inches from the keyboard....
Member Is Offline

Mood: ~-=SWINGS=-~

[*] posted on 26-1-2012 at 15:14


According to some testing/voting, etc vBulltin is the one people like but there are some issues that would cut the list.
The issue of freeware is a tough one becasue the features mentioned are generally wrapped in a pricy item. It MAY be possible to get an older version of vBullitin without any legal hassles because the company may simply not support whole version changes as many companies do (like Microsoft leaves OS's to rot after several years). These guys may have it...

http://www.oldversion.com/

There is some of the important issues discussed via Wiki:

http://en.wikipedia.org/wiki/Comparison_of_Internet_forum_so...


The WORST is the forum software that hosts the American Diabetes Association discussion Forum (really poorly designed & costly).




View user's profile View All Posts By User
Lambda-Eyde
International Hazard
*****




Posts: 851
Registered: 20-11-2008
Location: Norway
Member Is Offline

Mood: Cleaved

[*] posted on 26-1-2012 at 15:35


There are two forum software solutions I like: InvisionBoard and vBulletin. Neither of them are free, but I believe there's only a one-time cost that could easily be covered by donations/fundraising. I've moderated both Simple Machines and phpBB and didn't care much for any of them. I am also a moderator on both IB and vBulletin boards - the latter is quite pleasant to work with, vBulletin is a dream when it comes to moderation. I'm not familiar with the technical details, so I can't provide any details there.

What I like about XMB is the clean design. I've never seen anything like it, it's just pleasant to look at. vBulletin is also pretty clean, phpBB and SMF (especially the former) are absolutely horrible to look at.




This just in: 95,5 % of the world population lives outside the USA
You should really listen to ABBA
Please drop by our IRC channel: #sciencemadness @ irc.efnet.org
View user's profile View All Posts By User This user has MSN Messenger
Polverone
Now celebrating 14 years of madness
*********




Posts: 3121
Registered: 19-5-2002
Location: The Sunny Pacific Northwest
Member Is Online


[*] posted on 26-1-2012 at 16:38


People already donated enough to buy vBulletin or Invision, but commercial software is just not going to work. Not even if it costs only $5 for a perpetual license. I have to be able to freely redistribute the forum software so I can make and distribute the virtual machine forum backups.

It is possible that if I used an old version of vBulletin, nobody from the company would hassle me about it, but it would also mean being stuck on old software that never improves or gets bug fixes. That's the very situation we are in now that I'm trying to leave.

Further background, if it helps: I don't want to stick with XMB forever because the last version was released in 2009; development seems to be stalled. There are some bugs/quirks that will probably never be fixed, and I would like a forum with better tools to discourage bad behavior without outright banning people.

I want to add a Sciencemadness wiki that shares accounts with the forum. Woelen did a good job setting up a home chemistry wiki but it seems like it won't be used much as long as it requires a separate login from the Sciencemadness forum. I could integrate XMB with MediaWiki, but it would make leaving XMB even more painful in the future. So I want to pick some good free forum software with a more assured future, convert to it, and then do the MediaWiki integration. Based on features, availability of (at least partial) conversion tools, and developer/user community, Simple Machines seemed like the best choice.




PGP Key and corresponding e-mail address
View user's profile Visit user's homepage View All Posts By User
madscientist
Super Moderator
*******




Posts: 962
Registered: 19-5-2002
Location: American Midwest
Member Is Offline

Mood: pyrophoric

[*] posted on 26-1-2012 at 16:49


I agree that Simple Machines is the way to go. I will miss the old look of the site, though - it's the only site that looks the same as it did when I was still just a kid... ahh, nostalgia. Goodbye, SM smilies... :):(:D;):cool::mad::o:P



I weep at the sight of flaming acetic anhydride.
View user's profile View All Posts By User
watson.fawkes
International Hazard
*****




Posts: 2793
Registered: 16-8-2008
Member Is Offline


[*] posted on 27-1-2012 at 06:49


I can assist with the project. I've done plenty of code in my life.

At the very least, we should be able to re-theme an SMF instance to look much like the current board. Certainly the basic color scheme can be moved over. There's a full set of templates available for deep customization.

A public git repository would be useful for collaboration.
View user's profile View All Posts By User
Polverone
Now celebrating 14 years of madness
Thread Topped
6-2-2012 at 17:05
bfesser
Resident Wikipedian
*****




Posts: 2114
Registered: 29-1-2008
Member Is Offline


[*] posted on 12-2-2012 at 08:22


I'd like to offer assistance, if possible. I have previous experience with phpBB and a few other forum frameworks. I used to be able to code PHP, HTML, JSP, CSS, and some other applicable languages. I have familiarity with MySQL. I also have experience with templates and graphic design. (I did freelance webdesign and server administration in my teenage years--something I try to keep secret from my real world acquaintances, lest they nag for tech support.) I'll admit, I haven't done much with code or databases in some years, but I'd be willing to brush up on it.

I concur with watson.fawkes.
Quote: Originally posted by watson.fawkes  
A public git repository would be useful for collaboration.




View user's profile View All Posts By User
garage chemist
chemical wizard
*****




Posts: 1805
Registered: 16-8-2004
Location: Germany
Member Is Offline


[*] posted on 12-2-2012 at 08:52


What I'd really like to see is a forum search engine that lets you choose between displaying threads and single posts. I am sure most would agree with me on this.



www.versuchschemie.de
Das aktivste deutsche Chemieforum!
View user's profile View All Posts By User
quicksilver
Super Moderator
*******




Posts: 1820
Registered: 7-9-2005
Location: Inches from the keyboard....
Member Is Offline

Mood: ~-=SWINGS=-~

[*] posted on 12-2-2012 at 09:06


This is the largest site I could find that has FREE Forum software:

http://thinkofit.com/webconf/forumsoft.htm

There are a great many to choose from and it appears.....that it's up-to-date with most all Forum software commonly distributed.




View user's profile View All Posts By User
zoombafu
National Hazard
****




Posts: 255
Registered: 21-11-2011
Location: U.S.
Member Is Offline

Mood: sciencey

[*] posted on 16-2-2012 at 17:24


Just an idea, but It would be cool to add a wiki to the site. To edit the wiki I think that the member would have to prove their worth in the forum (by submitting quality information). The wiki could serve as the member publications area almost, and it could be very comprehensive. There could a wikiportal for different areas of chemistry (organic, energetics, inorganic, etc), and everything could then be cross referenced. The wiki would also probably help by cleaning up un needed forum posts, because information would be found much easier through a wiki interface than through digging through old forum posts.



View user's profile Visit user's homepage View All Posts By User
plante1999
International Hazard
*****




Posts: 1937
Registered: 27-12-2010
Member Is Offline

Mood: Mad as a hatter

[*] posted on 16-2-2012 at 17:27


Quote: Originally posted by zoombafu  
Just an idea, but It would be cool to add a wiki to the site. To edit the wiki I think that the member would have to prove their worth in the forum (by submitting quality information). The wiki could serve as the member publications area almost, and it could be very comprehensive. There could a wikiportal for different areas of chemistry (organic, energetics, inorganic, etc), and everything could then be cross referenced. The wiki would also probably help by cleaning up un needed forum posts, because information would be found much easier through a wiki interface than through digging through old forum posts.


Great idee! I love the idee of posting good quality synthesis/ extration on wiki. I would be the first to contribute.




I never asked for this.
View user's profile View All Posts By User
watson.fawkes
International Hazard
*****




Posts: 2793
Registered: 16-8-2008
Member Is Offline


[*] posted on 16-2-2012 at 18:00


Quote: Originally posted by zoombafu  
Just an idea, but It would be cool to add a wiki to the site.
So also does our gracious host here. See upthread:
Quote: Originally posted by Polverone  
I want to add a Sciencemadness wiki that shares accounts with the forum.

What's needed first is conversion software to take the old database file in XMB and convert them over. Are you able to help with this?
View user's profile View All Posts By User
AndersHoveland
Hazard to Other Members, due to repeated speculation and posting of untested highly dangerous procedures!
*****




Posts: 1986
Registered: 2-3-2011
Member Is Offline


[*] posted on 16-2-2012 at 18:06


I hope nothing gets lost in the transition. Hopefully everything gets archived, including all the pictures and attached files.
The transition might also be a good time to weed out some of the threads, or specific posts, that are obviously trash. But I would hope whoever is sorting through would err on the side of caution. Quicksilver recently sent a thread to detritus that I thought had some decent posts with some information I did not know before.
View user's profile Visit user's homepage View All Posts By User
watson.fawkes
International Hazard
*****




Posts: 2793
Registered: 16-8-2008
Member Is Offline


[*] posted on 16-2-2012 at 18:19


Quote: Originally posted by AndersHoveland  
I hope nothing gets lost in the transition. Hopefully everything gets archived, including all the pictures and attached files.
The transition might also be a good time to weed out some of the threads, or specific posts, that are obviously trash.
The whole reason to do custom conversion software is to lose nothing. Are you able to contribute?

Insofar as I know, there's no plan to do retroactive editing.
View user's profile View All Posts By User
watson.fawkes
International Hazard
*****




Posts: 2793
Registered: 16-8-2008
Member Is Offline


[*] posted on 1-3-2012 at 04:25
for developers on Windows


For those wishing to get the site backup and use it to test migration code, there's a relevant note for setting up the site backup on Windows.
View user's profile View All Posts By User
froot
National Hazard
****




Posts: 336
Registered: 23-10-2003
Location: South Africa
Member Is Offline

Mood: refluxed

[*] posted on 1-3-2012 at 06:51


Correct me if I'm wrong, but we're talking about database translation so that the new software can place everything in the right place on it's interface right?
Could we do it on this domain in a seperate directory and set up a seperate database totally seperate from the live board if the hosting package allows it?
Installation of the new software should be straight forward as per given instructions. The database side would be where intellectual input is needed where the current database is downloaded, translated and the uploaded to the new one. Testing can be done and corrections made while the site stays live.
I have done this before with phpBB but that was a while ago so it may be possible.

Edit: Maybe research translating the database to another forum software that has the facility to translate it back to the software you're after?

[Edited on 1-3-2012 by froot]




We salute the improvement of the human genome by honoring those who remove themselves from it.
Of necessity, this honor is generally bestowed posthumously. - www.darwinawards.com
View user's profile View All Posts By User
bfesser
Resident Wikipedian
*****




Posts: 2114
Registered: 29-1-2008
Member Is Offline


[*] posted on 11-3-2012 at 08:27


Has any progress on this been made? If so, who is actively participating and how can I join the effort?

As I said earlier, I really would like to contribute. I'm learning more than I ever wanted to about XMB through my symbol project, and I'd like to put that knowledge to use. I will host a shared workspace for this on my web host, if one has not yet been established. Just say the word.




View user's profile View All Posts By User
watson.fawkes
International Hazard
*****




Posts: 2793
Registered: 16-8-2008
Member Is Offline


[*] posted on 11-3-2012 at 12:19


Quote: Originally posted by bfesser  
Has any progress on this been made? If so, who is actively participating and how can I join the effort?

As I said earlier, I really would like to contribute.
There are two immediate steps we need:
1) A git repository. This is just a matter of finding one. I don't see a reason we can't use a free one, but I haven't taken the time to read a bunch of terms-of-service documents.
2) A trial run of the conversion software that Polverone mentioned at the beginning. It won't work completely, but once we're to a point where individuals can contribute incremental progress, we'll be able to proceed much faster.
View user's profile View All Posts By User
bfesser
Resident Wikipedian
*****




Posts: 2114
Registered: 29-1-2008
Member Is Offline


[*] posted on 13-3-2012 at 16:59


Quote: Originally posted by watson.fawkes  
There are two immediate steps we need:
1) A git repository...

I have tried to register with SourceForge.net but never received the registration confirmation email—same issue I had with them years ago. I'm waiting on an email response from their support staff.




View user's profile View All Posts By User
bfesser
Resident Wikipedian
*****




Posts: 2114
Registered: 29-1-2008
Member Is Offline


[*] posted on 14-3-2012 at 19:29


Ok, so are we moving to a Simple Machines forum (SMF)? Personally, I'd like to see us migrate to phpBB. I have many years of experience with phpBB, and wouldn't have to study as much to get started coding the migration tool. It also satisfies all of your requirements listed in the OP.
Quote: Originally posted by Polverone  
Important forum features:
-Free (I could pay for vBulletin, but then I couldn't prepare working forum backups in virtual machine images)
-Runs under Linux
-Supports user access controls and different moderator roles/powers
-Substantial history of use
-Substantial developer community and ongoing development
-Substantial user base

Polverone, ultimately you hold the power here. Will it be SMF or something else? Once you decide, I'm going to start working on it—since no-one else seems to have the initiative. As I mentioned in private U2U, it would be helpful to any of us who already have Linux boxes for you to release non-virtual machine sanitized backups. Just an archive file (.tar.gz is good) with the website/forum files and a sanitized backup of the MySQL database.

I've already started two SourceForge.net projects, one for XMB→SMF and another for XMB→phpBB. I can start another project, if a different forum platform is chosen.



I'm going to take this opportunity to lay out the first requirement of the migration tool. It needs to prune off some of the 133+ pages of non-participating registered users. They comprise more than half the total registered accounts. Nothing against lurkers, of course—I tend to lurk more than participate. The tool will give this as an option, with a way to set it to delete all 0-post accounts or delete them if they haven't been logged on in a variable ammount of time.

[Edited on 3/15/12 by bfesser]




View user's profile View All Posts By User
Polverone
Now celebrating 14 years of madness
*********




Posts: 3121
Registered: 19-5-2002
Location: The Sunny Pacific Northwest
Member Is Online


[*] posted on 15-3-2012 at 10:35


bfesser,

Thanks for taking the initiative. It took me a while to upload the bare db dump and code, but it is available now:

https://www.sciencemadness.org/xmb_sm_test.sql.gz
https://www.sciencemadness.org/smtalk.tgz

I don't have any particular affinity for smf over phpBB, but it looked like the conversion would be more complicated with phpBB. According to someone on the phpBB user forums, writing last year, the current xmb to phpBB converter does not work properly: the user instead had to translate xmb to smf, and then smf to phpBB. If we need to translate to smf anyway, it seemed well enough to me to stop there and not do another conversion to phpBB.

If you have experience with phpBB and prefer it over smf, the phpBB conversion from smf appears to be fine. Using phpBB might allow me to better tap your expertise should it ever need further modifications.




PGP Key and corresponding e-mail address
View user's profile Visit user's homepage View All Posts By User
bfesser
Resident Wikipedian
*****




Posts: 2114
Registered: 29-1-2008
Member Is Offline


[*] posted on 15-3-2012 at 12:26


md5sums?

[downloading]
At least it's not 4.8GB! Thanks for making this available, Polverone.

[digging in]
This is going to be such a pain in the . . .

[Edited on 3/15/12 by bfesser]




View user's profile View All Posts By User
Polverone
Now celebrating 14 years of madness
*********




Posts: 3121
Registered: 19-5-2002
Location: The Sunny Pacific Northwest
Member Is Online


[*] posted on 15-3-2012 at 17:31


I installed SMF 2.0.2 and the XMB to SMF converter found here:

http://download.simplemachines.org/?converters;software=xmb

Then I tried to run it from the command line and ran into a problem.

Quote:

root@desktop:/var/www/smf# php /var/www/smf/convert.php --path_to=/var/www/smf --path_from=/var/www/smtalk --convert_script=xmb_to_smf.sql --db_pass=XXX
Starting Conversion

* Converting members...
.

.

.

Successful.

* Converting categories...
Successful.

* Converting boards...
Successful.

* Converting topics...
.

.

.

.

.

.

.

Duplicate entry '18275' for key 'PRIMARY'


I'm not sure exactly how this comes up. I checked the XMB database and there's no duplicate use of tid 18275.

EDIT: I thought that maybe my database-sanitizing procedure on the public XMB dump somehow produced results which confuse the converter. But it is not so. I tried restoring a full db backup as used on the production forum and running the conversion -- same error.

[Edited on 3-16-2012 by Polverone]




PGP Key and corresponding e-mail address
View user's profile Visit user's homepage View All Posts By User
bfesser
Resident Wikipedian
*****




Posts: 2114
Registered: 29-1-2008
Member Is Offline


[*] posted on 16-3-2012 at 12:39
So, no md5sums then?


Hmm. I'll set up the database backup and SMF on my host and give that converter a go and we'll see if I get the same result.

Part of me also wonders if it has anything to do with the iframes that metalresearcher used in that thread. Perhaps converter.php doesn't play friendly with iframes. I know that phpBB3 no longer allows HTML in posts–in large part due to the historical abuse and exploitation of iframes. They now rely entirely upon BBCode . I've browsed around all the documentation on the SMF site, but they say nothing at all about support for HTML in posts. My bet is, they don't allow it. Maybe the iframe sandbox tags caused an unforseen logic or loop bug in the converter.php. I'll check their 'Community' board soon.

Coincidentally; I haven't received my confirmation email from the 'SimpleMachines.org Community' forum. Same issue I had with SourceForge.net. My email apparently sucks. No spam filters or blocking . . . just unreliability. When I get that, I'll poke around over there to see if anyone's encountered similar issues.

In the meanwhile, I'm sifting through the convert.php file—the comments leave much to be desired. No lie, here's a sample:
Code:
// We found the treasure :P.

Seriously? Why add such a useless comment to your code!? Why not just write your code in Ook! or brainfuck if you want it to be difficult to follow. Some people's children...

[Edited on 7/9/13 by bfesser]




View user's profile View All Posts By User
Polverone
Now celebrating 14 years of madness
*********




Posts: 3121
Registered: 19-5-2002
Location: The Sunny Pacific Northwest
Member Is Online


smile.gif posted on 26-3-2012 at 14:35
Subscript/superscript buttons now available


Thanks go to bfesser, who added new subscript/superscript buttons and supporting code to our creaky old XMB software. You can use them via the buttons above the message composition window or by manually inserting sub and sup bbcode tags.



PGP Key and corresponding e-mail address
View user's profile Visit user's homepage View All Posts By User
 Pages:  1  

  Go To Top