Sciencemadness Discussion Board » Non-chemistry » Forum Matters » call for participation: modernizing SM forum software Select A Forum Fundamentals   » Chemistry in General   » Organic Chemistry   » Reagents and Apparatus Acquisition   » Beginnings   » Responsible Practices   » Miscellaneous   » The Wiki Special topics   » Technochemistry   » Energetic Materials   » Biochemistry   » Radiochemistry   » Computational Models and Techniques   » Prepublication Non-chemistry   » Forum Matters   » Legal and Societal Issues

Pages:  1    3
Author: Subject: call for participation: modernizing SM forum software
Polverone
Now celebrating 18 years of madness

Posts: 3164
Registered: 19-5-2002
Location: The Sunny Pacific Northwest
Member Is Offline

Mood: Waiting for spring

call for participation: modernizing SM forum software

<h3>Changing software</h3>
The XMB software that this forum runs on is a little quirky and showing its age. Sciencemadness should eventually migrate to other forum software.

Important forum features:
-Free (I could pay for vBulletin, but then I couldn't prepare working forum backups in virtual machine images)
-Runs under Linux
-Supports user access controls and different moderator roles/powers
-Substantial history of use
-Substantial developer community and ongoing development
-Substantial user base

The option I'm leaning toward is Simple Machines Forum: http://www.simplemachines.org/

If anyone thinks that this is a poor choice or has alternative choices to endorse, please let me know in this thread.

<h3>Migrating historical data</h3>
There was a tool developed to migrate from XMB 1.9.8 to SMF. See this thread (perhaps skip near the end, it's long): http://www.simplemachines.org/community/index.php?topic=1407...

It doesn't work with XMB 1.9.11, which we are running. There is a user-contributed tool that sort-of/mostly converts XMB 1.9.11, but it still has some defects. I will want conversion to be very close to perfect before I can switch from XMB to some other forum system.

<h3>How you can help</h3>
I write software in my day job. At the current rate, working by myself in spare time, I will be prepared to do the XMB migration in about 40 years. I need technically-inclined members, especially those with programming and/or database experience, to investigate the XMB to SMF conversion and figure out what additional work needs to be done. Then we can start building tools to do that work. You can download a complete, ready-to-run virtual environment with Linux, XMB, MySQL, PHP, and a sanitized copy of the SM forum data here: https://www.sciencemadness.org/whisper/viewthread.php?tid=12...

It should also be possible to install SMF and the existing conversion tools in this virtualized server image.

Beyond simple data conversion we will need some sort of redirect scheme so that old thread/post URLs get transparently redirected to the corresponding data in the new SMF forum. This could be something as simple as a large .htaccess file derived from the contents of the thread table in the XMB database.

I'm sure that you can identify additional issues that may need to be addressed after a big change like this. Please post in this thread if you are interested and possibly have the time/skills to contribute. I am hoping to bring together members with the most appropriate skills to work as a small team planning the migration and building the tools for it. The conversion tools, at least those that might benefit others, will also be released as free software after they have done their job here. It might benefit another site or two looking to move away from XMB.

[Edited on 1-26-2012 by Polverone]

PGP Key and corresponding e-mail address
quicksilver
International Hazard

Posts: 1820
Registered: 7-9-2005
Location: Inches from the keyboard....
Member Is Offline

Mood: ~-=SWINGS=-~

According to some testing/voting, etc vBulltin is the one people like but there are some issues that would cut the list.
The issue of freeware is a tough one becasue the features mentioned are generally wrapped in a pricy item. It MAY be possible to get an older version of vBullitin without any legal hassles because the company may simply not support whole version changes as many companies do (like Microsoft leaves OS's to rot after several years). These guys may have it...

http://www.oldversion.com/

There is some of the important issues discussed via Wiki:

http://en.wikipedia.org/wiki/Comparison_of_Internet_forum_so...

The WORST is the forum software that hosts the American Diabetes Association discussion Forum (really poorly designed & costly).

Lambda-Eyde
International Hazard

Posts: 856
Registered: 20-11-2008
Location: Norway
Member Is Offline

Mood: Cleaved

There are two forum software solutions I like: InvisionBoard and vBulletin. Neither of them are free, but I believe there's only a one-time cost that could easily be covered by donations/fundraising. I've moderated both Simple Machines and phpBB and didn't care much for any of them. I am also a moderator on both IB and vBulletin boards - the latter is quite pleasant to work with, vBulletin is a dream when it comes to moderation. I'm not familiar with the technical details, so I can't provide any details there.

What I like about XMB is the clean design. I've never seen anything like it, it's just pleasant to look at. vBulletin is also pretty clean, phpBB and SMF (especially the former) are absolutely horrible to look at.

This just in: 95,5 % of the world population lives outside the USA
You should really listen to ABBA
Polverone
Now celebrating 18 years of madness

Posts: 3164
Registered: 19-5-2002
Location: The Sunny Pacific Northwest
Member Is Offline

Mood: Waiting for spring

People already donated enough to buy vBulletin or Invision, but commercial software is just not going to work. Not even if it costs only \$5 for a perpetual license. I have to be able to freely redistribute the forum software so I can make and distribute the virtual machine forum backups.

It is possible that if I used an old version of vBulletin, nobody from the company would hassle me about it, but it would also mean being stuck on old software that never improves or gets bug fixes. That's the very situation we are in now that I'm trying to leave.

Further background, if it helps: I don't want to stick with XMB forever because the last version was released in 2009; development seems to be stalled. There are some bugs/quirks that will probably never be fixed, and I would like a forum with better tools to discourage bad behavior without outright banning people.

I want to add a Sciencemadness wiki that shares accounts with the forum. Woelen did a good job setting up a home chemistry wiki but it seems like it won't be used much as long as it requires a separate login from the Sciencemadness forum. I could integrate XMB with MediaWiki, but it would make leaving XMB even more painful in the future. So I want to pick some good free forum software with a more assured future, convert to it, and then do the MediaWiki integration. Based on features, availability of (at least partial) conversion tools, and developer/user community, Simple Machines seemed like the best choice.

PGP Key and corresponding e-mail address
International Hazard

Posts: 962
Registered: 19-5-2002
Location: American Midwest
Member Is Offline

Mood: pyrophoric

I agree that Simple Machines is the way to go. I will miss the old look of the site, though - it's the only site that looks the same as it did when I was still just a kid... ahh, nostalgia. Goodbye, SM smilies...

I weep at the sight of flaming acetic anhydride.
watson.fawkes
International Hazard

Posts: 2793
Registered: 16-8-2008
Member Is Offline

Mood: No Mood

I can assist with the project. I've done plenty of code in my life.

At the very least, we should be able to re-theme an SMF instance to look much like the current board. Certainly the basic color scheme can be moved over. There's a full set of templates available for deep customization.

A public git repository would be useful for collaboration.
Polverone
Now celebrating 18 years of madness
6-2-2012 at 17:05
bfesser
Resident Wikipedian

Posts: 2114
Registered: 29-1-2008
Member Is Offline

Mood: No Mood

I'd like to offer assistance, if possible. I have previous experience with phpBB and a few other forum frameworks. I used to be able to code PHP, HTML, JSP, CSS, and some other applicable languages. I have familiarity with MySQL. I also have experience with templates and graphic design. (I did freelance webdesign and server administration in my teenage years--something I try to keep secret from my real world acquaintances, lest they nag for tech support.) I'll admit, I haven't done much with code or databases in some years, but I'd be willing to brush up on it.

I concur with watson.fawkes.
 Quote: Originally posted by watson.fawkes A public git repository would be useful for collaboration.

garage chemist
chemical wizard

Posts: 1803
Registered: 16-8-2004
Location: Germany
Member Is Offline

Mood: No Mood

What I'd really like to see is a forum search engine that lets you choose between displaying threads and single posts. I am sure most would agree with me on this.

www.versuchschemie.de
Das aktivste deutsche Chemieforum!
quicksilver
International Hazard

Posts: 1820
Registered: 7-9-2005
Location: Inches from the keyboard....
Member Is Offline

Mood: ~-=SWINGS=-~

This is the largest site I could find that has FREE Forum software:

http://thinkofit.com/webconf/forumsoft.htm

There are a great many to choose from and it appears.....that it's up-to-date with most all Forum software commonly distributed.

zoombafu
National Hazard

Posts: 255
Registered: 21-11-2011
Location: U.S.
Member Is Offline

Mood: sciencey

Just an idea, but It would be cool to add a wiki to the site. To edit the wiki I think that the member would have to prove their worth in the forum (by submitting quality information). The wiki could serve as the member publications area almost, and it could be very comprehensive. There could a wikiportal for different areas of chemistry (organic, energetics, inorganic, etc), and everything could then be cross referenced. The wiki would also probably help by cleaning up un needed forum posts, because information would be found much easier through a wiki interface than through digging through old forum posts.

plante1999
International Hazard

Posts: 1937
Registered: 27-12-2010
Member Is Offline

 Quote: Originally posted by zoombafu Just an idea, but It would be cool to add a wiki to the site. To edit the wiki I think that the member would have to prove their worth in the forum (by submitting quality information). The wiki could serve as the member publications area almost, and it could be very comprehensive. There could a wikiportal for different areas of chemistry (organic, energetics, inorganic, etc), and everything could then be cross referenced. The wiki would also probably help by cleaning up un needed forum posts, because information would be found much easier through a wiki interface than through digging through old forum posts.

Great idee! I love the idee of posting good quality synthesis/ extration on wiki. I would be the first to contribute.

watson.fawkes
International Hazard

Posts: 2793
Registered: 16-8-2008
Member Is Offline

Mood: No Mood

 Quote: Originally posted by zoombafu Just an idea, but It would be cool to add a wiki to the site.
So also does our gracious host here. See upthread:
 Quote: Originally posted by Polverone I want to add a Sciencemadness wiki that shares accounts with the forum.

What's needed first is conversion software to take the old database file in XMB and convert them over. Are you able to help with this?
AndersHoveland
Hazard to Other Members, due to repeated speculation and posting of untested highly dangerous procedures!

Posts: 1986
Registered: 2-3-2011
Member Is Offline

Mood: No Mood

I hope nothing gets lost in the transition. Hopefully everything gets archived, including all the pictures and attached files.
The transition might also be a good time to weed out some of the threads, or specific posts, that are obviously trash. But I would hope whoever is sorting through would err on the side of caution. Quicksilver recently sent a thread to detritus that I thought had some decent posts with some information I did not know before.
watson.fawkes
International Hazard

Posts: 2793
Registered: 16-8-2008
Member Is Offline

Mood: No Mood

 Quote: Originally posted by AndersHoveland I hope nothing gets lost in the transition. Hopefully everything gets archived, including all the pictures and attached files. The transition might also be a good time to weed out some of the threads, or specific posts, that are obviously trash.
The whole reason to do custom conversion software is to lose nothing. Are you able to contribute?

Insofar as I know, there's no plan to do retroactive editing.
watson.fawkes
International Hazard

Posts: 2793
Registered: 16-8-2008
Member Is Offline

Mood: No Mood

for developers on Windows

For those wishing to get the site backup and use it to test migration code, there's a relevant note for setting up the site backup on Windows.
froot
National Hazard

Posts: 347
Registered: 23-10-2003
Location: South Africa
Member Is Offline

Mood: refluxed

Correct me if I'm wrong, but we're talking about database translation so that the new software can place everything in the right place on it's interface right?
Could we do it on this domain in a seperate directory and set up a seperate database totally seperate from the live board if the hosting package allows it?
Installation of the new software should be straight forward as per given instructions. The database side would be where intellectual input is needed where the current database is downloaded, translated and the uploaded to the new one. Testing can be done and corrections made while the site stays live.
I have done this before with phpBB but that was a while ago so it may be possible.

Edit: Maybe research translating the database to another forum software that has the facility to translate it back to the software you're after?

[Edited on 1-3-2012 by froot]

We salute the improvement of the human genome by honoring those who remove themselves from it.
Of necessity, this honor is generally bestowed posthumously. - www.darwinawards.com
bfesser
Resident Wikipedian

Posts: 2114
Registered: 29-1-2008
Member Is Offline

Mood: No Mood

Has any progress on this been made? If so, who is actively participating and how can I join the effort?

As I said earlier, I really would like to contribute. I'm learning more than I ever wanted to about <a href="http://www.xmbforum.com/" target="_blank">XMB</a> through my <a href="viewthread.php?tid=19198">symbol project</a>, and I'd like to put that knowledge to use. I will host a shared workspace for this on my web host, if one has not yet been established. Just say the word.

watson.fawkes
International Hazard

Posts: 2793
Registered: 16-8-2008
Member Is Offline

Mood: No Mood

 Quote: Originally posted by bfesser Has any progress on this been made? If so, who is actively participating and how can I join the effort? As I said earlier, I really would like to contribute.
There are two immediate steps we need:
1) A git repository. This is just a matter of finding one. I don't see a reason we can't use a free one, but I haven't taken the time to read a bunch of terms-of-service documents.
2) A trial run of the conversion software that Polverone mentioned at the beginning. It won't work completely, but once we're to a point where individuals can contribute incremental progress, we'll be able to proceed much faster.
bfesser
Resident Wikipedian

Posts: 2114
Registered: 29-1-2008
Member Is Offline

Mood: No Mood

 Quote: Originally posted by watson.fawkes There are two immediate steps we need: 1) A git repository...

I have tried to register with <a href="https://sourceforge.net" target="_blank">SourceForge.net</a> but never received the registration confirmation email&mdash;same issue I had with them years ago. I'm waiting on an email response from their support staff.

bfesser
Resident Wikipedian

Posts: 2114
Registered: 29-1-2008
Member Is Offline

Mood: No Mood

Ok, so are we moving to a <a href="http://www.simplemachines.org/" target="_blank">Simple Machines</a> forum (SMF)? Personally, I'd like to see us migrate to <a href="http://www.phpbb.com/about/" target="_blank">phpBB</a>. I have many years of experience with phpBB, and wouldn't have to study as much to get started coding the migration tool. It also satisfies all of your requirements listed in the OP.
 Quote: Originally posted by Polverone Important forum features: -Free (I could pay for vBulletin, but then I couldn't prepare working forum backups in virtual machine images) -Runs under Linux -Supports user access controls and different moderator roles/powers -Substantial history of use -Substantial developer community and ongoing development -Substantial user base

Polverone, ultimately you hold the power here. Will it be SMF or something else? Once you decide, I'm going to start working on it&mdash;since no-one else seems to have the initiative. As I mentioned in private U2U, it would be helpful to any of us who already have Linux boxes for you to release non-virtual machine sanitized backups. Just an archive file (.tar.gz is good) with the website/forum files and a sanitized backup of the MySQL database.

I've already started two SourceForge.net projects, one for <a href="https://sourceforge.net/projects/migratexmbtosmf/" target="_blank">XMB&rarr;SMF</a> and another for <a href="https://sourceforge.net/projects/migr8xmbtophpbb/" target="_blank">XMB&rarr;phpBB</a>. I can start another project, if a different forum platform is chosen.

<hr width="300" />
I'm going to take this opportunity to lay out the first requirement of the migration tool. It needs to prune off some of the <a href="http://www.sciencemadness.org/talk/misc.php?action=list&order=postnum&page=133">133+ pages of non-participating registered users</a>. They comprise more than half the total registered accounts. Nothing against lurkers, of course&mdash;I tend to lurk more than participate. The tool will give this as an option, with a way to set it to delete all 0-post accounts or delete them if they haven't been logged on in a variable ammount of time.

[Edited on 3/15/12 by bfesser]

Polverone
Now celebrating 18 years of madness

Posts: 3164
Registered: 19-5-2002
Location: The Sunny Pacific Northwest
Member Is Offline

Mood: Waiting for spring

bfesser,

Thanks for taking the initiative. It took me a while to upload the bare db dump and code, but it is available now:

I don't have any particular affinity for smf over phpBB, but it looked like the conversion would be more complicated with phpBB. According to someone on the phpBB user forums, writing last year, the current xmb to phpBB converter does not work properly: the user instead had to translate xmb to smf, and then smf to phpBB. If we need to translate to smf anyway, it seemed well enough to me to stop there and not do another conversion to phpBB.

If you have experience with phpBB and prefer it over smf, the phpBB conversion from smf appears to be fine. Using phpBB might allow me to better tap your expertise should it ever need further modifications.

PGP Key and corresponding e-mail address
bfesser
Resident Wikipedian

Posts: 2114
Registered: 29-1-2008
Member Is Offline

Mood: No Mood

md5sums?

At least it's not 4.8GB! Thanks for making this available, Polverone.

[digging in]
This is going to be such a pain in the . . .

[Edited on 3/15/12 by bfesser]

Polverone
Now celebrating 18 years of madness

Posts: 3164
Registered: 19-5-2002
Location: The Sunny Pacific Northwest
Member Is Offline

Mood: Waiting for spring

I installed SMF 2.0.2 and the XMB to SMF converter found here:

Then I tried to run it from the command line and ran into a problem.

 Quote: root@desktop:/var/www/smf# php /var/www/smf/convert.php --path_to=/var/www/smf --path_from=/var/www/smtalk --convert_script=xmb_to_smf.sql --db_pass=XXX Starting Conversion * Converting members... . . . Successful. * Converting categories... Successful. * Converting boards... Successful. * Converting topics... . . . . . . . Duplicate entry '18275' for key 'PRIMARY'

I'm not sure exactly how this comes up. I checked the XMB database and there's no duplicate use of tid 18275.

EDIT: I thought that maybe my database-sanitizing procedure on the public XMB dump somehow produced results which confuse the converter. But it is not so. I tried restoring a full db backup as used on the production forum and running the conversion -- same error.

[Edited on 3-16-2012 by Polverone]

PGP Key and corresponding e-mail address
bfesser
Resident Wikipedian

Posts: 2114
Registered: 29-1-2008
Member Is Offline

Mood: No Mood

So, no md5sums then?

Hmm. I'll set up the database backup and SMF on my host and give that converter a go and we'll see if I get the same result.

Part of me also wonders if it has anything to do with the <a href="http://www.w3schools.com/tags/tag_iframe.asp" target="_blank">iframes</a> <img src="../scipics/_ext.png" /> that <strong><a href="member.php?action=viewpro&member=metalresearcher">metalresearcher</a></strong> used in <a href="viewthread.php?tid=18275">that thread</a>. Perhaps converter.php doesn't play friendly with iframes. I know that phpBB3 no longer allows HTML in posts&ndash;in large part due to the historical abuse and exploitation of iframes. They now rely entirely upon <a href="http://en.wikipedia.org/wiki/BBCode" target="_blank">BBCode</a> <img src="../scipics/_wiki.png" />. I've browsed around all the documentation on the SMF site, but they say nothing at all about support for HTML in posts. My bet is, they don't allow it. Maybe the iframe sandbox tags caused an unforseen logic or loop bug in the converter.php. I'll check their 'Community' board soon.

Coincidentally; I haven't received my confirmation email from the <a href="http://www.simplemachines.org/community/index.php" target="_blank">'SimpleMachines.org Community'</a> <img src="../scipics/_ext.png" /> forum. Same issue I had with SourceForge.net. My email apparently sucks. No spam filters or blocking . . . just unreliability. When I get that, I'll poke around over there to see if anyone's encountered similar issues.

In the meanwhile, I'm sifting through the <a href="http://download.simplemachines.org/?converters;software=xmb#thegoods_xmb" target="_blank">convert.php</a> <img src="../scipics/_ext.png" /> file&mdash;the comments leave much to be desired. No lie, here's a sample:
 Code: // We found the treasure :P.

Seriously? Why add such a useless comment to your code!? Why not just write your code in <a href="http://www.dangermouse.net/esoteric/ook.html" target="_blank">Ook!</a> <img src="../scipics/_ext.png" /> or <a href="http://en.wikipedia.org/wiki/Brainfuck" target="_blank">brainfuck</a> <img src="../scipics/_wiki.png" /> if you want it to be difficult to follow. Some people's children...

[Edited on 7/9/13 by bfesser]

Polverone
Now celebrating 18 years of madness

Posts: 3164
Registered: 19-5-2002
Location: The Sunny Pacific Northwest
Member Is Offline

Mood: Waiting for spring

Subscript/superscript buttons now available

Thanks go to bfesser, who added new subscript/superscript buttons and supporting code to our creaky old XMB software. You can use them via the buttons above the message composition window or by manually inserting sub and sup bbcode tags.

PGP Key and corresponding e-mail address
Pages:  1    3

 Sciencemadness Discussion Board » Non-chemistry » Forum Matters » call for participation: modernizing SM forum software Select A Forum Fundamentals   » Chemistry in General   » Organic Chemistry   » Reagents and Apparatus Acquisition   » Beginnings   » Responsible Practices   » Miscellaneous   » The Wiki Special topics   » Technochemistry   » Energetic Materials   » Biochemistry   » Radiochemistry   » Computational Models and Techniques   » Prepublication Non-chemistry   » Forum Matters   » Legal and Societal Issues