Sciencemadness Discussion Board
Not logged in [Login - Register]
Go To Bottom

Printable Version  
 Pages:  1    3  4
Author: Subject: Site Backup
Panache
International Hazard
*****




Posts: 1034
Registered: 18-10-2007
Member Is Offline

Mood: not letting the robots defeat me

[*] posted on 9-4-2009 at 17:56
Site Backup


Is the site still being backed-up and being made available?

EDIT BY POLVERONE JUNE 2012: Please skip to the end of the thread for latest instructions. The forum backup is now being distributed as a virtual machine appliance via bittorrent.

[Edited on 6-6-2012 by Polverone]




So Swiss Miss, she spends her waking hours absolutely focused on being near me, until anything living (other than a human or a plant) enters the picture, at that point she becomes absolutely focused on killing that living thing be it a fly, a kangaroo, a cow, a mouse, it's so otherworldly, she has no sense of herself during these phases, destroying her physical body without pause, or even acknowledgement and she's so quiet about it, very disturbing initially, when you first fully understand this about her.
View user's profile View All Posts By User
Panache
International Hazard
*****




Posts: 1034
Registered: 18-10-2007
Member Is Offline

Mood: not letting the robots defeat me

[*] posted on 11-4-2009 at 17:58


Archive.org stopped backing up this forum Feb 2008, over 12 months ago, does anyone know of anything archiving site that does this for this site?



So Swiss Miss, she spends her waking hours absolutely focused on being near me, until anything living (other than a human or a plant) enters the picture, at that point she becomes absolutely focused on killing that living thing be it a fly, a kangaroo, a cow, a mouse, it's so otherworldly, she has no sense of herself during these phases, destroying her physical body without pause, or even acknowledgement and she's so quiet about it, very disturbing initially, when you first fully understand this about her.
View user's profile View All Posts By User
Polverone
Now celebrating 15 years of madness
*********




Posts: 3126
Registered: 19-5-2002
Location: The Sunny Pacific Northwest
Member Is Offline


[*] posted on 12-4-2009 at 11:54


I still download site backups pretty frequently. The last one was about a month ago. I have unfortunately not updated the downloadable site archive in quite some time. The problem is that the downloadable version needs many small HTML changes to fix links and page organization. Each time the XMB version changes I need to change the programs that do these alterations. It is a tedious and slow process to update the programs.

It probably wouldn't be that hard to distribute a preconfigured Linux disk image for a virtual machine that has XMB already installed and just needs a fresh database dump to make it match Sciencemadness. If someone wants to make such an image I could provide sanitized database dumps so you can run your own local backup instance of sciencemadness. The sanitized dump would be missing U2U messages, member email addresses and passwords, and the contents of Whimsy and References but otherwise would match the live forum.

I could make such a disk image myself but it would take time that I am short of, which is the same reason I have not updated the conventional forum backup in so long. If there are any tech-savvy members who will take the initiative to produce the disk image, I promise that I will meet them half way and provide the sanitized database dumps.

Any Linux image for this purpose should have recent versions of Apache, MySQL, and PHP installed. It should also have the latest (1.9.11) version of XMB, Firefox or Opera, a PDF reader, a djvu reader, and host/guest integration tools for the selected virtualization platform (VMWare Player or (preferred) VirtualBox). Otherwise the image should probably be slimmed down as much as possible.




PGP Key and corresponding e-mail address
View user's profile Visit user's homepage View All Posts By User
hodges
National Hazard
****




Posts: 453
Registered: 17-12-2003
Location: Upper U.S.
Member Is Offline


[*] posted on 6-9-2009 at 15:22


Not sure if this is still needed, but I have a Ubuntu Linux image for VMWare with MySQL and PHP5 installed. So I was playing around with it today and installed XMB 1.9.11 and set up a database according to the instructions in the install. I was able to get a new bulletin board set up (with no content of course, just the default screens) fairly quickly.

If you wish, I can upload this image. Or, if you want to give me sanitized data to load I can load it first then upload.

The image can be played with VMWare Player, which is free for both Windows and Linux. The Ubuntu version has all the latest patches, and is supported until 2011 (at which time the version probably needs to be upgraded). It includes Firefox, as well as a viewer for PDF and DJVU files. The size of the zip file, including the VMWare Player for windows, is around 1GB, with the unzipped size being about 2.5GB. The data disk is configured to grow up to 8GB as required based on the data stored.

Let me know if you are interested.

Hodges
View user's profile View All Posts By User
MagicJigPipe
International Hazard
*****




Posts: 1547
Registered: 19-9-2007
Location: USA
Member Is Offline

Mood: Suspicious

[*] posted on 29-1-2010 at 10:11


Where can I find the most recent backup? The search engine only reveals one from 2008.

Thanks.

[Edited on 1-29-2010 by MagicJigPipe]




"There must be no barriers to freedom of inquiry ... There is no place for dogma in science. The scientist is free, and must be free to ask any question, to doubt any assertion, to seek for any evidence, to correct any errors. ... We know that the only way to avoid error is to detect it and that the only way to detect it is to be free to inquire. And we know that as long as men are free to ask what they must, free to say what they think, free to think what they will, freedom can never be lost, and science can never regress." -J. Robert Oppenheimer
View user's profile View All Posts By User This user has MSN Messenger
Polverone
Now celebrating 15 years of madness
*********




Posts: 3126
Registered: 19-5-2002
Location: The Sunny Pacific Northwest
Member Is Offline


[*] posted on 30-1-2010 at 12:21


Hodges, I am sorry that I missed your post earlier. I did almost exactly what you did, except I started out with a compact Turnkey Linux image and used VirtualBox instead of VMWare. I now have a distributable Linux-image that contains a live, runnable (but sanitized) version of the forum. Details following in next post.



PGP Key and corresponding e-mail address
View user's profile Visit user's homepage View All Posts By User
Polverone
Now celebrating 15 years of madness
*********




Posts: 3126
Registered: 19-5-2002
Location: The Sunny Pacific Northwest
Member Is Offline


smile.gif posted on 30-1-2010 at 12:25
Live, runnable backup copy of Sciencemadness forum now available


If you want a backup copy of the SM forum that can be run on your own computer, start downloading this file:

Offline Copy of Sciencemadness

It is big, about 1.8 GB. A successful download should have the MD5 hash 6a41b7a28c999a243a388f6db5d83d62. Use a tool that automatically resumes interrupted downloads, like wget or GetRight, to download it.

After you have downloaded the archive itself, you will also want to download and install VirtualBox, the free virtualization software. The .ovf format supposedly works with other virtualization software too, like VMWare, but I have not tried to validate this.

Unzip the sm_offline.zip, then import the .ovf file from the contents with VirtualBox's Import Appliance wizard:




The import process may take several minutes to complete.



After you have finished importing the image, be sure to set the "Enable IO APIC" feature under the virtual machine's system settings. If this feature is not enabled, the virtual machine will fail to start properly.



After IO APIC has been enabled, you should be able to press "start" for the virtual machine and see this virtual console appear after a few seconds of startup activity:



You will want to point your web browser to the same IP address as shown in the console. You should see a screen like this, being served from the virtual machine's web server:



If you click on the most prominent link, you should see the forum, looking much as it did at the time the backup copy was made:



You can browse down into all the saved forums:



When you are done, go back to the virtual console and choose "Shutdown" from the advanced menu:



Privacy

I have removed all U2U messages, posts from protected forums, and attachments that belonged to the removed posts. Members whose email addresses are not set to display have had their addresses blanked out. Invisible members have been made visible. All IP address information has been removed. All member passwords have been changed to 'reader'. Buddy, favorite, and ignore lists have been removed.

Passwords
As mentioned above, the password for every forum account in the backup is 'reader.' The root password on the virtual machine is 'science'.

Usage
This backup can be browsed and searched without a working internet connection. Should Sciencemadness ever disappear, this backup should retain all of the public forum posts. What's more, the actual XMB software configuration and MySQL database contained in the virtual image could be used to restart the forum on another web host if I became unable to maintain this site.

It is even possible, though probably pointless, to log in to the backup as an existing member and then post messages to the backup itself. If you are curious about the administrator's powers, you can even log in as Polverone with the same password. This gives you, for example, access to the admin control panel where you can search for attachments (a feature not available to ordinary members on the live site).

[Edited on 2-2-2010 by Polverone]




PGP Key and corresponding e-mail address
View user's profile Visit user's homepage View All Posts By User
Polverone
Now celebrating 15 years of madness
Thread Topped
2-2-2010 at 13:22
dann2
International Hazard
*****




Posts: 1523
Registered: 31-1-2007
Member Is Offline


[*] posted on 24-2-2010 at 06:05


Hello,
On attempting to download the 1.72Gb file with Wget, I managed to get to 46% before the thing 'give up'. (It's own description). There were many 'timeouts', and stops and starts along the way to the 46% mark and it resumed from where it started OK.
When I cranked up Wget again it starts to download to a new file from the beginning. Is there any way around this?

EDIT:
Good afternoon Dann2!
Use the -c switch (you silly buffoon).
eg: (at command prompt)...> wget -c http://library.sciencemadness.org/archive/sm_offline.zip

Thanks,
Dann2

To admin.
Delete this post of you so wish. I left it here is it may be useful.

[Edited on 24-2-2010 by dann2]
View user's profile View All Posts By User
dann2
International Hazard
*****




Posts: 1523
Registered: 31-1-2007
Member Is Offline


[*] posted on 25-2-2010 at 15:02



Hello,

Is anyone having problems with the Sun Virtual Machine.
I cannot get it to run.
It somes up with a 'No Interfaces configured' message box.
During the start up (after I press the green start arrow) it says (amongst other things) that the lease has failed.
Does this ring any bells?
I have the sm-offling.ovf file and the virtual box installed on an external hard drive if that makes any difference.

Dann2

photos.jpg - 70kB

[Edited on 25-2-2010 by dann2]

error.jpg - 5kB
View user's profile View All Posts By User
Polverone
Now celebrating 15 years of madness
*********




Posts: 3126
Registered: 19-5-2002
Location: The Sunny Pacific Northwest
Member Is Offline


[*] posted on 26-2-2010 at 16:39


Hi dann2,

I have been able to reproduce your problem. At home my computers are connected to a router. If I connect a computer to the internet directly, no router, or unplug the network connection altogether, I get the same error you saw when I try to start the VM. I do not know a lot about networking but I will try to find out how this problem can be solved.




PGP Key and corresponding e-mail address
View user's profile Visit user's homepage View All Posts By User
watson.fawkes
International Hazard
*****




Posts: 2793
Registered: 16-8-2008
Member Is Offline


[*] posted on 26-2-2010 at 17:40


Quote: Originally posted by Polverone  
If I connect a computer to the internet directly, no router, or unplug the network connection altogether, I get the same error you saw when I try to start the VM.
The DHCP client is failing, meaning it's trying to get an IP address from a DHCP server and failing. Without an IP address, you can't get a server to bind to a port. In the present situation, you don't need any network address other than localhost, 127.0.0.1. The relevant interface could be manually configured that way with "ifconfig" at boot time, replacing the use of the DHCP client.
View user's profile View All Posts By User
Polverone
Now celebrating 15 years of madness
*********




Posts: 3126
Registered: 19-5-2002
Location: The Sunny Pacific Northwest
Member Is Offline


[*] posted on 26-2-2010 at 18:06


Quote: Originally posted by watson.fawkes  
The DHCP client is failing, meaning it's trying to get an IP address from a DHCP server and failing. Without an IP address, you can't get a server to bind to a port. In the present situation, you don't need any network address other than localhost, 127.0.0.1. The relevant interface could be manually configured that way with "ifconfig" at boot time, replacing the use of the DHCP client.


I tried manually configuring it with 127.0.0.1, netmask 255.255.255.0. No success. I also tried with 127.0.1.1 since a boot message indicated that was being used as the host name -- still no luck.

The VM is set up with bridged networking because the network services (MySQL and HTTP) need to be accessible from the host machine, and all the VM guides I came across indicated that bridged networking was the way to go for that. I don't know why it works so easily with DHCP available but so dismally when I try to configure a static address. I will try a couple more things to get static working, and if that fails I'm going to see if running a DHCP server on the host machine can fix it.




PGP Key and corresponding e-mail address
View user's profile Visit user's homepage View All Posts By User
dann2
International Hazard
*****




Posts: 1523
Registered: 31-1-2007
Member Is Offline


[*] posted on 26-2-2010 at 18:26


Hello,

Ran the Virtual Box again and got the 'No interfaces configured'.
Then went to configure the NIC manually (a box came up asking me).

I used 127.0.1.1 for three of the entries except gateway where I used 0.0.0.0 (wild assed guess)

That got me a bit further.............untill the yoke started to look for a lamp login and password.
I tried some rubbish logins and passwords to no avail.
I tried 'science' for login and 'reader' for password to no avail.


I decided to start again.
Things went different this time (for some unknown reason).
I did not get the ' No interfaces are configured' message but got the box as per the picture depicting the IP Address to use (two of them the same)
+ Webmin http address
+ PHPmyadmin
+ SSH
(similar stuff to picture way above only differetn actual numbers).

I put the http address into my browser address bar and get the next correct screen shown
Turnkey linix application running etc etc (same as picture way above )

I click on ScienceMadness Forum Archve but get a 'Internet Explorer Cannot display web page' (standard message from MS Explorer).


Thanks for your time.

Dann2

Nearly lost that post only for my habit of select and copy before hitting post or preview. :o:o:o:o:o:o:o:o:o

[Edited on 27-2-2010 by dann2]
View user's profile View All Posts By User
Polverone
Now celebrating 15 years of madness
*********




Posts: 3126
Registered: 19-5-2002
Location: The Sunny Pacific Northwest
Member Is Offline


[*] posted on 26-2-2010 at 18:37


If you click on 'phpinfo' do you get some information displayed, or an error from the browser? If the phpinfo displays correctly I think I know why the forum is not working. The forum configuration file needs to be updated at startup-time with the address that you will use to get to the forum. I added a script that performs this update when the Turnkey Linux configuration screen first comes up. Since you configured the network interface after this interface program began running, the forum configuration file will not be properly updated.

If the phpinfo screen is not working, though, I am afraid the problem is deeper than forum configuration.




PGP Key and corresponding e-mail address
View user's profile Visit user's homepage View All Posts By User
dann2
International Hazard
*****




Posts: 1523
Registered: 31-1-2007
Member Is Offline


[*] posted on 26-2-2010 at 18:48


Hello,

I tried again this time with my network cable plugged in. It was not plugged in the last time as I purposly plugged in out when running the Virtual box.

With the cable plugged in all runs OK and I can read all of forum and post to 'SciMadness forum' as Polverone!! and Dann2 as well (I suppose) though I did not try.

I aslo get large long page of info when PHPINFO is clicked.

Dann2
View user's profile View All Posts By User
dann2
International Hazard
*****




Posts: 1523
Registered: 31-1-2007
Member Is Offline


[*] posted on 26-2-2010 at 18:52


Hello,

I attached the phpinfo page if you want it.

Why do I need to network cable plugged in the back (my interned connection). Sounds very iffey.

Dann2

Attachment: phpinfo().mht (70kB)
This file has been downloaded 660 times

View user's profile View All Posts By User
watson.fawkes
International Hazard
*****




Posts: 2793
Registered: 16-8-2008
Member Is Offline


[*] posted on 27-2-2010 at 06:02


Quote: Originally posted by Polverone  
I tried manually configuring it with 127.0.0.1, netmask 255.255.255.0. No success. I also tried with 127.0.1.1 since a boot message indicated that was being used as the host name -- still no luck.
If it needs to route between these two addresses (and I don't know if it does or not), the netmask would need to be 255.255.0.0, since otherwise the mask would prevent the two local-network addresses (127.*) from being seen as the same network. (A manual route would work in that case, but the netmask is better.)

Running a DHCP server in the virtual box, though, sounds like it might be the quickest solution.
View user's profile View All Posts By User
Rosco Bodine
International Hazard
*****




Posts: 6141
Registered: 29-9-2004
Member Is Offline

Mood: analytical

[*] posted on 11-10-2010 at 14:51


What is the most recent backup? A lot of information has been added in the past few months, so a current backup would be timely.
View user's profile View All Posts By User
Polverone
Now celebrating 15 years of madness
*********




Posts: 3126
Registered: 19-5-2002
Location: The Sunny Pacific Northwest
Member Is Offline


[*] posted on 12-10-2010 at 21:22


Thanks for the reminder. I have replaced the previous backup with one created yesterday. The link is still http://library.sciencemadness.org/archive/sm_offline.zip



PGP Key and corresponding e-mail address
View user's profile Visit user's homepage View All Posts By User
Rosco Bodine
International Hazard
*****




Posts: 6141
Registered: 29-9-2004
Member Is Offline

Mood: analytical

[*] posted on 15-10-2010 at 06:52


You are welcome. The attachments are a very important part of the database. Is there a way to include the attachments database so that attachments are available from the backup messages ?

Under her clothes this doll should be 100% anatomically correct :D :cool:

http://www.youtube.com/watch?v=Qrw5qEDRLw0&fmt=18

[Edited on 15-10-2010 by Rosco Bodine]
View user's profile View All Posts By User
Polverone
Now celebrating 15 years of madness
*********




Posts: 3126
Registered: 19-5-2002
Location: The Sunny Pacific Northwest
Member Is Offline


[*] posted on 15-10-2010 at 12:14


The attachments in public forums are part of the backup. If files were posted to external file hosting sites instead of directly attached they of course are not contained in the backup, though the links are still there and can be downloaded if the file host retained the file.

Files that were uploaded to the scipics account are not part of the backup either, but that is easy enough to back up. Just use a tool like wget with its mirror option to get everything in http://www.sciencemadness.org/scipics/. I have taken no measures to hide that directory or limit downloads from it. If you'd prefer I can include a scipics copy as part of future backups, though that will add several hundred megabytes.

I will not release backups containing the contents of the restricted forums as long as this site remains online. In the event that the site were shut down, I could release a VM image that contains those forums too from one of my more frequent private backups. Or, if you want to get really thorough about information survival, I could distribute encrypted full backups and leave the password with other moderators to distribute in case SM goes offline and I am unable to respond. That way in case of crisis some tech-savvy moderators or members can reboot the forum with most data intact on another web host, or even running as a TOR onion service for extreme survivability.

I am not going to distribute backups (even encrypted ones) that contain user U2U messages and passwords under any circumstances, so when I speak of "full" backups I just mean ones with all posts and attachments intact.

EDIT: be careful with using mirroring software like wget that you do not allow it to move back up the directory hierarchy and try to get the whole forum. If you do allow it to run away, and you have a fast internet connection, and you haven't set your options to delay or throttle downloads, you may get locked out of the site for up to 24 hours by an automatic protection script I have in place. I managed to lock myself out a few minutes ago while validating the scipics backup method, until I directly logged into the host machine and reset the restrictions.

[Edited on 10-15-2010 by Polverone]




PGP Key and corresponding e-mail address
View user's profile Visit user's homepage View All Posts By User
Rosco Bodine
International Hazard
*****




Posts: 6141
Registered: 29-9-2004
Member Is Offline

Mood: analytical

[*] posted on 15-10-2010 at 16:26


Yeah, I think the scipics directory should be included as part of the forum backup.
There has been suggestion that files be put there to preserve them as part of the forum archive so that images and files don't get lost or expired by external hosting sites. Especially the member publications section would suffer from missing images content. The pdf conversion files for those threads generally exceed the direct attachment file size limit and have to be uploaded to the scipics directory anyway. If the search engine and printable version functions are possible to be included or are already in the backup that would be great also.
View user's profile View All Posts By User
Polverone
Now celebrating 15 years of madness
*********




Posts: 3126
Registered: 19-5-2002
Location: The Sunny Pacific Northwest
Member Is Offline


[*] posted on 15-10-2010 at 18:16


The VM-image backups are actually running the forum software, so you can use the forum software's search tool (such as it is) and get printable versions of threads. I will include the scipics contents in the next backup.



PGP Key and corresponding e-mail address
View user's profile Visit user's homepage View All Posts By User
Polverone
Now celebrating 15 years of madness
*********




Posts: 3126
Registered: 19-5-2002
Location: The Sunny Pacific Northwest
Member Is Offline


shocked.gif posted on 24-12-2010 at 11:10
sciencemadness contingency plan


I have uploaded a new virtual machine image containing a sanitized snapshot of the forum database and all the supporting software needed to run it. This backup is much larger than the last one I made available because it incorporates the contents of the scipics directory plus encrypted copies of all posts and attachments from the private forums. Even in the protected private archives, though, I have removed personal information such as account password hashes, private email addresses, U2U messages, and IP addresses.

The new backup also contains scripts and instructions that should allow anyone with the archive password to use the VM image contents as a seed to restart the forum on a new web host, with a new domain name, if sciencemadness.org and/or I ever disappear. I am certainly not expecting to disappear, but I have seen other popular forums vanish for no good reason and I want to ensure that it can never happen to this site. This site is really made by the members who participate here, and I want to forestall the possibility of it ever being lost to them.

I don't know who here would be comfortable with the technical aspects of trying to revive the forum in the event of my disappearance. I know that some of our members must be technically capable. I want to spread the forum backup as far as possible, but I am leaving the password needed to recover the private forums in the hands of the moderators. They are asked to spread the password far and wide should I fall out of contact for more than 30 days.

As before, you should use a download client with resume-capability to get the latest offline copy of Sciencemadness. The instructions for bringing the virtual image to life are still the same. There is now a recovery guide included in the image, accessible via your web browser, explaining how to re-integrate the protected forum contents and reseed it on a new web host if you have the password.

You might want click on the printable version of this thread and print it to PDF if you haven't previously installed a VirtualBox image, since in an emergency this thread might not be available for reference.

I will try to keep the VM copy of sciencemadness no more than a few months out of date. The preparation is mostly automated. It just takes a while to download and upload a few gigabytes over my home internet connection.




PGP Key and corresponding e-mail address
View user's profile Visit user's homepage View All Posts By User
quicksilver
Super Moderator
*******




Posts: 1820
Registered: 7-9-2005
Location: Inches from the keyboard....
Member Is Offline

Mood: ~-=SWINGS=-~

[*] posted on 25-12-2010 at 10:32


Polverone , thank you for the work and time (Merry Christmas, too)

I am NOT of a paranoid mind-set but if I lived in certain countries that developed a precedent of filtering content or whatever......I think it might be a damn good idea to download a copy & keep it on DVD.


Edit:
-{I'm glad I got back in time to edit this.}-

I had some questions and I am glad to report some answers so that if any folks have similar questions they can make use of some experimenting I've done.
VMPLAYER (v2.x from 2008) is what you want if you want to run your backup on a old windoz machine. I have two machine running SUSE (a little one with 9x with very little processing power and a larger one with a great deal more RAM and processor. IF you want to run a VM on a machine without Linux but it's fairly new and strong - no problems. But IF you have an old laptop with 1Gb RAM and a CPU that has something like 1 GHz the old one will function even under some older version of windoz. So VMPLAYER (v2.x from 2008) is what you would likely be doing best at using as it's VERY limited but it has VERY low demands.

[Edited on 26-12-2010 by quicksilver]




View user's profile View All Posts By User
 Pages:  1    3  4

  Go To Top