Sciencemadness Discussion Board
Not logged in [Login - Register]
Go To Bottom

Printable Version  
 Pages:  1  ..  18    20    22  ..  28
Author: Subject: Tired of reporting spam
j_sum1
Administrator
********




Posts: 4604
Registered: 4-10-2014
Location: Oz
Member Is Offline

Mood: Metastable, and that's good enough.

[*] posted on 2-9-2018 at 14:17


There is at least one bot that hits all 14 forums. And for some reason is able to bypass the flood protection. Your post indicates two spam per minute but it is sometimes an much as three.

There are some other similar spam that arise and only hit one forum -- usually Gen Chem. I think these ones get thwarted by flood protection.
View user's profile View All Posts By User
Diachrynic
Hazard to Others
***




Posts: 126
Registered: 23-9-2017
Location: Germany
Member Is Offline

Mood: zenosyne

[*] posted on 10-9-2018 at 14:23


Idea: what if we take the ten most common English words and run a quick check over a new post. If there is a high enough deviation (which can be as much as only one word of that list in the entire post) it needs to be checked by a moderator first before it gets send. This should only apply to new topics. The moderator then either decides to show it or to trash it.

Of course bots can easily circumvent this, but maybe there is some use to this idea.

The idea is also flawed in that it is not automatic. In order to prevent legitimate topics from being trashed a human being is needed. But the mods could be completely overwhelmed by the number of posts.

Like I said, just something that came to my mind because I noticed the bots just shove a bunch of keywords into a post but almost never say "I am trying to do the..." or something like that.




Just killing time until the world ends. - Furude Rika, HNNKN
View user's profile View All Posts By User
Swinfi2
Hazard to Self
**




Posts: 70
Registered: 19-2-2018
Location: England
Member Is Offline

Mood: Catalytic

[*] posted on 10-9-2018 at 17:39


@streety the "don't you like sex" thread. I read the first 2-3 pages and it looks like a compilation of everything to do with nerve agents/drugs and other bad stuff that (when organised as such) kinda implicates the forum staff as they have posts amongst it.

I have a hard time believing the post is "real" as many posts reference poeple/posts that are missing and context jumps. It looks to me like an AI word salad of coppied posts to give the appearance of coherance.

How could that even happen? That thread freaks me out, it looks like a set up imo.
View user's profile View All Posts By User
fusso
International Hazard
*****




Posts: 1467
Registered: 23-6-2017
Location: Toaru city, Toaru nation, Asia, Earth, ∥ universe
Member Is Offline

Mood: Haidrofobik

[*] posted on 11-9-2018 at 14:01


Quote: Originally posted by Swinfi2  
@streety the "don't you like sex" thread. I read the first 2-3 pages and it looks like a compilation of everything to do with nerve agents/drugs and other bad stuff that (when organised as such) kinda implicates the forum staff as they have posts amongst it.

I have a hard time believing the post is "real" as many posts reference poeple/posts that are missing and context jumps. It looks to me like an AI word salad of coppied posts to give the appearance of coherance.

How could that even happen? That thread freaks me out, it looks like a set up imo.
And I briefly skimmed a few posts starting from the bottom. I noticed some spam posts at the bottom. Going up, the thread seemed to be detritused due to trolls. And I found a few currently active members there. I hope someone who was involved could explain it for dummies.



Useful sites:
Balance Chemical Equation: http://www.webqc.org/balance.php
Molecular mass and elemental composition calculator: https://www.webqc.org/mmcalc.php
Solubility table: https://en.wikipedia.org/wiki/Solubility_table
Azeotrope table: https://en.wikipedia.org/wiki/Azeotrope_tablesIt's not crime if noone finds out - Nyaruko
List of materials made by ScienceMadness users: https://docs.google.com/spreadsheets/d/1nmJ8uq-h4IkXPxD5svnT...
View user's profile View All Posts By User
j_sum1
Administrator
********




Posts: 4604
Registered: 4-10-2014
Location: Oz
Member Is Offline

Mood: Metastable, and that's good enough.

[*] posted on 11-9-2018 at 14:07


I don't recall the thread. U2u me a link. If it is in detritus then that might be too good a place for it. Sounds like nothing will be lost in killing it.
View user's profile View All Posts By User
fusso
International Hazard
*****




Posts: 1467
Registered: 23-6-2017
Location: Toaru city, Toaru nation, Asia, Earth, ∥ universe
Member Is Offline

Mood: Haidrofobik

[*] posted on 11-9-2018 at 14:13


Quote: Originally posted by streety  
In putting together these figures I discovered two odd posts. The post date is 1969. In the database they are represented by timestamp values of 0 and 1. They are also clearly spam but then there are 8 pages of legitimate content.
https://www.sciencemadness.org/whisper/viewthread.php?tid=21...
@jsum well here you are.



Useful sites:
Balance Chemical Equation: http://www.webqc.org/balance.php
Molecular mass and elemental composition calculator: https://www.webqc.org/mmcalc.php
Solubility table: https://en.wikipedia.org/wiki/Solubility_table
Azeotrope table: https://en.wikipedia.org/wiki/Azeotrope_tablesIt's not crime if noone finds out - Nyaruko
List of materials made by ScienceMadness users: https://docs.google.com/spreadsheets/d/1nmJ8uq-h4IkXPxD5svnT...
View user's profile View All Posts By User
j_sum1
Administrator
********




Posts: 4604
Registered: 4-10-2014
Location: Oz
Member Is Offline

Mood: Metastable, and that's good enough.

[*] posted on 11-9-2018 at 14:46


Quote: Originally posted by fusso  
Quote: Originally posted by streety  
In putting together these figures I discovered two odd posts. The post date is 1969. In the database they are represented by timestamp values of 0 and 1. They are also clearly spam but then there are 8 pages of legitimate content.
@jsum well here you are.


Ok. I took a look. It can stay in detritus. It actually looks like a collection of stuff from several different threads including trolling, spamming and legitimate discussion of two or three different topics.
I don't know the thread's history -- it all seems weird. But it is not doing any harm where it is.
View user's profile View All Posts By User
WGTR
International Hazard
*****




Posts: 846
Registered: 29-9-2013
Location: Online
Member Is Offline

Mood: Outline

[*] posted on 13-9-2018 at 17:00
Negative 15 posts?




spam.jpg - 49kB




View user's profile View All Posts By User
CharlieA
National Hazard
****




Posts: 425
Registered: 11-8-2015
Location: Missouri, USA
Member Is Offline

Mood: No Mood

[*] posted on 14-9-2018 at 16:47


Tonight is especially frustrating...3 pages of posts today, and 2 pages of them are spam. I think when a new account is made, the account shouldn't be able to post for a week. There is much talk here about this subject but nothing seems to get done about it. I wish I were computer literate but I'm not, so I'm unable to do anything about it (except leave the forum).:(
View user's profile View All Posts By User
streety
Hazard to Others
***




Posts: 105
Registered: 14-5-2018
Member Is Offline


[*] posted on 15-9-2018 at 05:25


This is an update on my post from the top of page 19.

The script was updated and run every 2 minutes. I restarted it on the 19th and analyze only full days below, so from the 20th August to the 14th September.

During that time I recorded 2803 spam topics for a rate of 100/day.

The histogram below shows the time it took to delete each spam topic.

spam_duration_20180915.png - 56kB

The minimum was again barely above the sampling frequency of the script so I still probably missed some topics that were deleted so quickly they were gone before my script downloaded the page. Should be less than last time though.

The average was 69 minutes(down from 84 minutes). 34% (up from 32%) were deleted within 30 minutes, 64% (up from 47%) within 60 minutes, and 83% (up from 79%) within 120 minutes. The median was 45 minutes (down from 65 minutes).

There isn't much effect of time of day on spam duration in this period.

time_of_day_posted_20180915.png - 107kB

time_of_day_deleted_20180915.png - 105kB

You may notice the topic deleted after more than 3000 minutes. It was not a spam post so I may need to think about how to handle deletions that are not spam related. In this case it seems to be a legitimate member who created a new topic in error instead of a reply to an existing topic. It was then later cleaned up by an administrator.

spam_posts_per_day_20180915.png - 141kB

Yesterday was particularly bad for spam. Hopefully it won't be the start of an up-tick in the posting frequency.
View user's profile View All Posts By User
JJay
International Hazard
*****




Posts: 3321
Registered: 15-10-2015
Member Is Offline

Mood: resigned

[*] posted on 21-9-2018 at 13:17


The spammers are in rare form today. I have never seen so much. I must have reported 20 of them this morning.



I'm no longer involved in this forum.
View user's profile View All Posts By User
WGTR
International Hazard
*****




Posts: 846
Registered: 29-9-2013
Location: Online
Member Is Offline

Mood: Outline

[*] posted on 21-9-2018 at 14:04


I would suggest that Polverone add some extra people to the trusted reporters list. When I check in I often find spam that's been here for 6 hours or more, with a couple dozen views and presumably at least half that many reports. Then when I report it, the thread is gone within a minute or two. That tells me that the people doing most of the reporting aren't "trusted reporters" yet. Keep reporting it though, because Polverone keeps an eye on who's doing the reporting, and sometimes adds additional people to the "trusted" list. Also, when I log in I can clear out several pages of spam at one go if others have already reported it.



View user's profile View All Posts By User
streety
Hazard to Others
***




Posts: 105
Registered: 14-5-2018
Member Is Offline


[*] posted on 21-9-2018 at 19:35


The past few days have been quite bad but today doesn't seem extraordinary.

spam_posts_per_day_20180921.png - 141kB

The median period of time spam posts are hanging around was a little higher than usual.



I have been working with woelen to deploy a script to help with the spam. I should really follow up on that. Essentially it would do exactly as you suggest and increase the impact more members could have.


Edit to include most recent day in duration plot. Changing the interpretation.
[Edited on 22-9-2018 by streety]

[Edited on 22-9-2018 by streety]

spam_posts_durations_per_day_20180921.png - 134kB
View user's profile View All Posts By User
symboom
International Hazard
*****




Posts: 801
Registered: 11-11-2010
Location: Wrongplanet
Member Is Offline

Mood: Doing science while it is still legal since 2010

[*] posted on 21-9-2018 at 21:12


Anyone know if they are using AI (artificial intelligence) yet
Got to be ahead of the game. Divergent and convergent thinking

Open message
To streety
This guy lesterpq11 look at for pattern see key words to outwit the AI

AI seems to try to imitate humans where ever it gets the info from


[Edited on 22-9-2018 by symboom]




Chemistry video Storage
https://www.mediafire.com/folder/kbll6gz9bdb4q/Videos
The State of Mad Science newsletter
http://www.sciencemadness.org/talk/viewthread.php?tid=98577
My open source scratch paper
Pastebin.com/u/symboom
Natures Intellectual Organic Peroxide. >>Ascaridole <<
2020 year of science
Oxone
Used for the production of --> CH2O/Cl2/ClO2/Br2/I2
------------------------------------->>Hydrogen Peroxide <<
Sodium had its fame long enough time for Calcium
View user's profile View All Posts By User
Melgar
Anti-Spam Agent
*******




Posts: 2002
Registered: 23-2-2010
Location: NYC
Member Is Offline

Mood: Aromatic

[*] posted on 22-9-2018 at 00:09


Oh, I did some analysis on the test database. Turns out 99% of our users have 0 posts. Most of those registrations were automatic, and had names that had obviously generated automatically. It seems our active membership is perhaps around 8000.

By the way, my efforts to get this site working using phpBB has had quite a few breakthroughs in the last week or so. Check it out:

http://35.185.63.230/talk/index.php

The theme is just a random one, and other themes can be added and used. There are some bbcodes that I've had difficulty converting, and there's definitely some more things that would need working out, but the hardest parts are all behind me now. Thoughts?

[Edited on 9/22/18 by Melgar]




The first step in the process of learning something is admitting that you don't know it already.

I'm givin' the spam shields max power at full warp, but they just dinna have the power! We're gonna have to evacuate to new forum software!
View user's profile View All Posts By User
fusso
International Hazard
*****




Posts: 1467
Registered: 23-6-2017
Location: Toaru city, Toaru nation, Asia, Earth, ∥ universe
Member Is Offline

Mood: Haidrofobik

[*] posted on 22-9-2018 at 05:19


Quote: Originally posted by Melgar  
By the way, my efforts to get this site working using phpBB has had quite a few breakthroughs in the last week or so. Check it out:

http://35.185.63.230/talk/index.php

The theme is just a random one, and other themes can be added and used. There are some bbcodes that I've had difficulty converting, and there's definitely some more things that would need working out, but the hardest parts are all behind me now. Thoughts?[Edited on 9/22/18 by Melgar]
Why is the last post in 2016 not something more recently (eg this year)?

[Edited on 22/09/18 by fusso]




Useful sites:
Balance Chemical Equation: http://www.webqc.org/balance.php
Molecular mass and elemental composition calculator: https://www.webqc.org/mmcalc.php
Solubility table: https://en.wikipedia.org/wiki/Solubility_table
Azeotrope table: https://en.wikipedia.org/wiki/Azeotrope_tablesIt's not crime if noone finds out - Nyaruko
List of materials made by ScienceMadness users: https://docs.google.com/spreadsheets/d/1nmJ8uq-h4IkXPxD5svnT...
View user's profile View All Posts By User
Melgar
Anti-Spam Agent
*******




Posts: 2002
Registered: 23-2-2010
Location: NYC
Member Is Offline

Mood: Aromatic

[*] posted on 22-9-2018 at 05:34


Because I'm working with an older database backup that was stripped of personal information by Polverone. The idea was to write the code to do it on the older database backup as a proof of concept, then use the same or similar code to convert the live database when we're ready to do that.

There have been pushes to do this for years now, and migrating would certainly solve the spam problem.




The first step in the process of learning something is admitting that you don't know it already.

I'm givin' the spam shields max power at full warp, but they just dinna have the power! We're gonna have to evacuate to new forum software!
View user's profile View All Posts By User
RogueRose
International Hazard
*****




Posts: 1267
Registered: 16-6-2014
Member Is Offline


[*] posted on 22-9-2018 at 10:16


This may have been asked and stated but I haven't seen it. How many reports of spam are needed for all the users posts to be deleted?

What I want to know is if I see 10 posts by a user, how many need to be reported for the program to sweep up all their posts?
View user's profile View All Posts By User
CuReUS
International Hazard
*****




Posts: 913
Registered: 9-9-2014
Member Is Offline

Mood: No Mood

[*] posted on 23-9-2018 at 02:19


anti spam measures - https://web.archive.org/web/20151125135503/http://www.ninjap...

distilled from wiki-https://en.wikipedia.org/wiki/Forum_spam#Spam_prevention
Quote:
1.Blacklisting services such as fspamlist, StopForumSpam and keep databases of IP addresses, usernames and e-mail addresses used to post spam or register forum accounts.Forum software can query these lists and either deny posts or registration, or submit the request for human moderation. This is similar to DNSBL services.

2.Simple CAPTCHA systems which display alphanumeric characters have proven vulnerable to optical character recognition software but those that scramble the characters appear to be far more effective

3.Textual confirmation,in which the user answers one or more random questions to prove that he/she is not a spambot - ( doesn't have to be chemistry questions,could be questions like "how many letters in sciencemadness" or in which language is it ?)

4.Confirmation e-mails to registering users prior to allowing the user a first log in, either containing a site-generated password or an activation code/link

5.Authoritative voice, using an external filtering service to get a verdict if the data is spam or not.( free filtering services available)

6.Denial of registration from certain domains that are a major source of spambots, or even domain extensions such as .ru, .br, .biz

7.Using a search engine to investigate usernames for hits as recognized spambots on other forums(this could be coded into SM so that it does it automatically.)

8.Changing technical details of the forum software to confuse bots — for example, changing "agreed=true" to "mode=agreed" in the registration page of phpBB

9.Blocking posts or registrations that contain certain blacklisted words ( can be automated)

10.A useful technique for proactive detection of well-known spammer proxies is to query a search engine for this IP. It will show up on pages that specialize in the listing of proxies.( again can be automated)

11.Redirecting spammers to "spam subforums" to direct spam away from human users on the main site( or even do a return to sender approach and beat them at their own game :D)


My own suggestions-
1.Most spammers have a link in their message,so we could detect posts with links and block them(for 1st post only)
2.Do not allow newly registered members to post more than 1 message or in more than 1 sub forum.
3.Block usernames or posts with non english alphabets

someone had posted an amazing idea is this thread,but I can't seem to find that post now.The idea was to run usernames through a password strength checker.Since bots use long alphanumerical strings,they would indirectly make very strong passwords,which could be detected and blocked.
We must do something fast,or pretty soon we would have to build another arc to escape this flood :(

[Edited on 23-9-2018 by CuReUS]
View user's profile View All Posts By User
RogueRose
International Hazard
*****




Posts: 1267
Registered: 16-6-2014
Member Is Offline


[*] posted on 23-9-2018 at 08:12


I have a feeling that there is more going on than just spam. I have a feeling that the posts might be a way to pass messages to others with no record of them. The posts are up for a short time and then the "system" erases them. While they are up they are grabbed. I would suspect that the spam bots wouldn't continue to post here if it wasn't getting some kind of return. It wouldn't post with such furry unless there was a benefit being had and I don't think it is members buying access to adult sites or ED pills.

This could be a serious issue that really needs taken care of and is wreckless allowing it to continue.

The mods should "deputize" some of the members to allow them to review the first posts of new members, all new posts get held until "OK'd" by a "deputy SPAM bot". The member checks a hidden sub forum where the new posts are sent and clears good posts. This would also work well to block people from signing up to reply to 9 year old threads applauding a cook and bashing someone protecting the forum.

View user's profile View All Posts By User
Melgar
Anti-Spam Agent
*******




Posts: 2002
Registered: 23-2-2010
Location: NYC
Member Is Offline

Mood: Aromatic

[*] posted on 23-9-2018 at 12:46


It's just bots that scan the internet for message boards that they're able to automatically register at. They use various tactics to try and make it harder to keep them out. Some of these tactics probably don't make any logical sense, but they kind of operate on a "throw a bunch of shit at the wall and see what sticks" philosophy. The only real way out is to migrate to software that's still being actively developed, which I'm trying to do with phpBB. If anyone has any thoughts on this, I'd love to hear to them. If you want server access, send me your RSA public keys, and I'll add you to authorized_hosts. If you want admin access on the phpBB test site, register at the link I posted above, and message me your username, and I'll make you an admin. If you want to look at the forum databases, I'm running a virtual machine of the XMB server within the phpBB server, and you can access both MySQL databases via TCP/IP if you're logged in. I have it set up so I can easily pull data in from both databases via a Ruby console. Like, to show data for a random user, you can type XMB.members.random, or PHPBB.users.random, respectively. I would really like some help with the PHP part, since my own PHP skills are severely lacking. I've gotten this far mainly by using Ruby and relying on my decent SQL background. Most of the rest of what's left is just annoyances. Like the fact that [size] and [attachment] tags work differently in phpBB. I'm not sure whether to try and script the conversion of XMB tags to their phpBB versions (something I did already with the [rquote] tags) or try and make the XMB bbcode tags work as they are.

Right now, the goal is to get it to a point where we like how it looks, then set up a new board with all the settings imported from the one I'm working on now. Presumably run by Polverone. Then use the tools we've developed to transfer the data over.

XMB hasn't been actively developed since 2009, and I'm pretty sure we had a consensus a long time ago that we're going to have to transition to new software if we ever want to address this problem. Correct me if I'm wrong.




The first step in the process of learning something is admitting that you don't know it already.

I'm givin' the spam shields max power at full warp, but they just dinna have the power! We're gonna have to evacuate to new forum software!
View user's profile View All Posts By User
RogueRose
International Hazard
*****




Posts: 1267
Registered: 16-6-2014
Member Is Offline


[*] posted on 24-9-2018 at 04:15


Quote: Originally posted by Melgar  
It's just bots that scan the internet for message boards that they're able to automatically register at. They use various tactics to try and make it harder to keep them out. Some of these tactics probably don't make any logical sense, but they kind of operate on a "throw a bunch of shit at the wall and see what sticks" philosophy. The only real way out is to migrate to software that's still being actively developed, which I'm trying to do with phpBB. If anyone has any thoughts on this, I'd love to hear to them. If you want server access, send me your RSA public keys, and I'll add you to authorized_hosts. If you want admin access on the phpBB test site, register at the link I posted above, and message me your username, and I'll make you an admin. If you want to look at the forum databases, I'm running a virtual machine of the XMB server within the phpBB server, and you can access both MySQL databases via TCP/IP if you're logged in. I have it set up so I can easily pull data in from both databases via a Ruby console. Like, to show data for a random user, you can type XMB.members.random, or PHPBB.users.random, respectively. I would really like some help with the PHP part, since my own PHP skills are severely lacking. I've gotten this far mainly by using Ruby and relying on my decent SQL background. Most of the rest of what's left is just annoyances. Like the fact that [size] and [attachment] tags work differently in phpBB. I'm not sure whether to try and script the conversion of XMB tags to their phpBB versions (something I did already with the [rquote] tags) or try and make the XMB bbcode tags work as they are.

Right now, the goal is to get it to a point where we like how it looks, then set up a new board with all the settings imported from the one I'm working on now. Presumably run by Polverone. Then use the tools we've developed to transfer the data over.

XMB hasn't been actively developed since 2009, and I'm pretty sure we had a consensus a long time ago that we're going to have to transition to new software if we ever want to address this problem. Correct me if I'm wrong.


Yes, I know very well how SPAM bots work in a normal manner, but what I'm saying is that how can you ever tell what it is doing? I've never seen spam posted like on this board, anywhere, where 5,000 character posts are repeatedly posted day after day with new (or slightly altered) content. To me, it looks like coded messages and unless you've dealt with that before, I doubt you would understand what to look for - and THAT IS what it looks like to me.

If there is anything I or anyone else can do to help, let us know
View user's profile View All Posts By User
WGTR
International Hazard
*****




Posts: 846
Registered: 29-9-2013
Location: Online
Member Is Offline

Mood: Outline

[*] posted on 24-9-2018 at 17:08


I just went to send a U2U, and was informed that I have to delete some messages, as I've exceeded my limit of 5000. I have something like 5,600 messages in my outbox.

Apparently when I report spam, a message gets sent to every admin, every time...and they don't get automatically deleted.

Now, since I have a few dozen sent messages that I'd like to keep, I have to go into my outbox and manually look through all 5,600 of these spam reports so that I don't accidentally delete something important. Sigh.




View user's profile View All Posts By User
j_sum1
Administrator
********




Posts: 4604
Registered: 4-10-2014
Location: Oz
Member Is Offline

Mood: Metastable, and that's good enough.

[*] posted on 24-9-2018 at 18:02


Quote: Originally posted by WGTR  
I just went to send a U2U, and was informed that I have to delete some messages, as I've exceeded my limit of 5000. I have something like 5,600 messages in my outbox.

Apparently when I report spam, a message gets sent to every admin, every time...and they don't get automatically deleted.

Now, since I have a few dozen sent messages that I'd like to keep, I have to go into my outbox and manually look through all 5,600 of these spam reports so that I don't accidentally delete something important. Sigh.

Set up a new folder for U2Us you want to keep. Scroll through and click everything that is not a spam report and then move to your new folder. Once done you can delete everything else.
It is a bit of a pain but it won't actually take too long. I did the same thing when I took on the mod role.
View user's profile View All Posts By User
WGTR
International Hazard
*****




Posts: 846
Registered: 29-9-2013
Location: Online
Member Is Offline

Mood: Outline

[*] posted on 24-9-2018 at 18:17


Huh. I didn't know you could add folders like that, but I figured it out. Thanks! It looks like that's exactly what I needed.



View user's profile View All Posts By User
 Pages:  1  ..  18    20    22  ..  28

  Go To Top