mobomo/profanity_filter

A Rails plugin gem for filtering out profanity.

The Profanity Filter for Rails¶ ↑

This plugin will allow you to filter profanity using basic replacement or a dictionary term.

Disclaimer¶ ↑

This plugin is provided as is - therefore, the creators and contributors of this plugin are not responsible for any damages that may result from it’s usage. Use at your own risk; backup your data.

Install¶ ↑

gem install profanity_filter
or
./script/plugin install git://github.com/intridea/profanity_filter.git

Example¶ ↑

You can use it in your models:¶ ↑

Notice – there are two profanity filters, one is destructive. Beware the exclamation point (profanity_filter!).

Non-Destructive (filters content when called, original text remains in the database)

profanity_filter :foo, :bar
  #=> banned words will be replaced with @#=>$%

profanity_filter :foo, :bar, :method => 'dictionary'
  #=> banned words will be replaced by value in config/dictionary.yml

profanity_filter :foo, :bar, :method => 'vowels'
  #=> banned words will have their vowels replaced

profanity_filter :foo, :bar, :method => 'hollow'
  #=> all letters except the first and last will be replaced

profanity_filter :foo, :bar, :method => 'stars'
  #=> all letters will be replaced with *

The non-destructive profanity_filter provides different versions of the filtered attribute:
  some_model.foo => 'filtered version'
  some_model.foo_original => 'non-filtered version'

Destructive (saves the filtered content to the database)

profanity_filter! :foo, :bar
  #=> banned words will be replaced with @#=>$%

profanity_filter! :foo, :bar, :method => 'dictionary'
  #=> banned words will be replaced by value in config/dictionary.yml

profanity_filter! :foo, :bar, :method => 'vowels'
  #=> banned words will have their vowels replaced

profanity_filter! :foo, :bar, :method => 'hollow'
  #=> all letters except the first and last will be replaced

profanity_filter! :foo, :bar, :method => 'stars'
  #=> all letters will be replaced with *

You can also use the filter directly:¶ ↑

ProfanityFilter::Base.clean(text)
ProfanityFilter::Base.clean(text, 'dictionary')
ProfanityFilter::Base.clean(text, 'vowels')
ProfanityFilter::Base.clean(text, 'hollow')

ProfanityFilter::Base.profane?(text) #=> true/false

Benchmarks¶ ↑

Inquiring minds can checkout the simple benchmarks I’ve included so you can have an idea of what kind of performance to expect. I’ve included some quick scenarios including strings of (100, 1000, 5000, 1000) words and dictionaries of (100, 1000, 5000, 25000, 50000, 100000) words.

You can run the benchmarks via:

ruby test/benchmark/fu-fu_benchmark.rb

TODO¶ ↑

• May break ProfanityFilter out on it’s own

• Clean up dictionary implementation and substitution (suboptimal and messy)

• Move benchmarks into a rake task

• Build out rdocs

• Ability to supplement the profanity database (with a yaml outside of the gem) via @seankibler

• Easy custom blacklists/dictionaries (essentially the same as above)

License¶ ↑

The Profanity Filter for Rails uses the MIT License. Please see the MIT-LICENSE file.

Contributors¶ ↑

Created by Adam Bair ([email protected]) of Intridea (www.intridea.com) in the open source room at RailsConf 2008. Originally called Fu-fu: The Profanity Filter for Rails.

• Paul Cortens and Greg Benedict - profane? method and tests.

• Neil Ang - punctionation delimited profanity

• Flinn Mueller - dynamic word list from method

• Scott Stewert - additional dictionary words

• Nola Stowe - added stars filter

• Nick Wientge - removed dependency on ActiveRecord (now ORM agnostic)

• Jean Regisser - lazily load the dictionary

profanity_filter's Issues

"He'll" being marked as profane word

Does it make sense for he'll to be marked as a profane word?

profane? method returns true only on exact matches, but not on strings whose substrings are profane

Example:

ruby-1.9.2-p0 > ProfanityFilter::Base.profane? "shit"
=> true

ruby-1.9.2-p0 > ProfanityFilter::Base.profane? "oh shit"
=> false

substrings issue

I'm sorry to bring up substrings again, but it's still an issue. For example, try the word "f__kwit" or "a__f__k" (censoring mine). So it's not really a matter of finding it in a string like "oh s**t" but within words itself.

Fairly important default blacklist value

I realize I can customize my own blacklist, but "nigger" wasn't filtered by this by default. I highly suggest you throw it in your default dictionary

New language_filter gem

Just wanted to give a heads up that I was looking at this and other Ruby profanity filters for work and wasn't finding what I needed, so ended up rolling my own language_filter gem, which works much better for my needs, including:

• The ability to create multiple, independently configured filter instances
• Simpler configuration
• Multiple pre-prackaged matchlists (i.e. blacklists) for language type discrimination (hate, profanity, sex, and violence)
• More robust exceptionlist (i.e. whitelist) handling
• And more in the gem's README...

It doesn't include active model integration yet, so instead of profanity_filter :foo, :bar in your Rails model, you'd currently have to have something like:

before_save :clean_up_language

def clean_up_language
  filter = LanguageFilter::Filter.new matchlist: :profanity, replacement: :stars
  foo = filter.sanitize(foo)
  bar = filter.sanitize(bar)
end

So there's still a lot of room for extensibility that I just haven't had time for or had use for yet, but I think it's a more flexible and robust base to work off for most use cases. Hope others find it useful. :-)

(Oh - and development was a bit rushed, so it wasn't test-driven unfortunately. I'll be writing a test suite soon.)