datamade / openness-project-nmid Goto Github PK
View Code? Open in Web Editor NEWMoney Trail NM - New Mexico In Depth's Campaign Finance Explorer
Home Page: https://moneytrailnm.com
Money Trail NM - New Mexico In Depth's Campaign Finance Explorer
Home Page: https://moneytrailnm.com
If the last filing is a "final filing", I think that's a good indicator that the the candidate/pac does not have any active committees.
Maybe we want to surface this?
We should have a view of people who are actively running for office right now. Relates to #56
For many candidates we have the full history of the campaign contributions. When have this, let's start the net funds from 0 instead of after their first filing.
Here's an example of campaign where we see the first filing. https://nmid.datamade.us/candidates/sarah-maestas-barnes-3080/
Not exactly sure where the 0 should be place on the timeline. Some options
The current home page is pretty simple and lifeless and also derivative of Illinois Sunshine.
I'd like to try out adding an image with some text on the home page similar to https://chicago.councilmatic.org/
Here's a few good pictures I've found of the NM State Capitol:
at /donations
we should show a list of donations, reported in chronological order similar to http://illinoissunshine.org/donations/
Since it is possible for a candidate to have more than one active committee at a time, let's make sure that we show all of the relevant latest filings. To accomplish this we'll:
If there is more than one filing for the most recent reporting period, show all of them.
Also show the most recent report for all non-final committees.
On the listing of all candidates, we show David Saline having a current balance of $0.00
But on his detail page, https://nmid.datamade.us/candidates/david-saline-3163/
He has 84 dollars in his net funds.
Turn on caching for 💯 performance!!!!
Create an enhanced about page that's more FAQ like.
Fish created here
Still needs editing/approval by Trip and Marjorie, suggestions welcome from Datamade folk
Lets add a new website section and on the home page for top earners, modeled on http://www.illinoissunshine.org/top-earners/
We might want to filter out contact information that are from out of date campaigns
For example, Brian Morris shows three times on this Maggie Stebbins page:
Leaving this here to remind me to do that so that the whole deployment pipeline will work.
CSV downloads should be made available for
On a few pages I've checked, including https://nmid.datamade.us/candidates/hector-balderas-3046/
Doesn't seem to be working for
Name: Firefox
Version: 48.0
Build ID: 20160728203720
User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:48.0) Gecko/20100101 Firefox/48.0
OS: Linux 4.4.0-36-generic x86-64
Multiprocess Windows: 0/2 (Disabled by add-ons)
Safe Mode: false
Ranking these donations for arbitrary time windows doesn't make a lot of sense. I think we should hide them.
It could be very nice to have some reference lines in the visualisations to indicate if a person has a lot or a little amount of money for the race they are running for.
Not exactly sure what the right reference point would be. Here are some ideas
It looks like we need to do some recoding of the variables in the election results table to make them human readable:
For example: https://nmid.datamade.us/candidates/maggie-stebbins-2386/
What does 3 and 1 mean?
It looks like the "contact type" table has some interesting information about what kind of organization is giving money to a candidate or committee or where the candidate or committee is spending money. It might be cool to show totals by election cycle for each of these organization types in a chart on the detail pages. The types that we have are:
State Party Committee
Legislative District Party Committee
Financial Institution
Minor Party Committee
Food/Catering Services
Business
Telecommunication
Office Supplies
Other
County Party Committee
Political Action Committee
Individual
Professional Services
Advertising
Legal
Native American Tribe
Union
let's rank each candidate and committee by the amount of money they have on hand. this should be calculated during the ETL update and stored in the database.
https://nmid.datamade.us/committees/patriot-majority-new-mexico-412/
Seems related to changes we made in how we handled debt.
The listing that we are calling candidates are not really candidates, because we are listing lots of people who are not running for anything, right now. This should be called something like "Politicians" instead.
See https://nmid.datamade.us/candidates/stephen-french-3699/
Relates to #34
When we're deploying changes, etc, we should stick the app into maintenance mode. There seems to be an OK django package for that here: https://github.com/fabiocaccamo/django-maintenance-mode
These pages seem to be very similar, despite being about different things.
Currently, the only difference I can see between them is the candidate page has a separate list of offices the person has run for. If this is the case, it would be much more DRY to merge these templates into one.
Right now we're just adding up everything for all time to create these numbers. Lets group them by election cycle as well so that you get top transactions by cycle.
This campaign has information from before 2010
https://nmid.datamade.us/committees/albuquerque-teachers-federation-93/
This pre-2010 info should be excluded, right?
It appears you're linking to a search of people when i click on a treasurer but with Thomas Lawley of CVNM Verde Voters Fund it doesn't show any treasurer things for him.
Same with other treasurer's clicked on for PAC. Some clicking reveals it's just a problem with them, it appears, not with candidate treasurers.
Site looks pretty bad on phone
Home page has been stubbed out with the following sections:
Any others we should add?
I'm going to start with just implementing the things that we had for Illinois Sunshine. If there are any new things that we want to add or old things to modify, lets discuss.
Create a widget for top earners (both candidates and committees) in the last 90 days.
IndexView – with its plethora of queries – loads slowly. Do some caching?
Looking at an older, closed PAC and when i click on largest or all donations getting a spinning wheel. There are no donations shown under the top 25, either, so presumably all donations are pre-2010? Same goes for some other old committees.
Importing Candidates.csv
Found 3374 records in Candidates.csv
Updated 0 records in camp_fin_candidate
Traceback (most recent call last):
File "/home/fgregg/public/openness-project-nmid/.env/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
context)
File "/home/fgregg/public/openness-project-nmid/.env/lib/python3.5/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
cursor.execute(statement, parameters)
psycopg2.IntegrityError: null value in column "full_name" violates not-null constraint
DETAIL: Failing row contains (57, null, A., B., Valdez, null, null, (505) 753-4018, 2010-01-01 00:00:00-06, dvaldez@cybermesa.com, null, null, 0, 3149, 2478, 1824, 1, null, null).
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "manage.py", line 22, in <module>
execute_from_command_line(sys.argv)
File "/home/fgregg/public/openness-project-nmid/.env/lib/python3.5/site-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
utility.execute()
File "/home/fgregg/public/openness-project-nmid/.env/lib/python3.5/site-packages/django/core/management/__init__.py", line 359, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/fgregg/public/openness-project-nmid/.env/lib/python3.5/site-packages/django/core/management/base.py", line 305, in run_from_argv
self.execute(*args, **cmd_options)
File "/home/fgregg/public/openness-project-nmid/.env/lib/python3.5/site-packages/django/core/management/base.py", line 356, in execute
output = self.handle(*args, **options)
File "/home/fgregg/public/openness-project-nmid/camp_fin/management/commands/import_data.py", line 112, in handle
self.doETL(entity_type)
File "/home/fgregg/public/openness-project-nmid/camp_fin/management/commands/import_data.py", line 162, in doETL
self.addNewRecords()
File "/home/fgregg/public/openness-project-nmid/camp_fin/management/commands/import_data.py", line 520, in addNewRecords
self.executeTransaction(insert_new)
File "/home/fgregg/public/openness-project-nmid/camp_fin/management/commands/import_data.py", line 668, in executeTransaction
self.connection.execute(query, *args)
File "/home/fgregg/public/openness-project-nmid/.env/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 906, in execute
return self._execute_text(object, multiparams, params)
File "/home/fgregg/public/openness-project-nmid/.env/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1054, in _execute_text
statement, parameters
File "/home/fgregg/public/openness-project-nmid/.env/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
context)
File "/home/fgregg/public/openness-project-nmid/.env/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
exc_info
File "/home/fgregg/public/openness-project-nmid/.env/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/home/fgregg/public/openness-project-nmid/.env/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 185, in reraise
raise value.with_traceback(tb)
File "/home/fgregg/public/openness-project-nmid/.env/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
context)
File "/home/fgregg/public/openness-project-nmid/.env/lib/python3.5/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) null value in column "full_name" violates not-null constraint
DETAIL: Failing row contains (57, null, A., B., Valdez, null, null, (505) 753-4018, 2010-01-01 00:00:00-06, dvaldez@cybermesa.com, null, null, 0, 3149, 2478, 1824, 1, null, null).
[SQL: ' \n INSERT INTO camp_fin_candidate (\n id, entity_id, prefix, first_name, last_name, middle_name, suffix, business_phone, home_phone, address_id, status_id, date_added, contact_id, email, date_updated, qual_candidate_id, deceased\n )\n SELECT raw."candidateid"::bigint AS id, raw."entityid"::bigint AS entity_id, raw."prefix"::varchar AS prefix, raw."firstname"::varchar AS first_name, raw."lastname"::varchar AS last_name, raw."middlename"::varchar AS middle_name, raw."suffix"::varchar AS suffix, raw."businessphone"::varchar AS business_phone, raw."homephone"::varchar AS home_phone, raw."addressid"::bigint AS address_id, raw."statusid"::bigint AS status_id, raw."dateadded"::date AS date_added, raw."contactid"::bigint AS contact_id, raw."emailaccount"::varchar AS email, raw."datelastupdated"::timestamp with time zone AS date_updated, raw."qualcandidateid"::bigint AS qual_candidate_id, raw."deceased"::varchar AS deceased\n FROM raw_candidate AS raw\n JOIN new_candidate AS new\n ON raw."candidateid" = new.id\n ']
When i'm looking at Gov. Martinez's page, the top donors for 2014 are ranked (the first year she had donation limits). But if click on largest donations, it gets those big $200K things from 2010. It seems like they should be in the top 25?
clicking on this about link instead gets to a donations page
also wondering if there should be links to donors on the donations page, since there are links on candidate and PAC pages?
We are showing Mr. Martinez as having an opening balance of $0.00
https://nmid.datamade.us/candidates/richard-martinez-1349/
This seems to be contradicted by the visualisations.
Same for Mr. Balderas.
NM's Future Leaders is shown with a $149.26 balance here but a zero balance here
same with this PAC
PAC West
this guy
this group
anyway if you want a longer list i can do that - i suspect it's something to do with substituting the total expenditures for closing balance when the latter is 0, though it doesn't appear to be the case with all the 0 balance committees.
It would seem that anything before 2010 is not reliable. Let's hide it.
Some examples:
analytics_lib.js
file)<meta name=“description”>
<meta name=“author”>
<meta property=“og:site_name”>
<meta property=“og:title”>
<meta property=“og:type”>
<meta property=“og:description”>
<meta property=“og:url”>
<meta property=“og:image”>
<meta name=“twitter:card”>
<meta name=“twitter:site”>
<meta name=“twitter:creator”>
<meta name=“twitter:description”>
(note that this needs to be under 200 characters)<meta name=“twitter:title”>
<meta name=“twitter:url”>
<meta name=“twitter:image:src”>
Test on various mobile devices:
If your site relies on a database or server-side code, it should use caching and be load tested. If it's a static HTML or Jekyll site, you can skip this section.
If the site is open source, make sure the Readme.md is complete and accurate.
Here's a few good examples:
The Readme should have the following sections:
Let's setup an FTP server for the NM Secretary of State to upload their data.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.