Comments (36)
In browser i also see below error :
API unaccessible. HTTP 502:
<title>502 Bad Gateway</title>502 Bad Gateway
nginx/1.4.6 (Ubuntu)
from security_monkey.
Go into the supervisor directory and run:
sudo supervisorctl -c security_monkey.ini
When you get the supervisor prompt, run "status" to see what is running. There should be two things running: "securitymonkey" and "securitymonkeyscheduler".
The "securitymonkey" entry is the Flask application that should be listening on port 5000. You can restart that process and then run status a few times to see if it worked.
> restart securitymonkey
> status
Most likely, the flask application is having difficulty starting. There should be an error message somewhere in /tmp/securitymonkey.log
I've seen problems where the flask application can't find the SECURITY_MONKEY_SETTINGS environment variable that is suppose to point to env-config/config-deploy.py
from security_monkey.
Thanks for pointing to right direction. Now i see the error in /tmp/securitymonkey.log saying :
DEBG 'securitymonkey' stderr output: variable_name)
RuntimeError: The environment variable 'SECURITY_MONKEY_SETTINGS' is not set and as such configuration could not be loaded. Set this variable and make it point to a configuration file
What i did is -
$ export SECURITY_MONKEY_SETTINGS=/srv/security_monkey/env-config/config-deploy.py
$ python manage.py db upgrade
and restarted the service, But again getting same error. Should i require to set this in any config file ?
I can also confirm that variable is set as per below output -
$ echo $SECURITY_MONKEY_SETTINGS
/srv/security_monkey/env-config/config-deploy.py
What else am i missing ?
from security_monkey.
Same path is mentioned in "security_monkey.ini" file.
directory=/srv/security_monkey/
environment=PYTHONPATH='/srv/security_monkey/',SECURITY_MONKEY_SETTINGS="/srv/security_monkey/env-config/config-deploy.py"
from security_monkey.
Solved -
I couldn't figured out the issue, but tried an workaround by creating a symlink of security_monkey in /home/ubuntu/ and restarted the services, It worked.
from security_monkey.
I'm glad you were able to fix it. I'll keep tracking this as I've seen it a few times.
If you have the variable in your environment, maybe you should start supervisor with sudo -E instead of just sudo:
sudo -E supervisorctl -c security_monkey.ini
That doesn't make a lot of sense though since the variable should be getting set in security_monkey.ini.
from security_monkey.
FYI - I just merged a pull request that may help with this issue: #25
from security_monkey.
Hi,
I am trying to setup SecurityMonkey for production environment, but facing more issue since last time.
- Email not working. SES is enabled for my account. Getting below error -
2014-07-29 07:27:38,212 INFO: Failed to send failure message: <type 'exceptions.Exception'> SESAddressNotVerifiedError: 400 Email address is not verified.
Sender
MessageRejected
Email address is not verified.
cc923280-16f1-11e4-9f40-135437038bc3
[in /secmonkey/security_monkey/security_monkey/init.py:83]
Should it require to use the ses endpoint and credentials somewhere ?
- securitymonkeyscheduler is not running.
supervisor> status
securitymonkey RUNNING pid 3883, uptime 0:00:26
securitymonkeyscheduler FATAL Exited too quickly (process log may have details)
2014-07-29 08:08:12,356 INFO exited: securitymonkeyscheduler (exit status 0; not expected)
2014-07-29 08:08:12,356 DEBG received SIGCLD indicating a child quit
2014-07-29 08:08:13,357 INFO gave up: securitymonkeyscheduler entered FATAL state, too many start retries too quickly - Forgot password opens 404 page.
from security_monkey.
Hey skdubey,
SES can be a pain to setup. I think you'll need to head into the console to setup/verify your email address.
http://docs.aws.amazon.com/ses/latest/DeveloperGuide/ses-errors.html
For the scheduler, could you backup your configurations try the develop branch?
$ git checkout develop
Is the 404 an HTTP vs HTTPS issue?
Thanks for the feedback. I want to get these things fixed soon so others don't run into them.
from security_monkey.
Hello Patrick,
I tried develop branch and still facing same issues.
- Scheduler is not starting
ubuntu@security-monkey:~/security_monkey/supervisor$ sudo -E supervisorctl -c security_monkey.ini
securitymonkey RUNNING pid 11567, uptime 0:06:19
securitymonkeyscheduler FATAL Exited too quickly (process log may have details)
supervisor> status
securitymonkey RUNNING pid 11689, uptime 0:00:16
securitymonkeyscheduler BACKOFF Exited too quickly (process log may have details)
supervisor> status
securitymonkey RUNNING pid 11689, uptime 0:00:22
securitymonkeyscheduler FATAL Exited too quickly (process log may have details) - It was https vs http but after using develop branch it still says 404 but on https only and doesn't change to http.
- While registering a new user it redirects to http page and shows 404 error, after entering the manual url including https it opens fine. I have attached the screen shot for same.
- I am able to send email through aws console, DKIM etc. is setup and running fine. How can i send you the system logs for debugging.
from security_monkey.
Hi,
Any update on this ?
from security_monkey.
I tried by re-installing everything, but facing same issue.
from security_monkey.
Hello,
I am running into the same issue. Is there any fix on this?
from security_monkey.
Finally i terminated the existing instance and started a new one with fresh setup and it worked for me this time except emails are not going through. My SES setup & configuration is correct. Can someone help me in this case ?
Where can i change the SES related configurations or does it requires ? I am in us-west-2 region, i hope it doesn't matter but just wanted to check if by default it assumes us-east-1.
from security_monkey.
I think SES got broken since the last merge to master, couldnt use SES since commit where SMTP was introduced here.
from security_monkey.
Thanks for update. I just tried by using user/pass section, restarted 'securitymonkey' service but noticed that it still using SES to send mail and not second provider. Am i missing something here ?
Mail Setting -
These are only required if using SMTP instead of SES
EMAILS_USE_SMTP = True # Otherwise, Use SES
MAIL_SERVER = 'mail.authsmtp.com'
MAIL_PORT = 2525
MAIL_USE_SSL = True
MAIL_USERNAME = 'xxxxxxxxx'
MAIL_PASSWORD = 'yyyyyyyy'
from security_monkey.
Is it fixed ? btw, what is the process to get new updates ? Is it - git pull > restart services ?
from security_monkey.
Hey skdubey. The region shouldn't matter as long as you're not in cn-north-1 or the gov region. I have instances running in us-west-1 as well. I also have this deployed in both ec2 and vpc.
I just brought up a new instance yesterday and SES is working correctly.
The mail settings that you pasted in are telling Security Monkey to use SMTP:
EMAILS_USE_SMTP = True
If you would prefer SES, that should be set to False.
Security Monkey does need a better upgrade mechanism. The Netflix way is to have the environment build and bake a new AMI for each version and replace the existing instance. (Instances at Netflix are intentionally somewhat immutable.) To do an in place upgrade, you would want to backup your custom configurations. These are probably in:
env-config/config-deploy.py
possibly supervisor/security_monkey.ini
You can run a "git status" to have git tell you which files have been modified.
Then do a git pull. Before you copy your config files back, compare them to the new config files that were just pulled down to see if you need to merge in any new settings. Then, restart the services.
I'd love to receive a PULL request for easier upgrades. It could be as simple as renaming all the config files to "config_filename"+".default" in the git repo, so that a git pull won't overwrite their settings. Could also add "*.default" to the .gitignore. The installation instructions would need to be modified slightly to instruct users to copy the ".default" files into place and then modify them.
from security_monkey.
SES works for me now, didnt notice that DEFAULT_MAIL_SENDER has been changed to MAIL_DEFAULT_SENDER
To update is kind of simple for us, I have saltstack pillar with fixed commit hash, once I change the commit hash, all the installation process.
from security_monkey.
Yeah, that's confusing. I believe Flask-Mail uses one of those and the Flask_Mail built into Flask-Security uses the other.
from security_monkey.
Hi. I've spend lots of time to create a new account ad fix the other issues, but it seems I have to address this here. I get confirmation mail using SMTP, but:
Internal Server Error
2014-08-29 15:28:48 [18285] [ERROR] Error handling request
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/gunicorn-18.0-py2.7.egg/gunicorn/workers/sync.py", line 131, in handle_request
respiter = self.wsgi(environ, resp.start_response)
File "/usr/local/lib/python2.7/dist-packages/Flask-0.10.1-py2.7.egg/flask/app.py", line 1836, in call
return self.wsgi_app(environ, start_response)
File "/usr/local/lib/python2.7/dist-packages/Flask-0.10.1-py2.7.egg/flask/app.py", line 1820, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/usr/local/lib/python2.7/dist-packages/Flask_RESTful-0.2.5-py2.7.egg/flask_restful/init.py", line 145, in error_router
return original_handler(e)
File "/usr/local/lib/python2.7/dist-packages/Flask-0.10.1-py2.7.egg/flask/app.py", line 1403, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/Flask-0.10.1-py2.7.egg/flask/app.py", line 1817, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/Flask-0.10.1-py2.7.egg/flask/app.py", line 1477, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python2.7/dist-packages/Flask_RESTful-0.2.5-py2.7.egg/flask_restful/init.py", line 145, in error_router
return original_handler(e)
File "/usr/local/lib/python2.7/dist-packages/Flask-0.10.1-py2.7.egg/flask/app.py", line 1381, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/Flask-0.10.1-py2.7.egg/flask/app.py", line 1475, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/Flask-0.10.1-py2.7.egg/flask/app.py", line 1461, in dispatch_request
return self.view_functionsrule.endpoint
File "/usr/local/lib/python2.7/dist-packages/Flask_Security-1.7.3-py2.7.egg/flask_security/decorators.py", line 205, in wrapper
return f(_args, *_kwargs)
File "/usr/local/lib/python2.7/dist-packages/Flask_Security-1.7.3-py2.7.egg/flask_security/views.py", line 122, in register
login_user(user)
File "/usr/local/lib/python2.7/dist-packages/Flask_Security-1.7.3-py2.7.egg/flask_security/utils.py", line 82, in login_user
identity=Identity(user.id))
File "build/bdist.linux-x86_64/egg/blinker/base.py", line 267, in send
for receiver in self.receivers_for(sender)]
File "/usr/local/lib/python2.7/dist-packages/Flask_Principal-0.4.0-py2.7.egg/flask_principal.py", line 469, in _on_identity_changed
self.set_identity(identity)
File "/usr/local/lib/python2.7/dist-packages/Flask_Principal-0.4.0-py2.7.egg/flask_principal.py", line 418, in set_identity
self._set_thread_identity(identity)
File "/usr/local/lib/python2.7/dist-packages/Flask_Principal-0.4.0-py2.7.egg/flask_principal.py", line 463, in _set_thread_identity
identity=identity)
File "build/bdist.linux-x86_64/egg/blinker/base.py", line 267, in send
for receiver in self.receivers_for(sender)]
File "/usr/local/lib/python2.7/dist-packages/Flask_Security-1.7.3-py2.7.egg/flask_security/core.py", line 214, in on_identity_loaded
for role in current_user.roles:
File "/usr/local/lib/python2.7/dist-packages/Werkzeug-0.9.4-py2.7.egg/werkzeug/local.py", line 338, in getattr
return getattr(self.get_current_object(), name)
File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/attributes.py", line 233, in get
return self.impl.get(instance_state(instance), dict)
File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/attributes.py", line 579, in get
value = self.callable(state, passive)
File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/strategies.py", line 478, in _load_for_state
(orm_util.state_str(state), self.key)
DetachedInstanceError: Parent instance <User at 0x7f9039903910> is not bound to a Session; lazy load operation of attribute 'roles' cannot proceed
What is wrong here?
Thanks in advance
from security_monkey.
- Despite of all the verifications SES still didn't worked for me. Error is as below :
2014-08-26 12:23:04,831 DEBUG: Failed to send failure message with subject: Welcome
<type 'exceptions.Exception'> SESAddressNotVerifiedError: 400 Email address is not verified.
Sender
MessageRejected
Email address is not verified.
b7dfc63f-2d1b-11e4-99b7-3bf0a7e96a8e
[in /home/ubuntu/security_monkey/security_monkey/common/utils/utils.py:88] - Apart from above i am also observing some strang behaviour on console. For example :
a) If i try to login to console i still see 'Anonymous' in console and no data on page, But if i click on 'Setting', I can see the added account. Not sure if it is browser cache (Firefox) issue or something else.
b) If i register a new user it redirects to /register with error page, but if i try to login with same created user, i am able to do so.
c). SMTP setting is working for me, but if set SECURITY_CONFIRMABLE & SECURITY_RECOVERABLE as True, i am not able to login to console, setting this to False works for me.
Need help on points below -
a) I don't want to register anyone just by knowing the url of console. How can i protect this ?
b) Where does all the created users for console stored? I looked at postgres db but couldb't found any.
Occassionally, i also noticed attached error.
from security_monkey.
Hey skdubey,
Try to use Chrome for the time being. I'm working on fixing an issue with Safari/Firefox.
The users are stored in a database table called "user". You should see these tables in your database:
account
alembic_version
association
item
itemaudit
itemcomment
itemrevision
itemrevisioncomment
role
role_users
technology
user
If you don't see any tables, you'll need to run python manage.py db upgrade
to have alembic/Flask-Migrate create the tables for you.
You will need to be able to receive the Welcome email and confirm the account using the link provided in the email before you can login to your account.
You can disable user registration by setting SECURITY_REGISTERABLE to false. You could then create individual users manually by adding a method to manage.py similar to the create_user() method listed here: https://pythonhosted.org/Flask-Security/quickstart.html
def create_user():
user_datastore.create_user(email='[email protected]', password='password')
db.session.commit()
That way, user registration is a process that involves the security team. I'll add a method for manual user creation to manage.py for the next release.
Your SES problems may be answered here:
http://aws.amazon.com/ses/faqs/
"Also, while all users must verify the email addresses or domains they will send from, sandbox users must also verify the email addresses or domains they will send to. The purpose of verification is to confirm that you own these email addresses or domains, and to prevent others from using them. See the Amazon SES Developer Guide for details on verifying email addresses and domains."
from security_monkey.
Hi,
Thanks a lot for your time. Please bear with me for some more queires.
- Chrome worked but it requires 2-3 browser refresh to load the data, some times it take more than that. But better than Firefox.
- Setting SECURITY_REGISTERABLE to false worked.
- I tried that but not able to see anything except db user. There are 5 users who are able to access console but in db it doesn't shows anything.
- Yes, i received a welcome email which doesn't have url to confirm but email id as below.
from security_monkey.
skdubey: just ran into that postgres thing today, try:
\c secmonkey;
select * from public.user;
I think user is a postgres table as well and that gets hit without the schema name.
from security_monkey.
Thanks gbarton,
Now i see the list of users, but couldn't able to find the meaning of
column 'active = t', 'confirmed_at = ', 'daily_audit_email = t/f ' and
'change_reports = ALL'. Where can i find these information and will it
affect if i change the value in db directly ?
On Tue, Sep 9, 2014 at 1:02 AM, gbarton [email protected] wrote:
skdubey: just ran into that postgres thing today, try:
\c secmonkey;
select * from public.user;I think user is a postgres table as well and that gets hit without the
schema name.—
Reply to this email directly or view it on GitHub
#18 (comment)
.
from security_monkey.
Ah!, sorry
Very simple, i realized it later : t = True and f = False and "change_reports = ALL" means all the changes should be notified to the particular users.
from security_monkey.
Thanks gbarton for helping out. I was stumped on the user table problem.
Skdubey, how are you doing with your install now? Can this issue be closed or are you still running into issues?
from security_monkey.
Hello Patrick,
I still didn't get the welcome email part, where confirmation is required.
Can some send me the sample email ? I created multiple users but couldn't
get the confirmation link to click on, but i do get the welcome email.
Also curious to know the status of enhancements like, report should be
downloadable in .csv or .pdf format from console.
On Tue, Sep 9, 2014 at 11:36 AM, Patrick Kelley [email protected]
wrote:
Thanks gbarton for helping out. I was stumped on the user table problem.
Skdubey, how are you doing with your install now? Can this issue be closed
or are you still running into issues?—
Reply to this email directly or view it on GitHub
#18 (comment)
.
from security_monkey.
Hi skdubey
I managed to have it all running although no confirmation mail.
I had the same issue with the confirmation mail, as you had, which now is fixed. Once more, make sure that you have full production for SES or confirm in SES email addresses/domains.
What happened for me: I am running in eu-west, and the defaults in boto ses are for us-east. After changing the defaults in
/usr/local/lib/python2.7/dist-packages/boto-2.32.1-py2.7.egg/boto/ses/connection.py
class SESConnection(AWSAuthConnection):
ResponseError = BotoServerError
DefaultRegionName = 'eu-west-1'
DefaultRegionEndpoint = 'email.eu-west-1.amazonaws.com'
everything worked and I could see mails comming.
Also, after getting the welcome mail form registering, if I look in the USER table, all the users are marked as ACTIVE (True) .
This is my 2 cents
from security_monkey.
Thanks vgomolu,
You gave me some hope, because i am running in us-west-2 region while the
Default is us-east-1. As per your update i changed the same file update it
to us-west-2 and restarted the both the services (securitymonkey &
schedular) with no luck, But found below error which id different than
earlier. Earlier i was getting that "your email is not verified" and now i
am getting this : "<type 'exceptions.Exception'> :1:0: syntax
error". Below is the actual output -
2014-09-09 12:30:38,888 DEBG 'securitymonkeyscheduler' stderr output:
Failed to send failure message with subject: [Accountname] Changes in
securitygroup
<type 'exceptions.Exception'> :1:0: syntax error
One more observation i have is, in my system there are two python instances
at /usr/local/lib/{python2.7 & python3.4}. I made changes in 2.7 as there
is nothing under "/usr/local/lib/python3.4/dist-packages/".
On Tue, Sep 9, 2014 at 5:34 PM, vgomoiu [email protected] wrote:
Hi skdubey
I managed to have it all running although no confirmation mail.
I had the same issue with the confirmation mail, as you had, which now is
fixed. Once more, make sure that you have full production for SES or
confirm in SES email addresses/domains.What happened for me: I am running in eu-west, and the defaults in boto
ses are for us-east. After changing the defaults in/usr/local/lib/python2.7/dist-packages/boto-2.32.1-py2.7.egg/boto/ses/connection.py
class SESConnection(AWSAuthConnection):
ResponseError = BotoServerError
DefaultRegionName = 'eu-west-1'
DefaultRegionEndpoint = 'email.eu-west-1.amazonaws.com'everything worked and I could see mails comming.
Also, after getting the welcome mail form registering, if I look in the
USER table, all the users are marked as ACTIVE (True) .This is my 2 cents
—
Reply to this email directly or view it on GitHub
#18 (comment)
.
from security_monkey.
skdubey, I had exactly that error wehn first time I've updated the connection.py with the SES SMTP endpoint instead of API endpoint; so for you should be
DefaultRegionName = 'us-west-2'
DefaultRegionEndpoint = 'email.us-west-2.amazonaws.com'
Please make sure it looks like above.
Also, you are correct, the update is only for python 2.7
Also check config-deploy.py and make sure you have to correct format.
from security_monkey.
Thanks a lot vgomoiu to make this working. It is working with SES now.
I have one more point - What should be the lowest check interval (default
is 10 min.) and how can i change that value ?
On Tue, Sep 9, 2014 at 6:27 PM, vgomoiu [email protected] wrote:
skdubey, I had exactly that error wehn first time I've updated the
connection.py with the SES SMTP endpoint instead of API endpoint; so for
you should beDefaultRegionName = 'us-west-2'
DefaultRegionEndpoint = 'email.us-west-2.amazonaws.com'Please make sure it looks like above.
Also, you are correct, the update is only for python 2.7
Also check config-deploy.py and make sure you have to correct format.—
Reply to this email directly or view it on GitHub
#18 (comment)
.
from security_monkey.
Also, the "Forgot Password" is throwing 404 error and redirecting to url #/reset.
from security_monkey.
Glad to hear it's working.
For the check interval:
/home/ubuntu/security_monkey/security_monkey/init.py
and find the "def setup_scheduler"
The SECURITY_CONFIRMABLE = False and SECURITY_RECOVERABLE = False don't seem to be implemented yet.
from security_monkey.
Again, thanks a lot vgomoiu.
You have been very helpful to me.
On Wed, Sep 10, 2014 at 12:59 PM, vgomoiu [email protected] wrote:
Glad to hear it's working.
For the check interval:
/home/ubuntu/security_monkey/security_monkey/init.py
and find the "def setup_scheduler"The SECURITY_CONFIRMABLE = False and SECURITY_RECOVERABLE = False don't
seem to be implemented yet.—
Reply to this email directly or view it on GitHub
#18 (comment)
.
from security_monkey.
Related Issues (20)
- not getting description of Unknown issue HOT 2
- Problemas al ejecutar pub get
- Error al ejecutar Pub build HOT 2
- Ubuntu- Create a Self Signed SSL Certificate
- error when running monkey db upgrade HOT 4
- Google SSO Crashes on /api/1/auth/providers HOT 1
- botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the AssumeRole operation: Access denied HOT 11
- Hardcoded reference to Riot Games HOT 4
- No way to see what we are about to justify
- monkey db upgrade fails HOT 2
- ALB/ELB scans are not working HOT 4
- monkey db upgrade - issues HOT 5
- frequent null value in column "item_id" violates not-null constraint exception HOT 4
- monkey db upgrade is throwing error HOT 1
- monkey db upgrade issue is still there HOT 1
- Python 3 inconsistent list ordering causing frequent change detections
- pkg_resources.DistributionNotFound: The 'octodns' distribution was not found and is required by the application HOT 1
- How to Generate a change item email HOT 11
- Where to Submit a Security Vulnerability
- How to run SecMonkey locally HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from security_monkey.