Giter Site home page Giter Site logo

yogeshojha / rengine Goto Github PK

View Code? Open in Web Editor NEW
6.7K 142.0 1.0K 254.04 MB

reNgine is an automated reconnaissance framework for web applications with a focus on highly configurable streamlined recon process via Engines, recon data correlation and organization, continuous monitoring, backed by a database, and simple yet intuitive User Interface. reNgine makes it easy for penetration testers to gather reconnaissance with minimal configuration and with the help of reNgine's correlation, it just makes recon effortless.

Home Page: https://yogeshojha.github.io/rengine/

License: GNU General Public License v3.0

Dockerfile 0.22% Python 37.42% CSS 3.92% JavaScript 20.53% HTML 36.69% Shell 0.98% Makefile 0.13% Batchfile 0.10%
security-tools osint recon recon-engine reconnaissance scanner scanner-web rengine information-gathering bugbounty

rengine's Introduction

reNgine Latest Version License   

             

  

 

Open Source Security Index - Fastest Growing Open Source Security Projects

reNgine 2.0-jasper
Redefining the future of reconnaissance!

What is reNgine?

reNgine is your go-to web application reconnaissance suite that's designed to simplify and streamline the reconnaissance process for security professionals, penetration testers, and bug bounty hunters. With its highly configurable engines, data correlation capabilities, continuous monitoring, database-backed reconnaissance data, and an intuitive user interface, reNgine redefines how you gather critical information about your target web applications.

Traditional reconnaissance tools often fall short in terms of configurability and efficiency. reNgine addresses these shortcomings and emerges as a excellent alternative to existing commercial tools.

reNgine was created to address the limitations of traditional reconnaissance tools and provide a better alternative, even surpassing some commercial offerings. Whether you're a bug bounty hunter, a penetration tester, or a corporate security team, reNgine is your go-to solution for automating and enhancing your information-gathering efforts.

reNgine 2.0-jasper is out now, you can watch reNgine 2.0-jasper release trailer here!

reNgine 2.0-Jasper would not have been possible without @ocervell valuable contributions. @ocervell did majority of the refactoring if not all and also added a ton of features. Together, we wish to shape the future of web application reconnaissance, and it's developers like @ocervell and a ton of other developers and hackers from our community who inspire and drive us forward.

Thank you, @ocervell, for your outstanding work and unwavering commitment to reNgine.

Checkout our contributers here: Contributers

-----------------------------------------------------

Documentation

You can find detailed documentation at https://rengine.wiki

-----------------------------------------------------

Table of Contents

-----------------------------------------------------

About reNgine

reNgine is not an ordinary reconnaissance suite; it's a game-changer! We've turbocharged the traditional workflow with groundbreaking features that is sure to ease your reconnaissance game. reNgine redefines the art of reconnaissance with highly configurable scan engines, recon data correlation, continuous monitoring, GPT powered Vulnerability Report, Project Management and role based access control etc.

🦾   reNgine has advanced reconnaissance capabilities, harnessing a range of open-source tools to deliver a comprehensive web application reconnaissance experience. With it's intuitive User Interface, it excels in subdomain discovery, pinpointing IP addresses and open ports, collecting endpoints, conducting directory and file fuzzing, capturing screenshots, and performing vulnerability scans. To summarize, it does end-to-end reconnaissance. With WHOIS identification and WAF detection, it offers deep insights into target domains. Additionally, reNgine also identifies misconfigured S3 buckets and find interesting subdomains and URLS, based on specific keywords to helps you identify your next target, making it an go to tool for efficient reconnaissance.

🗃️    Say goodbye to recon data chaos! reNgine seamlessly integrates with a database, providing you with unmatched data correlation and organization. Forgot the hassle of grepping through json, txt or csv files. Plus, our custom query language lets you filter reconnaissance data effortlessly using natural language like operators such as filtering all alive subdomains with http_status=200 and also filter all subdomains that are alive and has admin in name http_status=200&name=admin

🔧   reNgine offers unparalleled flexibility through its highly configurable scan engines, based on a YAML-based configuration. It offers the freedom to create and customize recon scan engines based on any kind of requirement, users can tailor them to their specific objectives and preferences, from thread management to timeout settings and rate-limit configurations, everything is customizable. Additionally, reNgine offers a range of pre-configured scan engines right out of the box, including Full Scan, Passive Scan, Screenshot Gathering, and the OSINT Scan Engine. These ready-to-use engines eliminate the need for extensive manual setup, aligning perfectly with reNgine's core mission of simplifying the reconnaissance process and enabling users to effortlessly access the right reconnaissance data with minimal effort.

💎  Subscans: Subscan is a game-changing feature in reNgine, setting it apart as the only open-source tool of its kind to offer this capability. With Subscan, waiting for the entire pipeline to complete is a thing of the past. Now, users can swiftly respond to newfound discoveries during reconnaissance. Whether you've stumbled upon an intriguing subdomain and wish to conduct a focused port scan or want to delve deeper with a vulnerability assessment, reNgine has you covered.

📃   PDF Reports: In addition to its robust reconnaissance capabilities, reNgine goes the extra mile by simplifying the report generation process, recognizing the crucial role that PDF reports play in the realm of end-to-end reconnaissance. Users can effortlessly generate and customize PDF reports to suit their exact needs. Whether it's a Full Scan Report, Vulnerability Report, or a concise reconnaissance report, reNgine provides the flexibility to choose the report type that best communicates your findings. Moreover, the level of customization is unparalleled, allowing users to select report colors, fine-tune executive summaries, and even add personalized touches like company names and footers. With GPT integration, your reports aren't just a report, with remediation steps, and impacts, you get 360-degree view of the vulnerabilities you've uncovered.

🔖    Say Hello to Projects! reNgine 2.0 introduces a powerful addition that enables you to efficiently organize your web application reconnaissance efforts. With this feature, you can create distinct project spaces, each tailored to a specific purpose, such as personal bug bounty hunting, client engagements, or any other specialized recon task. Each projects will have separate dashboard and all the scan results will be separated from each projects, while scan engines and configuration will be shared across all the projects.

⚙️    Roles and Permissions! Begining reNgine 2.0, we've taken your web application reconnaissance to a whole new level of control and security. Now, you can assign distinct roles to your team members—Sys Admin, Penetration Tester, and Auditor—each with precisely defined permissions to tailor their access and actions within the reNgine ecosystem.

  • 🔐 Sys Admin: Sys Admin is a super user that has permission to modify system and scan related configurations, scan engines, create new users, add new tools etc. Super user can initiate scans and subscans effortlessly.
  • 🔍 Penetration Tester: Penetration Tester will be allowed to modify and initiate scans and subscans, add or update targets, etc. A penetration tester will not be allowed to modify system configurations.
  • 📊 Auditor: Auditor can only view and download the report. An auditor can not change any system or scan related configurations nor can initiate any scans or subscans.

🚀   GPT Vulnerability Report Generation: Get ready for the future of penetration testing reports with reNgine's groundbreaking feature: "GPT-Powered Report Generation"! With the power of OpenAI's GPT, reNgine now provides you with detailed vulnerability descriptions, remediation strategies, and impact assessments that read like they were written by a human security expert! But that's not all! Our GPT-driven reports go the extra mile by scouring the web for related news articles, blogs, and references, so you have a 360-degree view of the vulnerabilities you've uncovered. With reNgine 2.0 revolutionize your penetration testing game and impress your clients with reports that are not just informative but engaging and comprehensive with detailed analysis on impact assessment and remediation strategies.

🥷   GPT-Powered Attack Surface Generation: With reNgine 2.0, reNgine seamlessly integrates with GPT to identify the attacks that you can likely perform on a subdomain. By making use of reconnaissance data such as page title, open ports, subdomain name etc, reNgine can advice you the attacks you could perform on a target. reNgine will also provide you the rationale on why the specific attack is likely to be successful.

🧭  Continuous monitoring: Continuous monitoring is at the core of reNgine's mission, and it's robust continuous monitoring feature ensures that their targets are under constant scrutiny. With the flexibility to schedule scans at regular intervals, penetration testers can effortlessly stay informed about their targets. What sets reNgine apart is its seamless integration with popular notification channels such as Discord, Slack, and Telegram, delivering real-time alerts for newly discovered subdomains, vulnerabilities, or any changes in reconnaissance data.

-----------------------------------------------------

Workflow

-----------------------------------------------------

Features

  • Reconnaissance:
    • Subdomain Discovery
    • IP and Open Ports Identification
    • Endpoints Discovery
    • Directory/Files fuzzing
    • Screenshot Gathering
    • Vulnerability Scan
      • Nuclei
      • Dalfox XSS Scanner
      • CRLFuzzer
      • Misconfigured S3 Scanner
    • WHOIS Identification
    • WAF Detection
  • OSINT Capabilities
    • Meta info Gathering
    • Employees Gathering
    • Email Address gathering
    • Google Dorking for sensitive info and urls
  • Projects, create distinct project spaces, each tailored to a specific purpose, such as personal bug bounty hunting, client engagements, or any other specialized recon task.
  • Perform Advanced Query lookup using natural language alike and, or, not operations
  • Highly configurable YAML-based Scan Engines
  • Support for Parallel Scans
  • Support for Subscans
  • Recon Data visualization
  • GPT Vulnerability Description, Impact and Remediation generation
  • GPT Attack Surface Generator
  • Multiple Roles and Permissions to cater a team's need
  • Customizable Alerts/Notifications on Slack, Discord, and Telegram
  • Automatically report Vulnerabilities to HackerOne
  • Recon Notes and Todos
  • Clocked Scans (Run reconnaissance exactly at X Hours and Y minutes) and Periodic Scans (Runs reconnaissance every X minutes/- hours/days/week)
  • Proxy Support
  • Screenshot Gallery with Filters
  • Powerful recon data filtering with autosuggestions
  • Recon Data changes, find new/removed subdomains/endpoints
  • Tag targets into the Organization
  • Smart Duplicate endpoint removal based on page title and content length to cleanup the reconnaissance data
  • Identify Interesting Subdomains
  • Custom GF patterns and custom Nuclei Templates
  • Edit tool-related configuration files (Nuclei, Subfinder, Naabu, amass)
  • Add external tools from Github/Go
  • Interoperable with other tools, Import/Export Subdomains/Endpoints
  • Import Targets via IP and/or CIDRs
  • Report Generation
  • Toolbox: Comes bundled with most commonly used tools during penetration testing such as whois lookup, CMS detector, CVE lookup, etc.
  • Identification of related domains and related TLDs for targets
  • Find actionable insights such as Most Common Vulnerability, Most Common CVE ID, Most Vulnerable Target/Subdomain, etc.

-----------------------------------------------------

Scan Engine

subdomain_discovery: {
  'uses_tools': [
    'subfinder',
    'ctfr',
    'sublist3r',
    'tlsx',
    'oneforall',
    'netlas'
  ],
  'enable_http_crawl': true,
  'threads': 30,
  'timeout': 5,
}
http_crawl: {}
port_scan: {
  'enable_http_crawl': true,
  'timeout': 5,
  # 'exclude_ports': [],
  # 'exclude_subdomains': true,
  'ports': ['top-100'],
  'rate_limit': 150,
  'threads': 30,
  'passive': false,
  # 'use_naabu_config': false,
  # 'enable_nmap': true,
  # 'nmap_cmd': '',
  # 'nmap_script': '',
  # 'nmap_script_args': ''
}
osint: {
  'discover': [
      'emails',
      'metainfo',
      'employees'
    ],
  'dorks': [
    'login_pages',
    'admin_panels',
    'dashboard_pages',
    'stackoverflow',
    'social_media',
    'project_management',
    'code_sharing',
    'config_files',
    'jenkins',
    'wordpress_files',
    'php_error',
    'exposed_documents',
    'db_files',
    'git_exposed'
  ],
  'custom_dorks': [
    {
      'lookup_site': 'google.com',
      'lookup_keywords': '/home/'
    },
    {
      'lookup_site': '_target_',
      'lookup_extensions': 'jpg,png'
    }
  ],
  'intensity': 'normal',
  'documents_limit': 50
}
dir_file_fuzz: {
  'auto_calibration': true,
  'enable_http_crawl': true,
  'rate_limit': 150,
  'extensions': ['html', 'php','git','yaml','conf','cnf','config','gz','env','log','db','mysql','bak','asp','aspx','txt','conf','sql','json','yml','pdf'],
  'follow_redirect': false,
  'max_time': 0,
  'match_http_status': [200, 204],
  'recursive_level': 2,
  'stop_on_error': false,
  'timeout': 5,
  'threads': 30,
  'wordlist_name': 'dicc'
}
fetch_url: {
  'uses_tools': [
    'gospider',
    'hakrawler',
    'waybackurls',
    'gospider',
    'katana'
  ],
  'remove_duplicate_endpoints': true,
  'duplicate_fields': [
    'content_length',
    'page_title'
  ],
  'enable_http_crawl': true,
  'gf_patterns': ['debug_logic', 'idor', 'interestingEXT', 'interestingparams', 'interestingsubs', 'lfi', 'rce', 'redirect', 'sqli', 'ssrf', 'ssti', 'xss'],
  'ignore_file_extensions': ['png', 'jpg', 'jpeg', 'gif', 'mp4', 'mpeg', 'mp3']
  # 'exclude_subdomains': true
}
vulnerability_scan: {
  'run_nuclei': false,
  'run_dalfox': false,
  'run_crlfuzz': false,
  'run_s3scanner': true,
  'enable_http_crawl': true,
  'concurrency': 50,
  'intensity': 'normal',
  'rate_limit': 150,
  'retries': 1,
  'timeout': 5,
  'fetch_gpt_report': true,
  'nuclei': {
    'use_nuclei_config': false,
    'severities': [
      'unknown',
      'info',
      'low',
      'medium',
      'high',
      'critical'
    ],
    # 'tags': [],
    # 'templates': [],
    # 'custom_templates': [],
  },
  's3scanner': {
    'threads': 100,
    'providers': [
      'aws',
      'gcp',
      'digitalocean',
      'dreamhost',
      'linode'
    ]
  }
}
waf_detection: {}
screenshot: {
  'enable_http_crawl': true,
  'intensity': 'normal',
  'timeout': 10,
  'threads': 40
}

# custom_header: "Cookie: Test"

-----------------------------------------------------

Quick Installation

Note: Only Ubuntu/VPS

  1. Clone this repo

    git clone https://github.com/yogeshojha/rengine && cd rengine
  2. Edit the dotenv file, please make sure to change the password for postgresql POSTGRES_PASSWORD!

    nano .env
  3. In the dotenv file, you may also modify the Scaling Configurations

    MAX_CONCURRENCY=80
    MIN_CONCURRENCY=10

    MAX_CONCURRENCY: This parameter specifies the maximum number of reNgine's concurrent Celery worker processes that can be spawned. In this case, it's set to 80, meaning that the application can utilize up to 80 concurrent worker processes to execute tasks concurrently. This is useful for handling a high volume of scans or when you want to scale up processing power during periods of high demand. If you have more CPU cores, you will need to increase this for maximised performance.

    MIN_CONCURRENCY: On the other hand, MIN_CONCURRENCY specifies the minimum number of concurrent worker processes that should be maintained, even during periods of lower demand. In this example, it's set to 10, which means that even when there are fewer tasks to process, at least 10 worker processes will be kept running. This helps ensure that the application can respond promptly to incoming tasks without the overhead of repeatedly starting and stopping worker processes.

    These settings allow for dynamic scaling of Celery workers, ensuring that the application efficiently manages its workload by adjusting the number of concurrent workers based on the workload's size and complexity

  4. Run the installation script, Please keep an eye for any prompt, you will also be asked for username and password for reNgine.

    sudo ./install.sh

    If install.sh does not have install permission, please change it, chmod +x install.sh

reNgine can now be accessed from https://127.0.0.1 or if you're on the VPS https://your_vps_ip_address

Unless you are on development branch, please do not access reNgine via any ports

Installation (Mac/Windows/Other)

Installation instructions can be found at https://reNgine.wiki/install/detailed/

Updating

  1. Updating is as simple as running the following command:

    cd rengine && sudo ./update.sh

    If update.sh does not have execution permissions, please change it, sudo chmod +x update.sh

    NOTE: if you're updating from 1.3.6 and you're getting a 'password authentication failed' error, consider uninstalling 1.3.6 first, then install 2.x.x as you'd normally do.

Changelog

Please find the latest release notes and changelog here.

-----------------------------------------------------

Screenshots

Scan Results

General Usage

Initiating Subscan

Recon Data filtering

Report Generation

Toolbox

Adding Custom tool in Tools Arsenal

-----------------------------------------------------

Contributing

Contributions are what make the open-source community such an amazing place to learn, inspire and create. Every contributions you make is greatly appreciated. Your contributions can be as simple as fixing the indentation or UI, or as complex as adding new modules and features.

See the Contributing Guide to get started.

You can also join our Discord channel #development for any development related questions.

-----------------------------------------------------

First-time Open Source contributors

Please note that reNgine is beginner friendly. If you have never done open-source before, we encourage you to do so. We will be happy and proud of your first PR ever.

You can start by resolving any open issues.

-----------------------------------------------------

reNgine Support

Please do not use GitHub for support requests. Instead, join our Discord channel #support.

-----------------------------------------------------

Support and Sponsoring

Over the past few years, I have been working hard on reNgine to add new features with the sole aim of making it the de facto standard for reconnaissance. I spend most of my free time and weekends working on reNgine. I do this in addition to my day job. I am happy to have received such overwhelming support from the community. But to keep this project alive, I am looking for financial support.

Paypal Bitcoin Ethereum
https://www.paypal.com/paypalme/yogeshojha11 35AiKyNswNZ4TZUSdriHopSCjNMPi63BCX 0xe7A337Da6ff98A28513C26A7Fec8C9b42A63d346

OR

  • Add a GitHub Star to the project.
  • Tweet about this project, or maybe blogs?
  • Maybe nominate me for GitHub Stars?
  • Join DigitalOcean using my referral link your profit is $100 and I get $25 DO credit. This will help me test reNgine on VPS before I release any major features.

It takes a considerable amount of time to add new features and make sure everything works. Donating is your way of saying: reNgine is awesome.

Any support is greatly appreciated! Thank you!

-----------------------------------------------------

reNgine Bug Bounty Program

huntr

Security researchers, welcome aboard! I'm excited to announce the reNgine bug bounty programme in collaboration with huntr.dev, which means that you will be rewarded for any vulnerabilities you find in reNgine.

Thank you for your interest in reporting reNgine vulnerabilities! If you are aware of any potential security vulnerabilities in reNgine, we encourage you to report them immediately via huntr.dev.

Please do not disclose vulnerabilities via Github issues/blogs/tweets after/before reporting to huntr.dev as this is explicitly against the disclosure policy of huntr.dev and reNgine and will not be considered for monetary rewards.

Please note that the reNgine maintainer does not set the bounty amount. The bounty reward is determined by an industry-first equation developed by huntr.dev to understand the popularity, impact and value of repositories to the open-source community.

What do I expect from security researchers?

  • Patience: Please note that I am currently the only maintainer in reNgine and it will take some time to validate your report. I ask for your patience during this process.
  • Respect for privacy and security reports: Please do not publicly disclose any vulnerabilities (including GitHub issues) before or after reporting them on huntr.dev! This is against the disclosure policy and will not be rewarded.
  • Respect the rules

What do you get in return?

  • Thanks from the maintainer
  • Monetary rewards
  • CVE ID(s)

Please find the FAQ and Responsible disclosure policy from huntr.dev.

-----------------------------------------------------

License

Distributed under the GNU GPL v3 License. See LICENSE for more information.

-----------------------------------------------------

(ChatGPT was used to write some or most part of this README section.)

rengine's People

Contributors

0x71rex avatar aju100 avatar anonymouswp avatar bin-maker avatar c0wnuts avatar creased avatar dependabot[bot] avatar dontclickthelink avatar jalvarezit avatar jxdv avatar khanhnt2 avatar khiemdoan avatar m00tiny avatar micr0seft avatar nerrorsec avatar ocervell avatar payloadartist avatar ph33rr avatar psyray avatar radaram avatar random-robbie avatar sbimochan avatar serizao avatar shirishupadhyay avatar siddharthbharadwaj avatar suprita-25 avatar valbrux avatar vinaynm avatar x-ai avatar yogeshojha avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

rengine's Issues

rengine stop working properly after setting DEBUG=0

Hi,

Thank you so much for making this tool publicly available. I encountered some errors when uploading a wordlist, And the error was disclosing many sensitive information as we know. As I am also running this tool on my server (Public IP). So, leaving DEBUG as True might be a problem in the future. So, Please make this little change in this tool.

Best regards,
Mubassir

Starting rengine-master_db_1 ... error

docker-compose up -d this is not worked

Starting rengine-master_db_1 ...
Starting rengine-master_db_1 ... error

ERROR: for rengine-master_db_1 Cannot start service db: driver failed programming external connectivity on endpoint rengine-master_db_1 (2df8216f6d26811d173c9e0471534928e41f890996067b9ec2481727b64228ea): Error starting userland proxy: listen tcp 0.0.0.0:5432: bind: address already in use

ERROR: for db Cannot start service db: driver failed programming external connectivity on endpoint rengine-master_db_1 (2df8216f6d26811d173c9e0471534928e41f890996067b9ec2481727b64228ea): Error starting userland proxy: listen tcp 0.0.0.0:5432: bind: address already in use

Feature - Add Search Feature

Add a search field for searching using different filters such as technology,port,status code and same in uri like path,parameters
Screenshot_38

Web login

Since most of the people run their recon tools on cloud servers, it would be great to have a web login.

Creating rengine_db_1 ... error

While installing facing this problems

/rengine# docker-compose up
Creating network "rengine_rengine_network" with the default driver
Creating rengine_db_1 ...
Creating rengine_db_1 ... error

ERROR: for rengine_db_1  Cannot start service db: driver failed programming external connectivity on endpoint rengine_db_1 (ecf18ca60cf994048dd5d06cdddf6e842ef11c97247afc04f6699447e6fd2d7e): Error starting userland proxy: listen tcp 0.0.0.0:5432: bind: address already in use

ERROR: for db  Cannot start service db: driver failed programming external connectivity on endpoint rengine_db_1 (ecf18ca60cf994048dd5d06cdddf6e842ef11c97247afc04f6699447e6fd2d7e): Error starting userland proxy: listen tcp 0.0.0.0:5432: bind: address already in use
ERROR: Encountered errors while bringing up the project.

Commands and tools

cat SecLists/Discovery/DNS/dns-Jhaddix.txt | subgen -d DOMAIN.TLD | zdns A --name-servers 1.1.1.1 --threads 500 | jq -r "select(.data.answers[0].name) | .name"

#bugbountytips

python3 dirsearch.py -e conf,config,bak,backup,swp,old,db,sql,asp,aspx,aspx~,asp~,py,py~,rb,rb~,php,php~,bak,bkp,cache,cgi,conf,csv,html,inc,jar,js,json,jsp,jsp~,lock,log,rar,old,sql,sql.gz,http://sql.zip,sql.tar.gz,sql~,swp,swp~,tar,tar.bz2,tar.gz,txt,wadl,zip,.log,.xml,.js.,.json -u https://coderedmarketing.eccouncil.org/ -t 100 -w /root/tools/bruteforce/ffufplus/wordlist/dicc.txt -b

Bug - nc not found

Issue Summary

When running the project using docker-compose up, we get the following error message:

web_1  | /app/docker-entrypoint.sh: 7: /app/docker-entrypoint.sh: nc: not found

Steps to Reproduce

docker-compose pull
docker-compose up

Technical details

The installation of netcat binary is missing from the Dockerfile.

Vulnerable version of the library 'jquery' found

Hi @yogeshojha I ran rengine in my local machine with burpsuite to find any vulnerability and with an automated extension like retire.js give some issue regarding jquery one. Have a look at it.

Issue detail

The library jquery version 3.1.1.min has known security issues. Supporting materials and references are listed below:

Impact

An attacker can able to inject properties on Object.prototype

Affected versions

The vulnerability is affecting all versions prior 3.4.0 (between * and 3.4.0)

Solutions

Updated the patched version of jquery

Floating button on Start scan page

While someone will be going to click on start the scan button he/she might get confused as the start scan button is coming below all the engines.

Issue on composing dockerfile

Hi, @yogeshojha I have recently cloned and followed up the instructions as mentioned in README.md. Here is the screenshot of the problem.
yogeshproblem

My machine Info:
Linux 5.4.0-40-generic x86_64 GNU/Linux

Steps to reproduce:
docker-compose up -- build

But I have run this web application using virtual environment which is easy to setup. I think we have to docs about setting up the environment using another method too .

Image gets loaded in the same page directly.

It would be great if Screnshots pops up and then can easily go back to where I was. It gets loaded directly in the same page and when I go back via browser back button, it leaves where I was previously.

Deleted scan engines are added due to the load data running inside entrypoint.

The default scan engines are loaded via fixtures from the entry point. In docker, entry point runs every time a container starts. So, if the default scan engines are deleted, on the next container run, the deleted scan engines are added back. The idea is to run

python manage.py loaddata fixtures/default_scan_engines.json --app scanEngine.EngineType

from RUN rather than

ENTRYPOINT ["/app/docker-entrypoint.sh"]

Feature Request: Grouping Domains for a Particular asset

Issue: Right now, all targets are separately placed and as such it is difficult to figure out who owns what and which one does a particular domain belong to.

Fix: Creating a grouping function where a group of targets can be listed under a single group body wherein targets of an asset can be easily located and tasks can be applied simultaneously.

TLDR: Grouping domains of a particular asset would be great as it would help organize the recon workflow and cause less headaches!

Bug - HTTP Crawler No result

Found a bug? Please fill out the sections below. 👍

HTTP Crawler not working

A summary of the issue.
I tried with 4-5 target but the HTTP Crawler result is zero always

Steps to Reproduce

  1. For a quick verification i created scan engine with Fetch All URLs only
  2. scanned 4-5 targets
  3. scan completed but no result

Screenshot_39

Feature - JSfiles Enumeration | Parameter Discovery | Patterns to discover SSRF - SQLi - RCE - IDORs parameters

Find below list of features that will make the tool perfect in my opinion:

1. Javascript files enumeration
https://github.com/KathanP19/JSFScan.sh
A JSfiles enumeration module to extract hidden endpoints and secrets such as APIs keys will be an awesome module.
i'm using a tool called JSFScan which is a combine of multiple tools that take a list of subdomains.txt file and do the below:

  • Gather Js Files Links
  • Gather Endpoints For JSFiles
  • Find Secrets For JSFiles
  • Use Meg for fetching JsFiles for manual testing
  • Make a wordlist using words from jsfiles

2. Parameter Discovery
https://github.com/s0md3v/Arjun
Parameter Discovery module will be a very nice addition to the framework personally i use Arjun.

3. SSRF - SQLi - IDORs - RCE patterns using GF-Patterns tool and gf from Tomnomnom
https://github.com/1ndianl33t/Gf-Patterns

I use Gf-pattern/gf to help me on grep all suspicious parameters from url that helps in (SSRF/SQLi/IDORs/RCE/Openredirect..etc), its combine of multiple tools such as gf and waybackurls and gives a results of txt file contains results based in what you chose, below example for SSRF, this module is very helpful.

▶ cat subdomains.txt | waybackurls | sort -u >> waybackdata | gf ssrf | tee -a ssfrparams.txt

▶ cat waybackdata | gf redirect | tee -a redirect.txt

reNgine Installation error on Windows

There is a known issue for installing reNgine on windows based docker.

standard_init_linux.go:211: exec user process caused "no such file or directory"

Looks like this is the problem with CLRF.

Will fix this soon

Feature: Grep using gf and gf-pattern

Optional Grepping the output after fetching the endpoints from gau and hakwaler.
Showing possibility of open redirect or other vulns using gf and gf patterns.

filtering subdomain takeover part is not possible

while getting more than 20k subdomains in a scan one can not look to each subdomain manually .
so if we could filter this part like if someone directly searches for takeover phrase and gets the subdomain that can be took over.

Feature - Schedule reoccurring scans and identify changes

Is your feature request related to a problem? Please describe.
Unable to detect changes in a target over a period of time and schedule reoccurring scans.

Describe the solution you'd like
Schedule new scans, weekly/monthly, and then detecting if there were any changes in the title, status code, new subdomains for a target, or a previous subdomain that wasn't active has became active.

The notification feature would come in handy along with this feature. Imagine getting notifications after a scan has detected changes in a target, allowing you to act quickly.

[Bug] Subdomains Discovered - are not showing after scan is complete.

Issue Summary

While scan was in progress, subdomains discovered was showing and after scan is completed, subdomains discovered are not visible anymore. They are available to export but aren't displayed from dashboard.

Steps to Reproduce

  1. Started scanning "droom.in".
  2. Used full scan configuration.
  3. After scan is completed, subdomains discovered aren't visible.
  • I have confirmed that this issue can be reproduced as described on a latest version/pull of reNgine: YES

Technical details

Please list out any technical details such as operating environment.
OS: Ubuntu 16.04.6 LTS (GNU/Linux 4.15.0-1080-gcp x86_64)

Docker:
Client: Docker Engine - Community
Version: 19.03.12
API version: 1.40
Go version: go1.13.10
Git commit: 48a66213fe
Built: Mon Jun 22 15:45:49 2020
OS/Arch: linux/amd64
Experimental: false

Server: Docker Engine - Community
Engine:
Version: 19.03.12
API version: 1.40 (minimum version 1.12)
Go version: go1.13.10
Git commit: 48a66213fe
Built: Mon Jun 22 15:44:20 2020
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.2.13
GitCommit: 7ad184331fa3e55e52b890ea95e65ba581ae3429
runc:
Version: 1.0.0-rc10
GitCommit: dc9208a3303feef5b3839f4323d9beb36df0a9dd
docker-init:
Version: 0.18.0
GitCommit: fec3683

Docker-Compose:
docker-compose version 1.26.2, build eefe0d31
docker-py version: 4.2.2
CPython version: 3.7.7
OpenSSL version: OpenSSL 1.1.0l 10 Sep 2019

Feature Request - Integrating Seclists

Hello there Yogesh.
Firstly, thanks a ton for creating this awesome tool. loving this tool.
If it will be possible to accommodate a small request by adding a feature where the tool can include seclists. This will help in directly selecting the pre-defined lists for discovery.

Thanks in advance.

Bug - Windows Installation Error

While installing rengine container on windows, the CRLF is causing problem.

Look at this:

https://github.com/dhoer/docker-on-windows-gotchas

Successfully tagged rengine_web:latest
Starting rengine_db_1 ... done                                                                                                    Recreating rengine_web_1 ... done                                                                                                 Attaching to rengine_db_1, rengine_web_1
db_1   |
db_1   | PostgreSQL Database directory appears to contain a database; Skipping initialization
db_1   |
db_1   | 2020-07-11 17:00:03.343 UTC [1] LOG:  starting PostgreSQL 12.2 (Debian 12.2-2.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
db_1   | 2020-07-11 17:00:03.343 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
db_1   | 2020-07-11 17:00:03.343 UTC [1] LOG:  listening on IPv6 address "::", port 5432
db_1   | 2020-07-11 17:00:03.792 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1   | 2020-07-11 17:00:04.274 UTC [25] LOG:  database system was shut down at 2020-07-11 16:59:19 UTC
web_1  | standard_init_linux.go:211: exec user process caused "no such file or directory"
db_1   | 2020-07-11 17:00:04.450 UTC [1] LOG:  database system is ready to accept connections
rengine_web_1 exited with code 1
Gracefully stopping... (press Ctrl+C again to force)

Feature - Add urlscan.io service's historical results data

In urlscan.io Submit URLs to be scanned, search for historical results by IP, domain or URL, retrieve results and pivot to further leads. You can search all Public scans as well as your own Private scans on our platform. You can subscribe to our API plans as a Team and share your available quota amongst multiple users on the urlscan.io platform.

Use the search for historical results feature for targets all subdomains and sort out alive endpoints .

Example : https://urlscan.io/api/v1/search/?q=domain:bugcrowd.com

Take a look at tweet for more info https://twitter.com/prial261/status/1254503609217150976

Cheers

Default Credentia

Hi
kindly provide or update in the readme the default credential for 1st time login.

installation crash

I follow the setup steps (docker and docker-compose are working well) but the installation crash on rengine_web_1 process and say that it cannot find "docker-entrypoint.sh" for container_linux.go. I delete the folder, clone it and restart installation but the same error is still display.

Technical details

My OS is windows 10.

Feature - Amass config, Subdomainizer & ParamSpider

Hi,
Thanks for your tool, great job, I had started something similar but what you have done is really clean / functional.

I offer you some suggestions in addition to what others have already asked for (especially GF & Nuclei & github-subdomains.py)

  • The ability to add an Amass config file, the detection power of the tool lies in the use of this file which allows to search for subdomains on services requiring API keys (like SecurityTrails which is a musthave).

All these things can be done with the following command from Amass :
amass enum -active -o results.txt -d domain.tld -brute -w wordlist.txt -config config.ini -dir aDirectory

The difference between two scans via Amass can then be found with the following command:

amass track -d $DOMAIN -dir aDirectory | grep 'Found:')

In addition, I suggest you add Subdomainizer or something similar and ParamSpider Which is a good complement to hakrawler (I found it more convenient to use GF on ParamSpider instead of the hakrawler output.)

Regards,
Jomar

Feature - Add a way to skip subdomain scan process & use external discovered subdomains lists

In import target option we can feed multiple domains in txt list . But if you can add a feature where we can skip the Subdomain scanning part and import a subdomains list from external scanned tools as there is many other way to gather Subdomains 😇 . Here we can’t skip the Subdomain discovery from scan engine . Hope you will take a look to skip the Subdomain discovery option , so the workflow will be

  • Import Subdomains list in txt what discovered by other tools and recon methodology .
  • Fetch live domains
  • Do port scan
  • Takeover scan
  • Directory search
  • Fetch all urls from wayback, OTX .
  • Also if possible add urlscan.io for fetching target data ex: https://twitter.com/prial261/status/1254503609217150976

Cheers

Feature - Github Subdomains/Endpoints

Is your feature request related to a problem? Please describe.
Github can contain subdomains and endpoints that can't be found anywhere else.

Describe the solution you'd like
There are a number of tools that implement the ability to search Github for endpoints/subdomains such as https://github.com/gwen001/github-search

Also, you may need to provide the ability to add github tokens/API keys to other tools from the GUI.

For Port Scan use this tool

Is your feature request related to a problem? Please describe.

Nmap takes a lot of time to scan the ports as well as with masscan etc.

Describe the solution you'd like

Add this tool

https://github.com/brandonskerritt/RustScan

Describe alternatives you've considered

Scans all 65k ports in 8 seconds (on 10k batch size).

Additional context

https://github.com/brandonskerritt/RustScan/blob/master/pictures/with_rustscan.gif

https://github.com/brandonskerritt/RustScan/blob/master/pictures/without_rustscan.gif

For Request Smuggling Identification - Add this Tool

Is your feature request related to a problem? Please describe.

Tool need to be powered with http request smuggling identification mechanism.

Describe the solution you'd like

Adding this tool will

https://github.com/defparam/smuggler

Widget Cards are not responsive

While on the dashboard, either zoom or resize the browser window, the cards do not resize properly. This has something to do with the Bootstrap grid system.

Screenshot 2020-07-05 at 8 41 49 PM

Maybe somebody can work on it, Good first issue!

Code for this should be inside /reNgine/dashboard/templates/dashboard/index.html

Exception on start scan without selecting scan type

When scan is started without selecting scan type, an exception is occurred

`

Request Method: POST
http://127.0.0.1:8000/start_scan/start/1
3.0.7
MultiValueDictKeyError
'scan_mode'
/usr/local/lib/python3.8/site-packages/django/utils/datastructures.py in getitem, line 78
/usr/local/bin/python3
3.8.3
['/app', '/usr/local/lib/python38.zip', '/usr/local/lib/python3.8', '/usr/local/lib/python3.8/lib-dynload', '/usr/local/lib/python3.8/site-packages']
Sat, 27 Jun 2020 23:13:16 +0530

`

error login panel

docker exec -it rengine_web_1 python manage.py createsuperuser
not work !!

''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
┌─[root@parrot]─[/home/bikram/Downloads/rengine-master]
└──╼ #sudo docker exec -it rengine_web_1 python3 manage.py createsuperuser
Error: No such container: rengine_web_1

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.