bluecatlabs/gateway-workflows

BlueCat DNS Gateway Workflows

BlueCat Gateway Workflows

BlueCat Gateway™ is a Python-based web utility that leverages the BlueCat Address Manager™ (BAM) API to allow you to create custom workflows for common tasks in order to maximize efficiency of enterprise DNS operations.

BlueCat Gateway consists of a set of Python classes forming an API to Address Manager and BlueCat DNS/DHCP Server (BDDS) along with a customized Python Flask web framework for building custom user interfaces and REST endpoints. It can run on most Linux variants with the correct packages installed.

Installation

The workflows are ready to use; they just need to be placed into the <bluecat_gateway>/workflows folder. The same can be done with the community examples.

This will copy over all of the Example workflows:

cp -r <gateway-example-repo>/Examples/* <bluecat_gateway>/workflows/Examples/.

If only a specific set of workflows is required they can be copied into the workflows folder individually. However, ensure that there are __init__.py files present in your workflows folder structure leading up to the workflow itself. This is required by BlueCat Gateway in order to discover the workflow.

Usage

Once the workflows have been copied over, just start the BlueCat Gateway container. In order to use the added workflows the permissions have to be adjusted through the administrative permissions workflow. The workflows contained in the Examples folder are always up-to-date with the latest BlueCat Gateway version. This is not the case for the Community workflows. The contributing authors are required to specify the version of BlueCat Gateway for which the workflow was created or updated. While all community workflows will be reviewed, they are delivered "as is".

Contributions

Contributing follows a review process: before a workflow is accepted it will be reviewed and then merged into the master branch. It will be the responsibility of the contributor to ensure that their workflow is supported for future releases of BlueCat Gateway.

Please review the Terms and Conditions.

Standards

When contributing to the Community examples please ensure that the code is of good quality

• BlueCat Gateway is written with the PEP8 standard in mind
• Ensure that each function contains a docstring explaining the purpose of the function, the input variables and, the return type
• Use plenty of comments to describe what the function is doing
• Use meaningful variable and function names
• Contributions should not directly access the BAM database

Process

1. Fork it!
2. Create your feature branch: git checkout -b my-new-feature
3. Move your workflow into Community/<your-workflow>
4. Create <your-workflow>/README.md explaining what the workflow does. Use the Template below.
5. Commit your changes: git commit -am 'Add some feature'
6. Push to the branch: git push origin my-new-feature
7. Submit a pull request

Community Template

When contributing a workflow ensure that it contains a README.md and that each file has the following notice header:

Copyright YYYY BlueCat Networks (USA) Inc. and its affiliates

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

By: Your Name ([email protected])
Date: DD-MM-YYYY
Gateway Version: X.X.X
Description: Brief description of what the workflow does and the expected behaviour

Credits

BlueCat Gateway would be so much less without the following people. Thank you for contributing your time to making this project a success.

The Team:

• Anita Cheng
• Brian Shorland
• Edwin Christie
• Mike Leaver
• Nishant Malhotra
• Lily Wickham
• Ajay Basnet
• Alexander Bartella
• Roy Fisher
• Chris Collins

Integration & Innovation Team:

• Bill Morton
• Chris Meyer
• Jubin George

Special Thanks:

• Glenn McAllister
• Robert Barnhardt™
• Roy Reshef
• Vadim Farafontov
• Victor Fradkin
• Xiao Dong
• Nikhil Jangi
• Anshul Sharma
• Ekim Maurer
• Lana Litvak
• Raymond Leong
• Rohina Dhunjeebhoy
• Shanice Cohen
• Chris Catral
• Steven Diao
• Murtaza Haider
• Hongbo Wang
• Maziar Esfandiarpoor
• Chris Storz
• Delme Herbert
• David Cohen
• Prerana Pradhan
• Lucas Tran
• Chester Wu
• Aman Tawakley
• Andreas Avramidis
• Martin Minkov

License

Copyright 2017-2024 BlueCat Networks (USA) Inc. and its affiliates

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

gateway-workflows's Issues

How to populate a TableField with data from MongoDB?

If I am to use PyMongo for instance to pull information from a Mongo database, how do I then populate a TableField on a workflow page?

How to rename a workflow?

For instance: The example workflow "Add Host Record Example". What would the easiest way to rename the entire workflow and all it's references to just "Add Host Record"?

When adding a new host record, how do you check if the IP is already in use?

In my specific case, I have attempted to implement a check via get_linked_entities on the ip4_address object, however it seems to always return API exceptions saying the IP can't be found. However, when removing this check it will allow the record to be added no problem. Is there something obvious I'm missing, an existing function that I could import perhaps?

            configuration = g.user.get_api().get_entity_by_id(form.configuration.data)
            view = configuration.get_view(request.form['view'])

            absolute_name = form.hostname.data + '.' + request.form['zone']

            ip4_address = request.form['ip4_address']
            ip4_address_string = str(ip4_address)
            ip4_address_list.append(ip4_address)

            for ip in ip4_address_list:
                ip4_object = configuration.get_ip4_address(ip.strip())
                linked_entities = ip4_object.get_linked_entities('HostRecord')
                if len(list(linked_entities)) != 0:
                    raise ValueError("Has linked records")

            # Add host Record
            host_record = view.add_host_record(absolute_name, ip4_address_list)

Instantiate a service account API instance mid-workflow?

Example: Deleting Host (A) Records. If a normal business user logs into Gateway with their LDAP account and wants to delete a record, they will require at least 'modify' permissions on the BlueCat Address Manager (BAM) back-end (from my understanding).

If you want to retain their user identity in the web session for auditing and accountability purposes, but carry out those actions via a service account how would you do so?

I know the autologin feature can accomplish this, but that exposes the entire endpoint as an unauthenticated security gap.

Can you setup a new 'user' temporarily midway through a workflow, or is there a better alternative I'm not seeing?

How to default and lock in the 'Configuration' and 'View' in the UI Example workflows?

For instance:
Examples > Host Record > Add Host Record Example

The options presented to the end-user are:

• Configuration
• View
• Zone
• IP Address
• Hostname

How would you set the Configuration and View to specific choices and not allow the user to change them? Or for that matter, remove them entirely from the form, yet keeping those choices active on the back end?

get_audit_info missing files

Key folders and files are missing. Noteably, js, css, etc.

https://github.com/bluecatlabs/gateway-workflows/tree/master/Community/get_audit_info

Adding a button to each row in a WTForm's TableField?

How would you render in either a checkbox or text button inside, or next to, each row?

User	Host	Action	Approval
myusername	host.domain.com	delete	[]

EDIT:
The closest I can find in existing source code would be the checkbox inputs appended to the return in the following:
Certified/cisco_aci_example/cisco_aci_example_page.py

The output I end up getting is just the HTML text without rendering

User	Host	Action	Approval
myusername	host.domain.com	delete	<input type="checkbox" id="{name}" name="{name}" {option} checked>

Known security vulnerabilities detected

These alerts were received by BlueCat from the Github Security Alert Service.

Please address the issues below.

Dependency pyyaml Version < 4.2b1 Upgrade to ~> 4.2b1
Vulnerabilities CVE-2017-18342 High severity
Defined in requirements.txt

Dependency flask Version < 0.12.3 Upgrade to ~> 0.12.3
Vulnerabilities CVE-2018-1000656 Moderate severity
Defined in requirements.txt

Dependency webargs Version < 5.1.3 : Upgrade to ~> 5.1.3
Vulnerabilities CVE-2019-9710 High severity
Defined in requirements.txt

Do no see existing worklows in the GUI

Hello,

Looking at /portal/bluecat_portal/workflows i can see different directory containing Workflow.
The only one showing up in the GUI is the /host_records
All others are not visible.
drwxr-xrwx 4 flask flask 59 Aug 27 09:09 dns/
drwxr-xrwx 6 6487 users 160 Aug 27 09:10 host_record/
drwxr-xr-x 4 flask flask 57 Sep 3 14:45 test/
drwxr-xrwx 7 flask flask 178 Aug 27 08:04 update_host_record_example/

the difference is the owner: flask:flask instaed of 6487:users.
I can not change the owner.
Any idea to correct the issue?

Thanks
Pascal

How to limit, or customize, what 'zones' can be chosen in the Host Record workflows?

By default it appears the Host Record workflows will pull down all possible zones based on your configuration and view, but if I were to want to present only certain zones, or customize the drop down list to my own choices, how would I accomplish this?

v20.1.1 Medium Vulnerability: Web Application Potentially Vulnerable to Clickjacking

Further actions needed to resolve a vulnerability. Same as Issue #71
Latest version seen on: BlueCat Gateway 20.1.1

Medium Vulnerability: Web Application Potentially Vulnerable to Clickjacking

https://www.tenable.com/plugins/nessus/85582
https://cwe.mitre.org/data/definitions/693

The following pages do not use a click-jacking mitigation response header and contain a clickable event:
/css
/icons
/img
/static
/static/css

Fix: Return the X-Frame-Options or Content-Security-Policy (with the 'frame-ancestors' directive) HTTP header with the page's response. This prevents the page's content from being rendered by another site when using the frame or iframe HTML tags.

Allow Bluecat API Gateway to run as arbitrary UID

Some distributions of kubernetes add a layer of security by randomizing the UID a container image runs as, ignoring the assertion that may be present in a docker formatted container image. In order to make the Bluecat API Gateway the most portable within the k8s ecosystem, please allow it to run with arbitrary UID.

Example distribution documentation, section "SUPPORT ARBITRARY USER IDS"

Missing Apache license information in additional community offerings

@billfmorton you are missing the required Apache 2.0 license template in several community workflows.
Please update the README.md file to include the license template as cited under Contributions>Community Template the gateway-workflows README: https://github.com/bluecatlabs/gateway-workflows/blob/master/README.md

https://github.com/bluecatlabs/gateway-workflows/tree/master/Community/bulk_record_export
https://github.com/bluecatlabs/gateway-workflows/tree/master/Community/ServiceNow%20CMDB
https://github.com/bluecatlabs/gateway-workflows/tree/master/Community/Service%20Requests
https://github.com/bluecatlabs/gateway-workflows/tree/master/Community/NewLocation

• also missing README.md
  https://github.com/bluecatlabs/gateway-workflows/tree/master/Community/GitLab%20Management
• also missing README.md
  https://github.com/bluecatlabs/gateway-workflows/tree/master/Community/create_server_vro
  https://github.com/bluecatlabs/gateway-workflows/tree/master/Community/move_to_bluecatgateway_udf
  https://github.com/bluecatlabs/gateway-workflows/tree/master/Community/get_audit_info

CiscoDNA IPAM breaks authentication for all workflows

Incorporating the code from CiscoDNA/ipam/ipam_page.py into an existing Integrity Gateway platform with existing API workflows breaks the ability to authenticate will all workflows. The existing workflows have the @util.rest_workflow_permission_required decorator applied.

Steps to Reproduce

1. Create an Integrity Gateway 21.8.1 installation with API workflows utilizing the @util.rest_workflow_permission_required decorator
2. Install the CiscoDNA IPAM driver
3. Attempt to interact with any API workflows.

Current behavior

The problem code appears to be this section:

@app.before_request
def my_before_request():
    token = request.headers.get('auth')
    if token is not None:
        u = get_user_from_session(token)
        g.user = u
        g.use_rest = True
        if u:
            g.user.logger.debug(request)

The following lines always return None, and set the g.user object to None:

u = get_user_from_session(token)
g.user = u

Attempting to use any API endpoint results in an HTTP 401 error with the following API payload:

{
    "message": "The server could not verify that you are authorized to access the URL requested. You either supplied the wrong credentials (e.g. a bad password), or your browser doesn't understand how to supply the credentials required."
}

Fix

In our installation, removing the entire @app.before_request section resolved our issue. This section seems to be an unnecessary and broken implementation of code already incorporated in the Gateway product at portal/app.py. The relevant section in app.py follows:

@app.before_request
def before_request():
    """ Before each request hook"""
    g.config = config
    g.use_rest = True
    if request.user_agent.browser:
        g.use_rest = False
    g.url_hit = urlparse(request.url).path
    g.path = request.path.strip("/")

    user, use_rest = _get_user_from_request(request)
    if user is None:
        user, use_rest = _get_user_from_session(session)
    else:
        session["access_token"] = user.get_unique_name()
        session["username"] = user.get_username()
        session["bam_url"] = user.get_api().get_url()

    g.user = user
    if use_rest is not None:
        g.use_rest = use_rest

    log_endpoint(f"Endpoint {request.path} retrieved by {request.remote_addr}")

Potential Bug? POST /admin/api/workspace/import

It appears that when attempting to load the GitHub workflows via API by sending a POST /admin/api/workspace/import request to a Gateway 20.1.1 server with a proper JSON body with parameters (as per the documentation and release notes) it only ever responds with a '403 Forbidden' response.

CiscoDNA IPAM create pool broken

Attempting to create pools with:

POST /ipam/pool

results in operations that do not seem to properly understand block hierarchy, refusing to create valid IP4 Blocks in the platform and instead erroneously report conflicts.

Steps to reproduce

1. Create Gateway installation running 21.8.1
2. Create a configuration "testconfig" in BAM platform and define a top-level block of 172.16.0.0/16.
3. Within the 172.16.0.0/16 block, create block 172.16.0.0/20 in the BAM UI.
4. Using the IPAM driver endpoint, attempt to create another block (pool):

{
    "view": "testconfig",
    "poolName": "DNACPool",
    "poolCidr": "172.16.16.0/20",
    "DHCPServerip": [],
    "DNSServerip": [],
    "ClientOptions": []
}

Current Behavior

The IPAM driver returns an HTTP error code 400 with the following payload:

Server raised fault: "Block conflicts with another in the address space. Adding: 172.16.16.0/20 Existing: 172.16.0.0/16"

This can be easily be proven to be an incorrect error message as the 172.16.16.0/20 block can be created in the BAM user interface without any issues.

Expected Behavior

The IPAM driver should be consistent with the user interface when creating IP4 Blocks. It should not incorrectly report a conflict with the containing IP4Block. For example, there is nothing wrong with creating 172.16.16.0/20 within 172.160.0.0/16.

How to add a background image for the entire Gateway site?

I've attempted adding a background-image: url("../image.jpg"); to many places in the CSS where it references the body element including main.css, style.css, etc. Is there a very obvious way to accomplish this that I'm misunderstanding?

Medium Risk Vulnerability - Web Application Potentially Vulnerable to Clickjacking

Vulnerability: Tenable Nessus detects the default login page for BlueCat Gateway as vulnerable to click-jacking.
https://www.tenable.com/plugins/nessus/85582

Fix: Return the X-Frame-Options or Content-Security-Policy (with the 'frame-ancestors' directive) HTTP header with the page's response. This prevents the page's content from being rendered by another site when using the frame or iframe HTML tags.
https://flask.palletsprojects.com/en/master/security/

How to populate a read only string field with a specified value on a Gateway UI form?

If I wanted to add a read-only pre-populated form field on an existing workflow form, what form field and parameters would I use? The closest that I can think would accomplish this would be something along the following:

username = CustomStringField(
    label='Username: ',
    readonly=True,
    value='myusername'
)

Get Text Records does not work

The /configurations/{configuration}/views/{view}/zones/{zone}/text_records/ API call does not work.

I have attempted to fix this by following the get CNames code, however when I make the get text records API call with the same parameters as the get CName records call, I get a 404 return code with the response No matching Zone(s) found.

Here is the WIP PR: #119

Here is the block of code that does not appear to be working:

gateway-workflows/Community/rest_api/dns_page.py

Lines 687 to 702 in 518c6fe

	class TextRecordCollection(Resource):
	@util.rest_workflow_permission_required('rest_page')
	@text_ns.response(200, 'Found Text records.')
	def get(self, configuration, view, zone=None):
	""" Get all text records belonging to default or provided Configuration and View plus Zone hierarchy. """
	configuration = g.user.get_api().get_configuration(configuration)
	view = configuration.get_view(view)
	zone_parent = view
	zone_hierarchy = zone.split('/zones')
	zone_entity = zone_parent.get_zone(zone_hierarchy[0])
	zone = check_zone_in_path(zone_entity, zone_hierarchy[0], zone_hierarchy[1:], zone_parent)
	text_records = zone.get_children_of_type(zone.TextRecord)
	result = [text.to_json() for text in text_records]
	return jsonify(result)

Here is a similar block of code for retrieving CName records that is working:

gateway-workflows/Community/rest_api/dns_page.py

Lines 593 to 606 in 518c6fe

	class CNameRecordCollection(Resource):
	@util.rest_workflow_permission_required('rest_page')
	@cname_ns.response(200, 'Found CName records.')
	def get(self, configuration, view, zone=None):
	""" Get all cname records belonging to default or provided Configuration and View plus Zone hierarchy. """
	configuration = g.user.get_api().get_configuration(configuration)
	zone = generate_zone_fqdn(zone, configuration.get_view(view))
	if zone is None:
	return 'No matching Zone(s) found', 404
	host_records = zone.get_children_of_type(zone.AliasRecord)
	result = [host.to_json() for host in host_records]
	return jsonify(result)

Bluecat Gateway running in Docker Container should log to STDOUT and STDERR

In most kubernetes distributions aggregate logging systems collect from STDOUT and STDERR only. Local logging is discouraged as well. Ephemeral disk local logging can lead to disk pressure events that causes the pods to be evicted. Mounted volume logging still requires other work including a separate process to ship those logs from a volume to the logging system of choice and logrotate that volume so it does not fill up.

Therefor, to make the gateway k8s friendly please log to STDOUT and STDERR only by default.

Request: example for uploading response policy items RESTapi

It would be great to view an example on how to add a domain to a response policy (i.e. redirect) through the RESTapi.

I have been attempting to use the function "....../REST/v1/uploadResponsePolicyItems?" without success, failing at "Error 415 Unsupported Media Type" .

Would it be possible for you guys to assist.

CiscoDNA IPAM Driver cannot delete pools

Deleting IP pools (IP4Blocks) using the CiscoDNA IPAM driver appears to be broken due to the specification of pool including the slash "/" character, causing the route matching mechanism of Flask to fail.

Steps to reproduce

1. Create Gateway installation running 21.8.1
2. Attempt to delete a block using HTTP DELETE /ipam/pool/:view/:poolcidr where poolcidr is any block containing a forward slash, e.g. "172.16.16.0/20"

Current behavior

Because the path to any pool looks like another URL segment, for example the "/20" portion above, Flask does not find a matching route. This results in a HTTP 404 not found error with the following payload:

{
    "code": 404,
    "description": "The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.",
    "status": "Not Found"
}

In short, this endpoint can never delete a pool because the specification of a pool and mask creates an incompatible URL path.

Expected behavior

The "delete pool" API endpoint /ipam/pool/:view/:poolcidr needs to be modified to properly parse URLs that contain the additional forward clash included in the CIDR notation.

Bluecat workflow add_host_record

Hello community:

I have dowloaded some of the workflows in here. I dont have so much experience on this but i feel that i have done great advances on it.

My issue: i have downloaded a workflow and uploaded to my environment. good.
But i have used the option "workflow_categories.txt" to mantain an order on my workflows.
The thing is that i have modified the __init_py file in one of the workflows and use a workflow_categories called: MANAGEMENT.
The folder is there but in the modification of __init_py i have changed the "title" and the "endpoint" and even when i deleted the folder MANAGEMENT and upload a correct __init_py file with the correct info, the system detects that __init_py is going to be used as the modified version ( not the correct one).

How can i make to "restart" flask or something to update the correct values?

Regards.

Missing Apache license template in multiple community offerings

@ryutamura you are missing the required Apache 2.0 license template in several community workflows.
Please update the README.md file to include the license template as cited under Contributions>Community Template the gateway-workflows README: https://github.com/bluecatlabs/gateway-workflows/blob/master/README.md

https://github.com/bluecatlabs/gateway-workflows/tree/master/Community/service_point_watcher
https://github.com/bluecatlabs/gateway-workflows/tree/master/Community/sdwan_firewall_rule_updater
https://github.com/bluecatlabs/gateway-workflows/tree/master/Community/register_mac_address
https://github.com/bluecatlabs/gateway-workflows/tree/master/Community/query_unused_mac_address
https://github.com/bluecatlabs/gateway-workflows/tree/master/Community/query_logger
https://github.com/bluecatlabs/gateway-workflows/tree/master/Community/network_exporter
https://github.com/bluecatlabs/gateway-workflows/tree/master/Community/mist_importer
https://github.com/bluecatlabs/gateway-workflows/tree/master/Community/meraki_importer
https://github.com/bluecatlabs/gateway-workflows/tree/master/Community/lease_history_mac
https://github.com/bluecatlabs/gateway-workflows/tree/master/Community/lease_history_ip
https://github.com/bluecatlabs/gateway-workflows/tree/master/Community/flip_normal_dr
https://github.com/bluecatlabs/gateway-workflows/tree/master/Community/bulk_register_user
https://github.com/bluecatlabs/gateway-workflows/tree/master/Community/bulk_register_mac_address
https://github.com/bluecatlabs/gateway-workflows/tree/master/Community/bulk_register_ip_address
https://github.com/bluecatlabs/gateway-workflows/tree/master/Community/bulk_register_group
https://github.com/bluecatlabs/gateway-workflows/tree/master/Community/SubnetStatus