vmware/flowgate

Project FlowGate is an open-source vendor-neutral project that helps enterprises integrate facility systems data and IT data to form a single holistic view of their operations. By combining these two disparate data sets into one view, data center operators/IT admins are better equipped to optimize operations which were previously impossible.

Flowgate

Project FlowGate is an open source vendor-neutral project that helps enterprises integrate facility systems data and IT data to form a single holistic view of their operations. By combining these two disparate data sets into one view, data center managers are better equipped to optimize operations and make smarter choices about workload placement. With Project Flowgate, we can fetch metadata and runtime metrics from various facility systems like DCIM and CMDB (Configuration Management Database), process the data from IT systems, correlate both together, and then push the results to IT infrastructure management and operation management systems. In that way, IT administrators can see updated data from facilities for IT assets in one single pane of glass. They can also view and analyze additional resources such as: power supply, cooling capacity, temperature/moisture situation for every specific server, and conduct many “facility-awared” operations, which were previously impossible.

Why

• In enterprise data centers, IT infrastructure and facility are generally managed separately, which leads to information gaps.
• Collaboration between facility and IT infrastructure systems are limited or manual, and virtualization adds more complexity.
• More and more workloads move into off prime colos or clouds where infrastructure are managed via outsourcing, making more gaps.
• Sound IT operations management and automation decisions may not be the best choice if not consider power, cooling or environmental constrains.

Features

• Built-in adapter for multiple DCIM and CMDB system integration.
  • Nlyte
  • PowerIQ
  • Infoblox
  • Labsdb
  • IBIS(TODO)
  • Open for other facility system integration.
• Built-in adapter for multiple IT stack systems
  • vCenter Server
  • Aria Operation Manager
  • Management Pack for Aria Operation Manager Pack ( Totally Free, Download Here)
  • Open for other IT stack integration. More systems will coming soon.
• UI based Integration proccess One click integration.
• Role based access control API level access control support.
• RestAPI support Provide unified facility information querying services. APIs for all operations and data query make it easy to integrate with other systems.

Get Start

DEMO

Flowgate Management Pack Demo

Demo Flowgate v1.1.2 or https://youtu.be/REbDV4Pu_-c

Demo Flowgate v1.0

Integrate with EdgeXfoundry

Install from Source code Compilation document

Install from binary Installation document

System architecture Architecture document

User Document Document

Contributing

The Flowgate project team welcomes contributions from the community. Before you start working with Flowgate, please read our Developer Certificate of Origin. All contributions to this repository must be signed as described on that page. Your signature certifies that you wrote the patch or have the right to pass it on as an open-source patch. For more detailed information, refer to CONTRIBUTING.md.

License

License

Feedback

If you find a bug or want to request a new feature, please open a GitHub Issue

flowgate's Issues

Enhance pdu load metrics

Backup/Restore/Upgrade Document

AssetNameAndIPMapping management pages

1>CRUD functions
2>Upload files to batch add mappings

New Adapter page , the content should be aligned with the column header

Cache flowgate summary data into redis.

We should create a backend job to calculate the summary data in backend and store the data into redis for later query.

refactor the base container.

Currently each component is build from a clean photon os container. we should create a base photon with jdk installed. which will save sometime for the building process.

should support update server and pdu mapping

We should provide a UI to support update the server and PDU mapping.

Remember me functionality doesn't work in login page.

UI should add the loading step when query API

Please check when it is necessary to add this loading stage for query page.

Infoblox status check

Currently when infoblox worker will not check the server status. We need to add the status check logic accordingly.

Cannot fetch Asset by name if the name contain dot

Eg. If the name like abc.efg then call api/v1/assets/name/abc.efg will not get the asset.

Labsdb Integration API

1> Add Integration
2>Modify Integration
3>Enhance Search Asset By category

Delete the mapping items when delete an integration.

Currently it only delete the integration. it should delete the mapping first.

Infoblox integration should support pass "_proxy_search" parameters

In some legacy configuration the API need to add "_proxy_search" parameter to support the AIP query to avoid permission issue.

We should make this flag optional in the configuration page.

@Pengpengwanga Please plan time to fix this issue.

PDU rated data

When there is only one value for the rated amps and voltage, we should only set the high value.

We need to support delete asset from flowgate.

Admin only functionality.

/by category

The unimplemented functionality should be hidden in the navigation menu.

Labdb Adapter UI

Create the UI for integration Labsdb system.

Generate some duplicate datas from the same Nlyte when sync metadata

Sync Three-phase power's data from PowerIQ system

Support sort in HostMapping page.

We should support in page sort in host mapping page.

We should clear the search result of the asset/sensor mapping page when re-entered

Create current load and power load metrics for host

Include the HOST usage metric in Flowgate

Currently we only include the facility metrics in Flowgate, it's quite easy to also include the host usage metrics from the IT system. Thus we can archive the bidirectional integration.
More details will discussed later.

Update the springboot version

We need to update the springboot version as the current version is going to out off support soon.

Add VROPSMP into search items in serverMapping list page

Update clarity to 4.05

update angular from 2.0 to 10.0
update clarity from 0.11 to 4.0.5

Cannot map the back humidity sensor

when try to mapping a group of hosts for their pdus and sensors,

after several round you will find that you cannot save the back humidity sensor.

refresh the page will solve the problem. So this should be a UI bug.

Enhance infoblox worker

Currently we don't have mac address information from infoblox, we should include it which will be used in the auto-mapping logic.

Issue Template

Describe the detail information of the Issue.

Loss metrics foumula for pdu assets

Should support server and switch mapping

We should support update server and switch mapping information in the UI.

Push power load and current load metrics to VROPs

Click Flogate log will redirect to a unwanted 404 page.

It should redirect to the default home page.

Save the rackU usage information in cabinet.

Content in serverMapping page cannot display correctly

When you switch between different types of the integration, some of the content will not display correctly.

Reproduce step:

1> Select "vRealize Operation Management Pack" and select a server.
2> Select the "VMware vCenter"
3> ReSelect "vRealize Operation Management Pack" you will see something like fowllow:

The job of merging PDU can't execute due to the assetName may be empty

BMC adapter support.

With BMC(Baseboard management controller) we can get a lot of useful metrics of a server. Currently we still not have this adapter ready.

There are two cases.

1. The user has a BMC management system. in this case we can extract data directly from the BMC management system directly which is more easier.

2. The use don't have BMC management system, instead the adapter need to connect directly to the BMC on the motherboard. In that case. we need to create a simple UI for the BMC connection management. Beside that there are different type of BMC implementation from different vendors. We need to address which we will support in the first version.

DocumentDoesNotExistException in the job of aggregateAndClean pdus

The API for getting server metrics data should not return the all outlets value of pdus which are connected with server

The type of facilitySoftware intergration is not display when open the edit integration page

The new integration is ok but the old data is not. we should compatible with old data

We should support delete integration with MP type.

We should support delete integration of MP type incase that MP cannot delete the integration by default.

Mapping Sensor is not working.

By default the sensor and server mapping are created automatically , but in some case we still need manually mapping. The manually Mapping sensor and host button is not working now.

Can't find any serverMapping when the type of SDDCSoftwareConfig is vropsMP.

Host name and IP mapping

1>APIs for mapping management
2>Testcases for these new apis

Update integration status

1.When an integration throws an exception, the web page should display the corresponding state to prompt the user to perform the corresponding action to solve the problem.
2.Check the status of the integration before performing each job.

It will be better to display chassis summary data in summary page

We recently add a new Chassis type in Flowgate, so it will be better also display the chassis summary in summary page.

Sync Chassis information from nlyte system

Dynamically calculate the item number of predefined data in database start up script.

Currently we use
if [ -n "$DATACOUNT" ] && [ ${DATACOUNT#*:} = "95" ];then
in make/database/init.sh
to check whether the database has been successfully initialized. Each time we add new data items we need to change the number. we should dynamically calculate the items number instead of hardcode.

Enhance the schedule for adapter