Giter Site home page Giter Site logo

prescottprue / fireadmin Goto Github PK

View Code? Open in Web Editor NEW
239.0 7.0 40.0 22.44 MB

Application for Managing Firebase Applications. Includes support for multiple environments and data migrations

Home Page: https://fireadmin.io/

License: MIT License

JavaScript 71.05% CSS 0.07% HTML 0.78% Dockerfile 0.51% TypeScript 27.57% Shell 0.02%
firebase firestore firestore-react admin-dashboard redux-firebase redux-firestore data-migration data-management

fireadmin's Introduction

fireadmin

Build Status Cypress Dashboard License Code Style

Application for Managing Firebase Applications. Includes support for multiple environments and data migrations

Table of Contents

  1. Features
  2. Getting Started
  3. NPM Scripts
  4. Application Structure
  5. Run Your Own
  6. Requirements
  7. Before Starting
  8. Testing
  9. Cloud Functions Unit
  10. App E2E
  11. Deployment
  12. FAQ

Features

  • Manage multiple environments as a single project
  • Project Sharing (invite by email coming soon)
  • "Action Runner" for common project actions such as data migrations, and generating reports
  • Action Features include support for:
    • Multiple steps allowing many actions in one run
    • Backup phase (for easy backing up data before running your actions)
  • Project level tracking of actions which have been run through Action Runner
  • Get/Set CORS Config of Storage Buckets
  • Testing for React App (Cypress) and Cloud Functions (Mocha)

coming soon

  • Support for copying Single Firestore Document in Copy Action
  • Map action - for mapping each item in a collection both on RTDB and Firestore
  • Authorized Google API Request Panel
  • Invite new users by email
  • User manager (including role assignment)
  • Data Viewer

Interested in adding a feature or contributing? Please open an issue!

Getting Started

Since this is source code, a great place to start is checking the hosted version of Fireadmin available at fireadmin.io.

NPM Scripts

While developing, you will probably rely mostly on npm start; however, there are additional scripts at your disposal:

yarn <script> Description
start Serves your app at localhost:3000
start:dist Builds the application to ./dist and Serves it at localhost:3000 using firebase serve
functions:start Runs Functions REPL locally (uses firebase functions:shell
functions:build Builds Cloud Functions to ./functions/dist
functions:test Runs Functions Unit Tests with Mocha
build Builds the application to ./dist
test Runs E2E Tests with Cypress. See testing
lint Lints the project for potential errors
lint:fix Lints the project and fixes all correctable errors

Husky is used to enable prepush hook capability. The prepush script currently runs eslint, which will keep you from pushing if there is any lint within your code. If you would like to disable this, remove the prepush script from the package.json.

Application Structure

├── .github                      # Github Settings + Github Actions Workflows
│   ├── deploy.yml               # Deploy workflow (called on merges to "master" and "production" branches)
│   └── verify.yml               # Verify workflow (run when PR is created)
├── cypress                      # UI Integration Tests
├── docs                         # Docs application (built with Gatsby)
│   ├── content                  # Content of docs (written in markdown)
│   ├── components               # React components used in docs app
│   ├── gatsby-config.js         # Gatsby plugin settings
│   └── gatsby-node.js           # Gatsby node definitions (how templates are combined with content)
│   └── package.json             # Docs package file (docs specific dependencies)
├── functions                    # Cloud Functions (uses Cloud Functions for Firebase)
│   ├── src                      # Cloud Functions Source code (each folder represents a function)
│   └── index.js                 # Functions entry point
├── public                       # Public assets
│   ├── favicon.ico              # Favicon
│   ├── firebase-messaging.sw.js # Messaging Service worker (loaded by Firebase SDK)
│   └── index.html               # Main HTML page container for app
├── src                          # Application source code
│   ├── components               # Global Reusable Presentational Components
│   ├── containers               # Global Reusable Container Components
│   ├── layouts                  # Components that dictate major page structure
│   │   └── CoreLayout           # Global application layout in which to render routes
│   ├── routes                   # Main route definitions and async split points
│   │   ├── index.js             # Bootstrap main application routes with store
│   │   └── Home                 # Fractal route
│   │       ├── index.js         # Route definitions and async split points
│   │       ├── assets           # Assets required to render components
│   │       ├── components       # Presentational React Components
│   │       ├── modules          # Collections of reducers/constants/actions
│   │       └── routes **        # Fractal sub-routes (** optional)
│   ├── static                   # Static assets
│   └── utils                    # Application-wide utils (form validation etc)
├── .firebaserc                  # Firebase project settings (including settings for CI deployment)
├── cypress.json                 # Cypress Integration Test settings
├── database.rules.json          # Firebase Real Time Database Rules
├── firebase.json                # Firebase resource settings (including which folders are deployed)
├── firestore.indexes.json       # Firestore Indexes
├── firestore.rules              # Firestore Database Rules
└── storage.rules                # Cloud Storage Rules

Run Your Own

Requirements

Before Starting

  1. Make sure you have enabled billing on your Firebase account - external API communication requires setting up a payment method (you are only charged based on usage)
  2. Create an account on Algolia - Create a new app, you will need the API keys later
  3. Install Firebase Command Line Tools: npm i -g firebase-tools

Local Environment Setup

  1. Install dependencies: yarn install

  2. Create a Web app within the Firebase Console of your project (config will be used in next step)

  3. Create a .env.local that has the following format (with your values filled from previous step):

    REACT_APP_FIREBASE_apiKey=<- api key ->
    REACT_APP_FIREBASE_authDomain=<- auth domain ->
    REACT_APP_FIREBASE_databaseURL=<- database URL ->
    REACT_APP_FIREBASE_projectId=<- project ID ->
    REACT_APP_FIREBASE_storageBucket=<- storageBucket ->
    REACT_APP_FIREBASE_messagingSenderId=<- message sender ID ->
    REACT_APP_FIREBASE_appId=<- project app id ->
    REACT_APP_FIREBASE_PUBLIC_VAPID_KEY=<- project app id ->
    REACT_APP_ALGOLIA_APP_ID=<- ->
    REACT_APP_ALGOLIA_API_KEY=<- ->
  4. Create functions/.runtimeconfig.json file that looks like so:

    {
      "algolia": {
        "api_key": "<- your API KEY ->",
        "app_id": "<- your Algolia APP ID ->"
      },
      "gmail": {
        "email": "<- gmail account for sending invite emails ->",
        "password": "<- password for ^ email ->"
      },
      "encryption": {
        "password": "<- your own made up encryption password for service accounts -> "
      }
    }
  5. Set Functions config variables to match the file you just made (for the deployed version of your functions):

    firebase functions:config:set $(jq -r 'to_entries[] | [.key, (.value | tojson)] | join("=")' < functions/.runtimeconfig.json)
  6. Build Project: yarn build

  7. Deploy to Firebase: firebase deploy (deploys, Cloud Functions, Rules, and Hosting)

  8. Start Development server: npm start NOTE: You can also use yarn start:dist to test how your application will work when deployed to Firebase

  9. View the deployed version of the site by running firebase open hosting:site

Deployment

CI Deploy (recommended)

Note: Config for this is located within .github/workflows/app-deploy.yml. firebase-ci has been added to simplify the CI deployment process by getting settings from the .firebaserc. All that is required is providing authentication with Firebase:

  1. Have at least two Firebase projects to ready to use, one for each environment (staging and production)

  2. Replace info within .firebaserc under both the projects, ci, and targets sections

  3. Login: firebase login:ci to generate an authentication token. This token will be used to give the CI provider rights to deploy on your behalf. Settings are provided for Gitlab, but any CI provider can be used.

  4. Set FIREBASE_TOKEN environment variable within Github Actions secrets

  5. Add the following environment variables to Github Actions's variables (within /settings/ci_cd):

    FIREBASE_TOKEN; // Used to deploy to Firebase (token generated in last step)
    CYPRESS_RECORD_KEY; // Record key for Cypress Dashboard
    SERVICE_ACCOUNT; // Used to authenticate with database to run hosted tests
  6. Run a build on Github Actions by pushing code to your Git remote (most likely Github)

For more options on CI settings checkout the firebase-ci docs.

Manual deploy

  1. Build Project: yarn build
  2. Deploy to firebase: firebase deploy NOTE: You can use firebase serve to test how your application will work when deployed to Firebase, but make sure you run yarn build first.

Docs

Documentation is available at fireadmin.io/docs

All source code and content for docs is located within the docs folder. Docs are generated from markdown into static files using Gatsby based on settings in gatsby-config.js.

Visit the docs README for more info.

Testing

NOTE: If you have setup CI deployment, E2E tests and Unit Tests can automatically run against your staging environment before running the production build.

Cloud Functions Unit Tests

Cloud Functions Unit tests are written in Mocha with code coverage generated by Istanbul. These tests cover "backend functionality" handled by Cloud Functions by stubbing the functions environment (including dependencies).

Run Locally
  1. Go into the functions folder: cd functions
  2. Confirm you have dependencies installed: npm i
  3. Run unit tests: npm test
  4. To also generate coverage while testing, run yarn test:cov

App UI Tests

End to End tests are done using Cypress and they live within the cypress folder. These tests cover UI functionality and are run directly on the hosted environment of Fireadmin. Application end to end tests are run automatically in Github Actions the after deploying to the staging environment before deploying to production.

Run Locally
  1. Create a service account within the Firebase console

  2. Save the service account as serviceAccount.json within the root of the project

  3. Get the UID of the user that you want to use while testing from the Authentication tab of the Firebase console to

  4. Create cypress.env.json with the following format:

    {
      "TEST_UID": "<- user account's UID ->"
    }
  5. Run yarn emulators. This will bootup the emulators (pointed to during testing)

  6. In a different terminal yun yarn start:emulate. This will bootup the application pointed to the emulators

  7. In a different terminal tab, run yarn test:emulate:run. This will run Cypress integration tests pointed at emulators (for seeding and verifing)

To Open Cypress's local test runner UI where you can run single tests or all tests use yarn test:emulate.

FAQ

  1. Why node 12 instead of a newer version? Cloud Functions runtime supports up to 12, which is why that is what is used for the CI build version. This will be switched when the functions runtime is updated
  2. Uploading service accounts? Where do they go and how are my service accounts stored? When uploading a service account, it first goes to a Google Cloud Storage Bucket which has security rules and does not have CORS access. The copyServiceAccountToFirestore Cloud Function converts it into an encrypted string, stores it within Firestore, then removes the original file from Cloud Storage. Firestore rules keep anyone that is not a collaborator on your project using or reading the service account. Since it is associated with a specific environment, you can then limit access to what can be done with it right in the Users/Permissions tab of Fireadmin.

fireadmin's People

Contributors

dependabot[bot] avatar gregfenton avatar prescottprue avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

fireadmin's Issues

bug(actionRunner): when action run fails, message does not contain error

Describe the bug
When running an action - if the action fails, the message should be clearly visible to the user. It would also be good if this could be accessed in the events view.

To Reproduce
Steps to reproduce the behavior:

  1. Go to the action runner
  2. Run an action that is bound to fail (i.e. trying to copy an rtdb path)
  3. Wait for error to appear
  4. Message does not clearly indicate that the failure was because of data not being there (even though actionRunner cloud function does write that message back to the db)

Possible Solution

  • Display the message which already exists within the snack that is displayed
  • Do the same for browser notification

Additional context/Screenshots

feat(actions): ability to toggle strictTriggerValidation from an action

Is your feature request related to a problem? Please describe.
Feature - When running a copy action with data that is large into a database which has cloud functions with database triggers, it can cause too many functions triggers to go off. This can be toggled off with strictTriggerValidation through a curl call:

curl -X PUT -d "false"  https://<namespace>.firebaseio.com/.settings/strictTriggerValidation/.json?auth\=<SECRET>

As mentioned in this stackoverflow post

Describe the solution you'd like
Either a "database settings" action or an option to add an input for toggle this to copy action templates

Describe alternatives you've considered

Additional context

Question: avoiding gRPC errors with Cypress

Hello, glad to find this library! I spent about a couple days trying to figure out why I couldn't get my cypress tests to run. I opened an issue on the Cypress repo (cypress-io/cypress#3570), but ultimately ended up trying TestCafe which is having it's own issues getting gitlabCI running correctly. Honestly, I preferred the sensibilities of Cypress and want to try again.

Can you help with this issue? It appears that your library is mostly for manually calling into Firebase/Firestore within the test. I would like my tests to be purely concerned with clicking and expecting the interface to change. Just like the real app, there is no manually POSTing or GETing during the tests. Do you have examples like that?

To clarify, it would be a test like:

  1. Cypress selects a button and clicks it.
  2. App code makes a GET request to firestore, and returns data that the app uses to populate a list.
  3. Cypress verifies the contents of the list.

key elements in table of contents are missing

In the main README.md, some key elements from table of contents are missing. For example, getting started is missing. I am not able to get it working on my local system and having something like getting started would be a nice welcome.
Thanks.

fireadmin.io is blank

Bug Description

In the current Chrome version (77) the fireadmin.io website is completely blank (see screenshot). It works fine with Firefox and Safari. The console outputs the following error messages:

Failed to load resource: net::ERR_BLOCKED_BY_CLIENT
Uncaught ReferenceError: Raven is not defined

Expected Behavior

I expect to see some content.

Additional Context/Screenshots

Screenshot 2019-09-30 at 08 30 42

feat(core): data viewer/querier/search

Description Of Potential Feature

  • A data viewer/querier that allows for querying Real Time Database + Firestore with a specified service account

Additional Issues This May/Does Solve

  • Multiple filters on queried data (since Firebase console is limited)
  • Creating reports on numbers of documents
  • Generating data export files which consist of a subset of data

Additional Context

Querying data within the Firebase console is pretty limiting. For Firestore you can only query by one parameter, and for Real Time Database there is only viewing the data in tree form. This viewer/querier/searcher could allow for queried data to also be filtered.

Ideas

  • Allowing for export of json file which could then be used in insert data into another database
  • Searching could be based on provided Algolia information or through user wiring up a webhook to Fireadmin (instructions would be included)

Some more discussion in #90

Can we add docker support ?

Adding docker support would help people to quickly deploy, make a quick change and clear the environment and it could also help in dependency management as well.

feat(actionRunner): generated URL for sharing/loading actionRunner settings

Description Of Potential Feature
We often times want to repeat the same migration between two environments that we recently ran between two other environments. It would be great to have the following abilities

  • generate a URL representing the settings currently within the action runner
  • load settings into actionRunner through custom URL

Issues This May/Does Solve

  • Migrations can easily be part of a PR by providing the settings URL
  • Another method other than "redo" button to load previous settings
  • Test will be able to run faster since action runner can be pre-populated with a template instead of searching Algolia each time

bug: delete member incorrectly updates project permissions

Bug Description

Steps To Reproduce

  1. Delete a member from a project
  2. See that no members are now visible in the members UI

Expected Behavior

Possible Solution

Additional Context/Screenshots

Came after recently reactfire update, may be related

"Error: Failed to get document because the client is offline" when using FireAdmin.io

Bug Description

I am experiencing a bug when first logging into the platform, where I am getting an “Error: Failed to get document because the client is offline” toast message. When I try to run an Action, it runs and then ends up failing and therefore, not migrating the campaign.

Steps To Reproduce

Login to fireadmin.io
See an error will appear: “Error: Failed to get document because the client is offline.”
Go to Action Runner
Choose template: ‘Copy Firestore collection’
Add custom values

Expected Behavior

The Actions should run and the campaigns be migrated successfully

Additional Context/Screenshots

In the Project Events section, the system is showing an error.
Screenshot 2020-06-12 at 1 55 12 pm

Sign up link leads to a 404

Bug Description

FYI, the "SIGN UP" button on the login page leads to a 404.

Steps To Reproduce

  1. Go to https://fireadmin.io/login
  2. Click on SIGN UP and it will result in 404.

Expected Behavior

Maybe the button should be removed with some additional text on how the signup/login are the same?

feat(actions): ability to set "runner type" for more resource intensive actions (long time or high memory/CPU)

Is your feature request related to a problem? Please describe.
When running actions that use a large amount of memory or run for a long amount of time, they often fail. This is due to the fact that actions are currently run using the actionRunner cloud function, which can easily time out or use all of its memory

Describe the solution you'd like
"Runner Type" dropdown that would allow the user running the action to specify a more robust runner

Additional context
Something else to keep in mind is that this may want to be part of the action template as well as the ability to edit this. For example - when making an action template that requires many collections to be copied, the template should be able to dictate a runner that can handle long running time as well as not allow the user to change this (since it could cause that action to fail).

The amount of runs per user and project should be tracked as well since running dedicated VMs can rack up price quickly. In the future it should be easy enough to limit the usage of dedicated resources based on a payment plan.

bug(actionRunner): inputs should not required in action-templates

Bug Description

  • Action runner templates should not require inputs, but if they are not passed currently

Steps To Reproduce

  1. Create an action-template
  2. Add steps with constants in them (no inputs)
  3. Attempt to run an action using this template - it throws an error

Expected Behavior

  1. The action should run without error

Possible Solution

Additional Context/Screenshots

image

still using "fireadmin"?

Scott,

I see you're work revolves around similar concerns to mine (redux and firebase anyway). I imagine that you may still be using this repo but since there were no recent commits to it I thought I'd ask because I have firemodel and firemock which i'm currently developing and purely from a naming convention standpoint I'd love to have fireadmin for a private repo I'd like to OSS. That said, if you're using it I can just find another name.

chore(functions): replace googleapis with google-auth-library

Description Of Potential Feature

  • Switch from using googleapis dependency to google-auth-library in Cloud Functions - it is only used for JWT auth using service accounts, so should be virtually the same syntax
  • Remove the skipping of googleapis in dependabot settings

Additional Issues This May/Does Solve

  • Not having to update major versions of googleapis - which is released often without an impact to functionality being used

Additional Context

  • google-auth-library is lighter weight and is just for auth - googleapis actually uses it for auth under the hood

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.