ssantoshp/cloudnerd

Hackathon project - Predict weather by scanning clouds

Data used

CCSN Database

Get Started

1. Clone or download the project on your computer and install Yarn. Yarn is the software package that will install all the dependencies and make sure the code automatically reloads when changes are made.

2. Run yarn install to install required dependencies and run yarn start to start the server in development mode. This will open a web browser to localhost:3000. By default, this project is using the TensorFlow.js exported model from Lobe found in the public/model/ folder.

3. To use your own model file, open your Lobe project, go to the Use tab, select Export, and click on the TensorFlow.js model file. When exported, drag the model.json, signature.json, and all the *.bin files to the public/model/ folder.

Additional Information

Check out the Create React App documentation for more information on React and the project structure.

There are three main components: Camera, Prediction, and StaticImage. The Camera, which runs in components/camera/Camera.tsx is responsible for displaying a live full screen view of the user's selected webcam. The Prediction component components/prediction/Prediction.tsx is the box in the lower left hand corner, and is responsible for displaying the top prediction results and their confidences. The StaticImage component components/staticImage/StaticImage.tsx displays an image selected from the file picker and runs it through the model from a canvas element.

Known Issues

TensorFlow.js on Safari may have problems initializing the WebGL backend for acceleration and will fall back to the CPU. You can use the WebAssembly (wasm) backend as an alternative to WebGL: https://www.tensorflow.org/js/guide/platform_environment#wasm_backend

Contributing

GitHub Issues are for reporting bugs, discussing features and general feedback on the Web Bootstrap project. Be sure to check our documentation, FAQ and past issues before opening any new ones.

To share your project, get feedback on it, and learn more about Lobe, please visit our community on Reddit. We look forward to seeing the amazing projects that can be built, when machine learning is made accessible to you.

cloudnerd's Issues

Add confidence scores to predictions

Having the top 3 predictions rather than just the first is quite nice. However, it would be even nicer to have a confidence score associated with each prediction. For example, a prediction tag could read "🎈 fair weather with confidence 84%

"

Predictions too Resource Intensive

The webapp currently predicts the weather from a camera image multiple times a second. This causes large resource usage (for me, of my GPU), while not being particularly useful. Since the sky does not change that quickly, I think that letting the user take individual pictures of the sky to then get predictions on that single image would be an efficient way of freeing up resources.

This would also make obtaining stable predictions easier. Currently, trying to keep the webcam on the same spot in the sky for the prediction to stay the same is a bit hard

.