Giter Site home page Giter Site logo

theodi / comma-chameleon Goto Github PK

View Code? Open in Web Editor NEW
279.0 18.0 27.0 5.5 MB

A desktop CSV editor for data publishers

Home Page: https://comma-chameleon.io/

License: MIT License

HTML 23.17% JavaScript 74.15% CSS 2.68%
data-publication electron csv csvlint desktop-csv-editor nodejs

comma-chameleon's Introduction

Stories in Ready Build Status Dependency Status

Comma Chameleon

Comma Chameleon is a desktop CSV editor (built with Electron.js) that provides a simple interface for editing data.

Developed by the ODI as a proof of concept, it has now been replaced by Data Curator (GitHub).

Summary of features

Validate your CSV on the fly using CSVlint, so you can be sure your data is reuse-ready before you publish it
Comma Chameleon lets you export your data as a Data Package, ready for publication and reuse
Integrate with Octopub and publish your data to Github

Download and Install App

Choose a platform from the Releases page.

Drag the application to your applications folder. If you encounter a warning message informing you the application cannot be opened due to emanating from an unknown developer try the following. This occurs due to Mac OS quarantining applications where it cannot determine the certificate used to sign the application.
Steps:

Right click the app, then option+click on Open.

Development

Requirements

node and npm

You can use npm to install all relevant packages and development dependencies using the following set of commands. Node and Bower install the dependencies contained in package.json and bower.json respectively

npm install -g bower electron
npm install
bower install

External (non-JS) dependencies

Comma Chameleon relies on CSVlint.sh to carry out CSV validation. This is not included in the repo, and is downloaded and installed by the build.js script. As with the page build script, this script is also run when the app is started with the npm start command.

Development: Running the full application locally

To open the app run:

npm start

This will download the non-JS dependencies (namely CSVlint.sh), build the handlebars views (from comma-chameleon/views-content) and start the app.

Application Architecture

Comma Chameleon is built using Electron.js, a framework that allows developers to build desktop applications using web technology.

There are two parts of the application, the main process and the renderer process. The main process deals with things like carrying out file operations, validating CSVs, rendering views, and exporting to Github. The renderer acts very much like client side JS in a web browser, dealing with things like presentation, and user interactions.

IPC messaging

Electron passes and listens for messages between main and renderer using the IPC module, one for the main process and one for the renderer process.

For example, when importing an Excel file, the main process reads the Excel file and opens a new window in the renderer process. Once the window has opened, the main process sends the worksheet names to the renderer process, which are then displayed to the user. Once a user selects a worksheet, the renderer process sends a message back to the main process, and the main process converts the relevant worksheet to CSV and loads it into a new window.

Views

To keep things DRY, the HTML views are written using Handlebars, see the views-content folder. The views are then built using the page-build.js script, which also gets run when the app is started with the npm start command.

Tests

Electron-Mocha has been adopted for testing, it enables both DOM and node.js testing and provides command line options to enable testing of both.

Assuming you have installed electron-mocha globally (via npm i electron-mocha -g), you can run the tests like this:

npm test

Or to run the main and renderer tests separately, you can run:

npm run test-main  # run tests for the runtime components provided by Electron
npm run test-renderer # run tests that execute client side

Otherwise you can run:

electron-mocha test/main/
electron-mocha --renderer test/renderer/ # run tests that execute client side

Deployment: Packaging

Building a new package

(This assumes you're running OSX)

Install gulp

brew install wine # This allows us to specify the icon for Windows pacakges
npm i electron-packager -g
gulp build

You can also build a package for a specific platform by running gulp build --platform={one of 'linux', 'darwin' or 'win32'}.

comma-chameleon's People

Contributors

chris48s avatar danielgavrilov avatar davidmiller avatar floppy avatar olivierthereaux avatar pezholio avatar pkqk avatar quadrophobiac avatar sheetjsdev avatar stephen-gates avatar zkamvar avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

comma-chameleon's Issues

Render CSVlint messages in standard way

Return the validation JSON in matter that correlates to the CSVLint translations.yaml file that contains ordered messages to go with errors warnings and info_messages.
Something like this but longer
document.getElementById("message-panel").innerHTML = JSON.stringify(errors)+JSON.stringify(warnings)+JSON.stringify(info_messages);
});
}

Schema Editor (JSON) Support

As a CSV schema checker,
I would like GUI support for multi file editor screens,
Including a pane to show an inline schema formatted to detect JSON

Can't make additional rows/columns via keyboard

What I tried (from a blank sheet.)
Type into column A, press tab, moves to B
Type into column B, press tab, moves to C
Type into column C, press tab, nothing happens.
Press Enter. nothing happened.

What I expected:
A column 'D' to be created; A row '2' to be created.
Could not find menu items which allowed for creation of new columns.

structure: strict order of json manifests

The current order of the respective package.json files has created duplication of bower_components and node_modules.

To resolve this for future versions I suggest creating an additional root package.json that handles node dependencies and deprecating node dependencies listed in app/package.json (node require within app/ will resolve to searching parent directory) and leaving app/package.json responsible for stipulating the electron main process.

Bower.json can remain within app/ for now as the complete modularisation of javascript requirements is still pending (there are lots of script tags that need refactoring)

app/package.json

{
  "name": "comma-chameleon",
  "version": "0.1.0",
  "main": "main.js"
}

/package.json

{
  "name": "comma-chameleon",
  "version": "0.1.0",
  "license": "MIT",
  "main": "main.js",
  "scripts": {
    "start": "electron app"
  },
  "repository": {
    "type": "git",
    "url": "https://github.com/theodi/comma-chameleon.git"
  },
  "dependencies": {
    "request": "^2.60.0",
    "xlsx": "^0.8.0",
    "node-zip": "^1.1.1"
  },
  "devDependencies": {
    "chai": "^3.2.0",
    "electron-mocha": "^0.3.1",
    "electron-prebuilt": "^0.33.0"
  }
}

Run validation against schema

As a CSV editor
I want to run a validation against an existing schema
So I can check my data is in the correct format

Insert Above/Below misnamed

In spreadsheets, Insert Above when selecting a cell/row will add a new blank row immediately above that row (e.g. if done on row 3, there will be a new blank row 3, the contents of row 3 will be on row 4 etc.) Instead the application adds at the top and bottom.

(It's not clear whether the names or the behaviour should be changed. If the behaviour is changed, the behaviour for left/right should also be changed.)

Import excel files

As a CSV editor
I want to import my initial data from Excel
So I can work on exporting it as a correctly-formatted CSV

Warning returns column number not name (letters)

E.g. I have a warning telling me that data in' column 35' is inconsistent.

But this makes it difficult the probelm, of course, are identified by letters, so in this case AI, but I have to do this manually.

Better to say 'column AI' etc. Also, column could be highlighted, even better.

Text Editing Functionality

As a desktop editing software user
I want to be able to
[ ] add new rows
[ ] add new columns
[ ] rename headers
[ ] enter text into rows and columns

Sign the OSX app

Currently the app isn't signed, so gets flagged under the unknown developer security thing.

Drag and Select to Generate Schema

As a CSV editor
I want to be able to select a range of cells
and generate from that selection a JSON schema with some fields prepopulated based on my selection

Export schema

As a CSV editor
Given I have applied a header
I would like to export a schema as JSON
So I can share it with other people

Validate CSV

As a CSV editor
I want to run a validation against CSVlint
So I can improve the quality of my data

Fixed size viewport

image
Resizing the window/Maximising the window leads to unusable screen areas which cannot be clicked on or viewed without using the scrollbar.

UI: Export as datapackage

  • First field not selected by default
  • Tab order wrong (Go/Cancel before Generate Headers
  • Keywords can contain spaces (ko, ko will lead to two different keywords)
  • Possible to not specify headers or not -- should be visibly set to default.

Visual design

As a CSV editor
I want to implement a visually appealing appearance to the application.
I want it to have a consistent theme.
To appeal to users and make it more user-friendly.

JavaScript error: Electron menu

A JavaScript error occurred in the main process.

Uncaught Exception:
TypeError: Menu.sendActionToFirstResponder is not a function
    at MenuItem.click
(<path>/resources/atom....:25)
    at Function.delegate.executeCommand
(<path>/resources/atom....:69)

What I did: "Electron / About Electron" on brand new Windows x64 install, on Windows 10.

Add loading indicator for large files

Should be possible with the Handsontable afterLoadData callback, but this seems to fire on initialize too, so any loading indicator gets dinged on initialisation

Header UI improvement

rename row 1 (as presented in hands on table) to be labelled 'header' and therefore improve user legibility of the terminology employed by ODI WRT schema generation

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.