guillaumejchauveau/pictoshare

An automated document digitizer and classifier for MacOS.

PicToShare

Introduction

This educational project is an automated document digitizer and classifier for MacOS. As a group project, it aims to introduce the students to project management and establishing specifications with a client.

Objectives

PicToShare allows the user to import Documents to their Mac either by taking pictures with an iPhone/iPad or by selecting files on the computer (any kind of file). They can then indicate how to process and classify the files by selecting a Document Type. The Document Types, configured before importing a Document, allow the customisation of the importation process. They can specify an optional processing script, an AppleScript that uses the path of the Document to import to modify it or create new ones. For example, the user can choose to process pictures with an OCR software to import a PDF with the text instead. Another setting is a list of Annotations (aka. context annotations), string metadata that will be added to the Document as keywords, like the current location. Finally, the user can indicate Integrations, that will attach the Document to another application. For example, the user can choose to add a link to the Document in currently active events in their calendar. The Document can later be found via Spotlight, that indexes the content and keywords of the Document. Additionaly, a folder specific to each Document Types will contain a bookmark pointing to the Document.

Implementation details

This version of PicToShare is a native MacOS application, using SwiftUI and some AppKit views for the user interface. In order to take pictures with external devices, it uses Continuity Camera. It differs from previous versions of PicToShare, where a multiplatform application, or a Storyboard/Objective-C application, were working with an iOS app. Also, this version allows any kind of Documents, whereas the previous ones focused on pictures to PDFs on a mobile experience (hence the name).

Structure

The main XCode projet (PicToShare/PicToShare) as multiple parts. The Core folder contains the standalone logic for running the application, the Views folder the SwiftUI views for the UI, and finally files for the entrypoint and Operating System integrations like Continuity Camera or the Status Bar Item.

pictoshare's Issues

Add tests for Core Objects persistence

#19 does not include tests for this part of the Configuration Manager.

Can an Apple Script-launched app access a file in the PTS container ?

Integrate Continuity Camera

Bad news time. Continuity Camera is used by adding a NSMenuItem in a NSMenu with a specific Identifier string. SwiftUI doesn't support that for the process menus ("commands" in SwiftUI). For now the theorical workarounds are:

• have a NSMenu pop out of a button in a window, like the welcome window, using NSViewRepresentable.
• have it in the Menubar target and/or the Dock menu, we don't know how to do that so far.
• finally the worst thing: transition back to AppKit and the storyboard system, just to add the button in the process menus. We can display SwiftUI views in this system using NSHostingView, however the data management is uncertain at this point.

For now we need more exploration of the different options, but they all have compromises, take some time, and no garanties if they actually work.

Test usage of PicToShare on a clean session

Processing script execution should print stderr on error

Preference Panes

Preference panes could be use to configure PTS, instead of creating a dedicated window. We should test the capabilities of this feature.

External scripting

How can we execute an external script (say Python) that can iteract with the kernel ?

Contextual menus in Finder

Show what are the possibilities of adding options in the contextual menu (right-click) in Finder on a folder or a file (we want to have different options for both cases).

API Continuity

Peut-on choisir la résolution ?

Importation user interface

Show how we can create a window with simple UI elements inside.

Rename UI terms

Rename Annotations to Context Annotations and Context to Event Type in the UI. We will leave the internal terms as they are.

Improve code comments

Error manager

Create a manager and view for errors occurring in PTS, so that the user knows something went wrong without the app crashing.

PTS cannot import another Document if a Type prompt is in progress

The Importation Manager currently drops consecutive document type prompts if one is currently being processed. We could add a queue to store the different prompts, but the user will not be able to distinguish the different Documents. Document Formats should provide a preview using a View object to indicate which Document is being processed.

Applescripts for OCR's

We need scripts for Finereader (see the old project) and Tesseract. The args will be the input file path and the output file path (or folder, we'll see).

Core exportation

The Core infrastructure created in #10 does not support document exportation yet.

Dock

Can we drag'n'drop on the PTS icon in the dock ? The icon needs to open the main folder, but also needs to have custom options on right-click.

Calendar integration confirmation

For each importation that use calendar integration, show an interface to confirm which calendar events should be annotated with a link.

Folder and file management

Document types need to have a corresponding folder in the PTS folder. The name should correspond to the type's description. These folders will store links to imported documents. During the importation process, if a content annotator is configured, the original document is cloned with a suffix in the name for safety. The script is executed and should either modify the cloned file, or create a new one. If a new file is created, the clone is deleted. The result of the script or the cloned file if no script is configured will be linked in the corresponding type folder.

Put family names in caps in the user doc

Menus

We need to implement menus, at least in the Dock icon and the Menubar, if possible in the Finder with a Finder Sync extension.

The menu should propose to import a local file, show the Continuity options, and a button for the configuration panel.

Background service

As mentioned in #5, we will need a background service to monitor the PTS directory, and will host the core of the application. The other entry points (Finder Sync, Menubar, Dock) will communicate instructions to the service.

Show how we can create a service starting automatically and how a standard entry point (like the hello world app) can give instruction to the service.

Document type configuration view

Finish the configuration DocumentTypeView, in order to edit the document type's description, and the optional content annotator's applescript path. Context annotators will be added.

Core major refactoring

In order to simplify the importation process , the following changes will be implemented:

• Sources will be hard-coded, the Source Protocol is no longer relevant.
• Documents will no longer be represented as Data, but as temporary files. Format classes are no longer necessary. The Sources will return a URL to the input file (or not, see #32).
• The difference between Content Annotators and Context Annotators is reintroduced. The first will be in charge of creating the output file (similarly to the actual Exporter). The second will be selected by the user among a know list defined by PTS.
• Content Annotators will be implemented by Apple Scripts only, no classes required.
• Context Annotators will provide a list of Metadata that will be added to the output file. Since the possible Metadata is fixed, the Annotator Types will be hard-coded and available as a list when configuring the Document Type.
• Exporters are no longer relevant, as they are replaced by Content Annotators, represented by an URL to an Apple Script.
• The layered Configuration system is no longer relevant and will be removed.
• The Library system is no longer relevant as there are no additional classes needed (only paths to scripts).

Use bookmark path instead of Document path in Calendar integrator

Core external scripting

Add the ability to add concrete Sources, Formats, Annotators and Exporters to the Core using Python scripts.

File system interactions

• Finder Sync

The objective is to show how we can monitor new files added in a specific folder, create folders and move files.

This will be the base for the drag'n'drop and storage of documents.

Change Continuity Camera files destination

Put them in a subfolder of the PTSFolder

File name template

If the file is already existing on the computer (i.e. we're not using Continuity), we must keep the original name. Otherwise, we put the context instead.
Then, we add the document type and the date
We can seperate the different part with underscores

Select calendars for annotations/integrations

In the settings, use checkboxes to select calendars for both annotations and integrations.

Menubar

Display the possibilities of the menubar.

File storage

If the file is already existing, don't move the source.
If it is created (with Continuity for exemple), put the final product in the right folder and no link is needed