benawad/destiny

Prettier for File Structures

Destiny

Prettier for File Structures

---

Motivation: https://www.youtube.com/watch?v=rGYbrIf-y58

What does this do?

1. Scans a folder for all the files in it
2. Creates a directed graph based on how the JavaScript/TypeScript files import each other
3. Creates a fractal representation of the graph (following similar rules outlined here: https://hackernoon.com/fractal-a-react-app-structure-for-infinite-scale-4dab943092af)
4. Moves existing files into the fractal structure
5. Fixes imports
6. Removes all empty folders
7. Prints files that are "unused" (not imported by anyone and doesn't import anything)

The end result is a "prettified" file structure.

Disclaimer

• This is a work in progress and 100% has bugs of some kind in it
• BEFORE running this tool on your codebase, make sure you have committed to git or made a backup (I don't expect the tool to destroy your work (although it's possible), but just in case you don't like the results)
• Snapshot tests don't format correctly
• Only works on JavaScript/TypeScript codebases (although this concept could probably be extended to any language)

How to run it

$ destiny --help
destiny - Prettier for file structures.

USAGE

  destiny [option...] [path]

  The path argument can consist of either a file path or a glob.

OPTIONS

  -V, --version              Output version number
  -h, --help                 Output usage information
  -w, --write                Restructure and edit folders and files
  -S, --avoid-single-file    Flag to indicate if single files in folders should be avoided
  --debug [?output file]     Print debugging info

Dry run which will output what the resulting file structure will look like:

npx destiny "src/**/*.*"

This will actually move files around and fix imports:

npx destiny -w "src/**/*.*"

Documentation

You can find the full documentation at this url.

This tool might be useless

It might be better to just name your folders.

Why did you name it destiny?

Contributing

Pull requests are welcome :)

Badge

[![file structure: destiny](https://img.shields.io/badge/file%20structure-destiny-7a49ff?style=flat)](https://github.com/benawad/destiny)

destiny's Issues

Doesnt work on windows

Running npx destiny "src/**/*.*" or just npx destiny src prints out a tree that looks like a nice end result - but files and folders are untouched.

Feature requests and suggestions

This tool has great potential. Here are some features I could think of.

• Resolving multiple files with the same extension(Not sure if it's already supported)
  If a folder has .js and .jsx and has been imported without extension, resolve order matters here. Make it configurable, optionally use butler.config.js?

• Watch mode?

  • Need to rethink performance issues as we are reading and modifying most of the files on every run
  • When to really move the file in watch mode?
    ‎

• Dry run
  Potentially there will be too many folders/files movement, this might not be what the user always wants by default. There should be a way to see what happens before it actually happens. I could think of two options(Note: we already have information about this in the graph).

  • Print visual diff in the terminal before applying the changes(maybe too long for large projects)
  • Move files and folders and prompt user do you want to keep the changes, If no, revert the changes

These features are already suggested, but, re-adding here with a bit more context.

• Ignore directories/files/regex pattern
  Some tools like next.js and Gatsby depend on the folder structure. Allow skipping some of the folders and files(#11)

• Support running programmatically
  Allow running via node(or any other tool) for flexibility. also, we might need some sort of config file like butler.config.js(oh, no, another config file?)(#8)

We can also have a GitHub project board where we can track progress. I'm really excited about the future of this project 🎉

Enable Gitter

Hey 👋,

Some of the discussions we have in issues are better suited elsewhere like Gitter. @benawad what do you think and could you possibly enable it for destiny? A alternative is of course IRC @ freenode, depending on how many use IRC on a daily basis here.

https://gitter.im

Improve test coverage

Hey 👋,

We would love some help with improving our test coverage. You can either visit https://codecov.io/gh/benawad/destiny/branch/develop or run yarn test:coverage to get more details. I recommend going through the generated html files to specifically see what coverage is missing.

I think this is pretty crucial now when we're starting to iron out bugs and other issues. It would be very comforting if we could trust our tests a bit more.

Pull Request merging strategy

Since we are using conventional commits and semantic-release we should probably only use squash and rebase for merging PRs. If the commits for a PR are using the conventional commits spec it would be merged using a rebase, else using a squash merge to change the name of the commit/PR.

reserved paths

So I tried destiny on a next.js project and realized that we need to have some way to exclude folders/files.

Example, in next.js we need to keep code for pages and api in

pages
pages/api

Destiny's trying to restructure scss files

By now, Destiny could easily restructure scss files. And it seems like it's trying, without knowing, but not succeeding.

If a js/ts file imports a scss file like so:

import './index.scss';

and this index.scss is importing another scss file without specifying the .scss extension in the path:

@import './foo';

it will throw this error:

(node:9201) UnhandledPromiseRejectionWarning: Error: Cannot find module './foo' from '/home/anatole/dev/destiny-cra/src/index'
    at Function.resolveSync [as sync] (/home/anatole/.npm/_npx/9201/lib/node_modules/destiny/node_modules/resolve/lib/sync.js:81:15)
    at customResolve (/home/anatole/.npm/_npx/9201/lib/node_modules/destiny/lib/destiny.js:144:18)
    at /home/anatole/.npm/_npx/9201/lib/node_modules/destiny/lib/destiny.js:182:33
    at Array.forEach (<anonymous>)
    at buildGraph (/home/anatole/.npm/_npx/9201/lib/node_modules/destiny/lib/destiny.js:175:21)
    at formatFileStructure (/home/anatole/.npm/_npx/9201/lib/node_modules/destiny/lib/destiny.js:534:9)
    at run (/home/anatole/.npm/_npx/9201/lib/node_modules/destiny/lib/destiny.js:650:9)
    at Object.<anonymous> (/home/anatole/.npm/_npx/9201/lib/node_modules/destiny/lib/destiny.js:654:3)
    at Module._compile (internal/modules/cjs/loader.js:936:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:947:10)
(node:9201) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)
(node:9201) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

So the question is: are we extending to preprocessors like Prettier, or are we ignoring non js/ts files?

---

I made a repo to replicate:
https://github.com/AnatoleLucet/destiny-scss-error
Simply clone it, and run npm destiny src

Problems with global scss

If am has a global imported scss file that defines e.g. colors, it seems that this is not recognized and the following error message appears:

INFO: Generating tree for: src/**/*.*
ERROR: Cannot find import ./src/css/styles

edit:
example scss import:
src/css/styles.scss

@import './src/css/styles';

Duplicate unused files

For some reason, it displayed an unused file twice.

Also, I'd like to see a tree view of the unused files so there's less ambiguity of files/more visual. AT least for the CLI portion of this.

src/
|-- /index.html

$ /home/wayne/Documents/apollo-orm-test/node_modules/.bin/destiny src
INFO: Resolving files.
INFO: Generating a graph and fractal tree.
INFO: Fixing imports.
INFO: Moving files.
WARN: Unused files:
        index.html
        index.html
INFO: Successfully restructured!
Done in 0.16s.

Feature Request/Exploration: `de-structure` flag

I've very much enjoyed using destiny, and I have found it to be very useful even with it's limited API (in many ways, I actually enjoy the lack of customizability).

Anyways, during the course of my usage I've come to the conclusion that it might be very helpful to have a de-structure flag. That is, a flag that takes a structured folder and returns a flattened version in which all the files are in the same folder at the same level. This is, of course, not useful for viewing the files, but it could be very useful for version control since keeping files in an unstructured way would eliminate file move base commit diffs.

The workflow I imagine, is keeping my local system formatted via destiny and using a git hook to de-structure my project before committing and re-structure my project on pull. I'd also, push the destiny config. So, when cloning to a new system I could re-structure the project.

The advantages to this are that each dev could structure their code how they like (assuming the structure can be generated programmatically), and version control diffs would be minimized to actual code changes. I realize that this would not be a workflow all destiny users would enjoy, but I expect, some would at least like to try it.

Fails on imports that are part of a string

Our project is somewhat unique in that it has to handle evaluating arbitrary code, so our tests include stringified code examples, some of which include import statements. Since this project uses a regex to capture all import statements it is being tripped up by e.g.:

const sampleUserCode = `import { MyButton } from "./mybutton"`

In our project we use Typescript to parse and tokenise a source file, and then traverse the nodes to find all of the import statements. In a very simplified nutshell it looks something like this:

import * as TS from 'typescript'
const sourceFile = TS.createSourceFile(filename, sourceText, TS.ScriptTarget.ES3)
const topLevelNodes = sourceFile.getChildren(sourceFile)
const imports = topLevelNodes.filter(TS.isVariableStatement)

It's unlikely to be a simple change but would certainly make this project more robust (I saw there was previously this similar issue #42). We used https://ts-ast-viewer.com/ extensively to help us understand that AST generated by Typescript (it's open source too: https://github.com/dsherret/ts-ast-viewer). Alternative to that there is also acorn (https://github.com/acornjs/acorn) for doing a similar thing with js code.

Destiny only treat import paths that start with a dot

Our regex only match with import paths that start with a dot.

That's an issue because destiny is trying to resolve this import's package name as a path:

import '..something';
// or
import '.something';

Also I don't think npm is allowing dots in packages name. But destiny should probably ignore this kind of import anyway...

See:
https://regexr.com/4vc8q

Glob stays in the tree's root path

The root file of the tree is src/**/*.*, it should probably be a simple src.

config file and flags

As mentioned in #3, it could be nice to have some options. Either with some flags in the cli or a config file.

Here's some of my obvious ideas when we speak about options for a cli:

• Ignored pattern & files/folders names. For example: butler-cli src --ignore *.js ingore js files
• Some kind of regex (like ESLint is doing) for the first argument which is the path. For example butler-cli src/**/*.{ts, tsx} would only apply changes to .ts & .tsx files.

Any thought?

pre-commit hook

It would be great to have a pre-commit hook native in the destiny project:

https://pre-commit.com/

Updating imports without moving files

Hey 👋,

One issue we're having is that when you run destiny on the src folder we fix the imports but not on other folders that also import those files like a test folder. This issue is a bit tricky as one might argue that if we specify only the src folder it should only restructure and fix the src folder.

I do think that most people would assume that the imports in test was fixed too. If you run destiny on itself you have to, in many cases, manually go and repair the test imports.

Test File Location

When using jest, it's common practice to name the test files ComponentName.test.js and put them in an __tests__ directory next to the component. Since these tests import the component and the component does not import the tests, destiny naturally tends to flip the directory structure around. This is unfortunate as it makes it look like the test is the "entry point" rather than a supporting file. Another example of the same pattern is storybook stories, that are typically files that end in .story.js and should probably be handled in the same way as jest tests.

Ideas:

1. We could treat anything that has a "sub extension" i.e. some-file.some-extenstion.js as being a "supporting file" and keep it beneath the component. Not sure how you do the __tests__/__stories__ bit, but maybe it would be sufficient just to exclude it from the graph and then put it next to the file?
2. We could accept a config for what files should be treated specially.

I think the best option might be to force people to abandon the __tests__/__stories__ folders and just have

Input:

- MyComponent.js
- __tests__/MyComponent.test.js
- __stories__/MyComponent.story.js

Output:

- MyComponent.js
- MyComponent.test.js
- MyComponent.story.js

but crucially never:

- shared/MyComponent.js
- MyComponent.test.js
- MyComponent.story.js

Unexpected token { and no logging or other error output

Hey destiny team,

I'm working with a webpack / babel project with Javascript and JSX mixed in and I'm getting this error when running the utility.

Fri Feb 28 12:03 PM code_base: npx destiny "src/**/*.*"
Unexpected token {

Unfortunately I have no idea how to proceed further without any additional output.

Cannot require referenced json file via relative path within a typescript project having an outDir setting

Given a typescript project that has set: outDir as "./dist/" in the compilerOptions. There's one script that loads in its own package.json via require and a relative path:

const pkg = require("../../../package.json");
console.log(pkg);

$ npx tsc && node dist/bar/baz/file.js 

{ name: 'ex',
  version: '1.0.0',
  description: '',
  main: 'index.js',
  scripts: { test: 'echo "Error: no test specified" && exit 1' },
  author: '',
  license: 'ISC',
  dependencies: { '@types/node': '^13.7.6', typescript: '^3.8.2' } }

destiny throws an error on the require line:

$ destiny "src/**/*.*"
INFO: Resolving files.
INFO: Generating tree for: src/**/*.*
ERROR: Cannot find import ../../../package.json
If you think this is a bug, you can report it: https://github.com/benawad/destiny/issues

I assume the outDir trips destinity up as it then assumes an incorrect root folder.

Suggestion: Avoid duplication as file+folder

Currently we do this:

App/dep.js
App.js

This is confusing to me because if I import ./App - it seems controversial to have a folder and a file of the same name in the same directory...

I recommend that we do this instead:

App/dep.js
App/index.js

Disallow folders with one file

For me, having only a single file sitting in directory is not nice.
For example in this repo: https://github.com/benawad/butler-cli/tree/9ad6e4c11ee096314402060f541047253336c5e1/src/index/formatFileStructure/syncFileSystem/fixImports

I would prefer if butler places the makeImportPath.ts into syncFileSystem folder next to fixImports.ts.

If you could add this disallow-leaf-folders-with-one-file config that would be just great! WDYT?

Appears to crawl up to dangerous parent directories.

No worries, no harm was done, but I ran into this error and didn't think it was worth trying to fix to get the script continuing:

ENOENT: no such file or directory, scandir '/Users/utils'

I was operating the tool from /Users/amadeus/Development/git_repo/subfolder

Could probably use something to prevent it from trying to crawl up and outside of the existing execution directory.

Ignore-rules

I have a folder where some typescript code is being generated into, so I need to be able to add some kind of ignore rule so that it leaves those folders alone.

Use TypeScript's resolver to resolve modules to files

This would allow support for repositories with TypeScript's path-altering settings like paths and alias.

Errors for commented out imports and exports

The tool does not recognize when an invalid import statement has been commented out, like this:

// export { nonsense } from './invalid/path';

It throws this error until we delete the line:

UnhandledPromiseRejectionWarning: Error: Cannot find module './invalid/path' from 'some/folder'

It would be great if it would ignore these lines - does the currently logic depend on pattern matching or does it parse the AST of the files?

Restructure subfolders

In this example of fractal structure from this article, they are showing how they use the fractal structure only in sub folders of src/.
So src/ isn't structured in a fractal way but sub folders like components, pages, modules... are.

I only see one way of doing this with the current version of Destiny. Executing Destiny in each sub folders (destiny src/components, destiny src/pages...). For sure that's not great.

CLI idea:

destiny src --sub=components --sub=pages # ...
# or 
destiny src/components src/pages # ...
# or
destiny src/* --ingore src/utils

--ignore flag: #8, #11, #17

Config file idea:

paths: [
  'src/component',
  'src/pages',
  // ...
]

// or a more "tsc like" way
rootDir: './src',
paths: [
  'component',
  'pages',
  // ...
]

Does not work with a plain create-react-app project

Try:

npx create-react-app test
cd test
npx butler-cli src

npx: installed 2 in 0.992s
Object.values(...).flat is not a function

I added a GitHub Action for Deploying to NPM

@sQVe @AnatoleLucet

I just grabbed an existing NPM publish action, so feel free to swap it out if you want something different, but you can now deploy the package to NPM.

Now, when someone changes the version in package.json to 1.2.3 and pushes a commit with the message Release 1.2.3, the GitHub action will create a new tag v1.2.3 and publish the package to the NPM registry.

Feature: Support for moving linked files in tandem

Files can have linked files that should also move when the original file moves.

• RN images: Moving icon.png should also move [email protected] and [email protected].
• Jest: Moving add.js should also move add.test.js.
• TypeScript: Moving add.ts should also move add.d.ts.

Possible CLI syntax:

• destiny src --linked @2x,@3x
• destiny src --linked .d
• destiny src --linked .test

Improve debuggability

I think we need a debugger for the file structure because when people use this in their real projects, they may not want to sure their whole repository before and after the change.

Something small that reads the folder and file structure before and after a change.

Motivation for this come from using destiny in a private repository and having some unexplained folder and file renames.

Are there any other features for debugging that might come in handy? I'm pretty sure I can build this.

• debug flag
  • add debug flag that logs the following:
    • pre structure change
    • post structure change
    • print all trees, graphs, imports, dependencies and the pretty print directory structure.
    • git diff with normal and staged changes
  • flag as boolean prints to stdout, flag with file path prints to file
• check if git is clean: no changes left uncommitted.

Adding Configuration Options is Next

I'm pretty happy with the engine now and should make it easier to layer configuration options on it.

Biggest change I made is buildGraph now takes an array of files instead of a folder. Will make it easier to ignore files and process files by root.

Add CI

Hey 👋,

Do you have any thoughts on a CI system for this repo? I think it would be great to have some automated tests for PRs etc.

I would recommend Travis and I would be happy to set it up if needed.

React Native Project Import Image @1x, @2x, @3x

$ destiny ./src

INFO: Generating tree for: ./src
ERROR: Cannot find import ./Img/sam.png

Actually, the files are [email protected], [email protected], [email protected] but we just have to use it as
require('./Img/sam.png')

For Example

.
├── button.js
└── img
    ├── check.png
    ├── [email protected]
    └── [email protected]

<Image source={require('./img/check.png')} />

For detail check Images - React Native

Error when importing files with same name from different folders

For

| index.js
| - dir1/file.js
| - dir2/file.js

I get a UnhandledPromiseRejectionWarning: Error: fatal: destination exists because the fractal tree resolves this to

{
    "index.js": "index.js",
    "dir1/file.js": "index/file.js",
    "dir2/file.js": "index/file.js",
}

Two options to approach this:

1. Keep sub-directories ("dir1/file.js": "index/dir1/file.js",)
2. Rename the file (e.g. ("dir1/file.js": "index/dir1-file.js",)

I think both approaches might get quite messy when dealing with deeper nested structures tho 🤔
But I'd favor option 2 since it makes it easier to manually rename to something more useful.

Defining fixed point/public files

Sometimes a single repository can have multiple entry points. e.g. my-library/server and my-library/client. Ideally there should be a way to mark a file as being a "fixed point" that other files can be positioned around, but that shouldn't move themselves. Otherwise the public API of a package can be modified by destiny.

Files are not moved in a Typescript codebase

I don't know if the fact my codebase is in Typescript is actually related but I'm just guessing here. I don't have a pure JS codebase to test with.

When I run the tool, it properly prints out the trees of the reorganized files, but when I look at the actual files nothing was moved. The imports were however updated but since no files were moved they are all incorrect.

The tool ends up with this:

INFO: Fixing imports.
ERROR: Cannot find import ./AudioPlayer/IconButton
If you think this is a bug, you can report it: https://github.com/benawad/destiny/issues

But the import is only wrong because the files were not moved, it is correct before running destiny.

Can't resolve imports where file name contains multiple dots but no .js in the end

Example output:

Could not resolve import:  [ '/cant/share/folder/structure/useForm.js',
  '../../services/logger.service' ]

Error: EISDIR: illegal operation on a directory

➜  destiny-upstream git:(master) node dist/src/index.js "tests/fixtures/simple"
Files to structure:
[ 'tests/fixtures/simple' ]
(node:5810) UnhandledPromiseRejectionWarning: Error: EISDIR: illegal operation on a directory, read
    at Object.readSync (fs.js:494:3)
    at tryReadSync (fs.js:333:20)
    at Object.readFileSync (fs.js:370:19)
    at Object.findEdges (/home/anatole/dev/destiny-upstream/dist/src/index/formatFileStructure/shared/findEdges.js:9:31)
    at Object.buildGraph (/home/anatole/dev/destiny-upstream/dist/src/index/formatFileStructure/buildGraph.js:31:21)
    at /home/anatole/dev/destiny-upstream/dist/src/index/formatFileStructure.js:20:74
    at Generator.next (<anonymous>)
    at /home/anatole/dev/destiny-upstream/dist/src/index/formatFileStructure.js:8:71
    at new Promise (<anonymous>)
    at __awaiter (/home/anatole/dev/destiny-upstream/dist/src/index/formatFileStructure.js:4:12)
(node:5810) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:5810) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

A flag to only remove non imported files and empty folders

This flag would tell Destiny to run but only to remove non imported files and empty folders. So Destiny wouldn't do anything else like for example, moving files and fixing import paths.

What about something like --remove-non-imported-only? It isn't that short, so if anyone has a better idea...

Path are messed up on Windows

Here is a example screenshot:

Repo: vscode-hemtt

Ryadih

Name feedback and a few suggestions

I don't think the name is that bad, but I would recommend dropping the -cli suffix.

I've had something like this in mind ever since I ran across arkit

I think this is a great idea and could greatly simplify things for teams in the same ways prettier has.

I'm imagining adding this into my workflow as a pre-commit stage using husky + lint-staged.

Would be nice to have a flag that causes a failure on unused files, so cruft doesn't accumulate.

I think people will have strong preferences which could be solved with a couple config options, but I imagine having a consistent and enforced structure is almost universally preferable.

👍

Support module aliases

In our projects we make heavy use of flow's module.name_mapper and also babel's module alias configuration. This lets us make imports like this import SideNav from 'modules/SideNav';.

Then in .flowconfig we have module.name_mapper='^modules/\(.*\)$' -> '<PROJECT_ROOT>/src/modules/\1'

But this doesn't work in destiny as they aren't relative paths and the graph can't be built. I don't know if this issue becomes irrelevant when using destiny or not, but it does make it harder to start using it.

Destiny tries to create folder named index.jsx

With my React project destiny tries to make a folder named index.jsx instead of just index.

This is my file structure right now:

│   data.json
│   index.html
│   index.jsx
│
├───components
│   │   App.jsx
│   │   Contribute.jsx
│   │   CreateProject.jsx
│   │   Donator.jsx
│   │   Donators.jsx
│   │   donatorShape.js
│   │   Expandable.jsx
│   │   FinishedProjects.jsx
│   │   Footer.jsx
│   │   ImageCarousel.jsx
│   │   ImageSection.jsx
│   │   ShareButton.jsx
│   │   ShowMoreButton.jsx
│   │
│   └───ProjectCard
│           Progress.jsx
│           ProjectCard.jsx
│
└───util
        formatMoney.js

And this is what destiny suggests:

├──index.jsx
├──index.jsx\shared\App.jsx
├──index\App\CreateProject.jsx
├──index\App\data.json
├──index\App\FinishedProjects.jsx
├──index\App\Footer.jsx
├──index\App\ProjectCard.jsx
├──index\App\ProjectCard\Contribute.jsx
├──index\App\ProjectCard\Donators.jsx
├──index\App\ProjectCard\Donators\Donator.jsx
├──index\App\ProjectCard\ImageSection.jsx
├──index\App\ProjectCard\ImageSection\ImageCarousel.jsx
├──index\App\ProjectCard\Progress.jsx
├──index\App\ProjectCard\ShareButton.jsx
├──index\App\ProjectCard\ShowMoreButton.jsx
├──shared\donatorShape.js
├──shared\Expandable.jsx
└──shared\formatMoney.js

Why double index?

In this image on the docs I see both index.js and index/ folder.

Why use this approach? Isn't it bad for the compiler/ambiguous to have both?

Why don't simply put index/ folder's contents outside of it? It would look clear and it wouldn't break encapsulation.

I'm not bashing it, to be clear, I'm just curious about why this solution?

Preview tree

I think the default behavior of destiny should be to output a preview of the fractal tree. This goes in line with how prettier handles things as it only writes when given the --write option.

What do you guys think @benawad @AnatoleLucet @sokki? I'm currently writing a treePrinter module but the core application also needs to be restructured a bit to handle this nicely.

Handle `.spec.js` and `__mocks__` different

This tool is a great idea Ben!
It would be awesome if this would end the endless debates about folder structures like prettier ended the debates about formatting.

I ran it against a small project and quickly realised, that it doesn't play well when u have .spec. files and __mock__ folders inside your src folder.

So this

becomes:

instead of:

Object.values(...).flat is not a function

I am trying to run this on a project with: npx butler-cli src and I'm getting: Object.values(...).flat is not a function returned. Let me share with you my envirement.

/Users/jchambers/.nvm/versions/node/v10.15.0/bin/node

NPM version: 6.4.1

I thought I would submit a bug report.

Architecture, Config, CLI, API — Recommendations & Discussion

I've been thinking about and working on this project for weeks and am growing a passion towards it.
Thank you for all being so accepting, patient and prompt in your responses.

I'd like to voice some ideas and possible solutions. There's a lot here, so take your time to read and respond if you choose to do so.

Quality of life

Refactor

A great idea, but some of the code is messy, hard to interpret and difficult to test. I've posted a few pulls to refactor code. I will keep doing this, taking anything from this discussion into account.

Config File

I personally would prefer a configuration that supports destiny.config.(ts|js), with the default export as what will be read.

If typescript, the user should have ts-node installed . This way we can assign an interface to the object. I'm pretty sure webpack and/or rollup have this feature built in.

Architecture

I think we can start researching how we may implement an architecture that can support other file extensions via middleware. Prettier has something similar for each language they support, which include parsers and formatting rules.

The way this program could be laid out:

1. Create graph representing current structure
   a. Find a file
   b. Add import path to graph
2. Create graph representing new structure
   a. ...details
3. Move files and folder structure

There are some details that I've overlooked, but it's general.

Graphs

Currently we're managing our own graphs. It is simple with object-index pattern:
{ [index: string]: string }.

It works with strings as the nodes, but what if we ever needed to use an instance on each end? We'd have to rewrite it.

I believe we should go down the OOP route for our graph generation and management. It would make extensibility and management easier. I love functional programming, but well managed mutability is where the power is in graphs!

We could use a library for manipulating graphs called cytoscape.js, which comes with some fantastic abstractions.

On first glance it looks like it's built to support the DOM. However, it can run in headless mode, meaning all the rendering capabilities are disabled and it can be used to strictly manipulate data. Score!

CLI API

I wrote a cli in yargs and came up with this:

Usage: index.ts [options] [files / regex]

Dry run and preview directory structure:
  index.ts src/**/*.*

Write the changes:
  index.ts -w src/**/*.*


Options:
  --help             Show help                                         [boolean]
  --version          Show version number                               [boolean]
  --config, -c       path to your config file.
                     Can be in any language supported by destiny.
  --dir, -d          Sets the root directory
  --init             Creates a `destiny.config.*`, defaulting to `.ts`
                     file extension                        [choices: "ts", "js"]
  --interactive, -I  Prompts user to process directory structure
                     when a dry run is complete
  --ignore, -i       Locations to be ignored whilst processing           [array]
  --subfolder, -s    Operates on each folder in the directory,
                     instead of on the directory itself                  [count]
  --test, -t         A regex to match test files against                [string]
  --write, -w        Writes the changes to the directory               [boolean]

Supported formats:
  .js, .jsx, .ts, .tsx

Generated by this:

import * as yargs from "yargs";

const argv = yargs
  .usage(
    `
Usage: $0 [options] [files / regex]

Dry run and preview directory structure:
  $0 src/**/*.*

Write the changes:
  $0 -w src/**/*.*
  `
  )
  .epilogue("Supported formats:")
  .epilogue("  .js, .jsx, .ts, .tsx")
  .demandCommand(1)
  .option("config", {
    alias: "c",
    description: `path to your config file.
    Can be in any language supported by destiny.`
  })
  .option("dir", {
    alias: "d",
    description: `Sets the root directory`
  })
  .option("init", {
    description: `Creates a \`destiny.config.*\`, defaulting to \`.ts\`
    file extension`,
    choices: ["ts", "js"]
  })
  .option("interactive", {
    alias: "I",
    description: `Prompts user to process directory structure
    when a dry run is complete`
  })
  .option("ignore", {
    alias: "i",
    array: true,
    description: `Locations to be ignored whilst processing`
  })
  .option("subfolder", {
    alias: "s",
    count: true,
    description: `Operates on each folder in the directory,
    instead of on the directory itself`
  })
  .option("test", {
    alias: "t",
    string: true,
    boolean: true,
    description: `A regex to match test files against`
  })
  .option("write", {
    alias: "w",
    boolean: true,
    description: `Writes the changes to the directory`
  }).argv;

Use mvjs as a dep

Hi @benawad and other maintainers, this is not really an issue, but I'd like to understand how you'd feel like about using this lib that I created for all the file moving operations https://github.com/fabiomcosta/mvjs/
I can definitely help with any special use cases you might have and expose the appropriate APIs, if there is not already one in place.
Let me know if there is any interest and I try to create a PR with a POC using it so we can actually see if there is some benefit to it.
If there is no interest plz let me know before-hand so we can avoid the extra work, but I believe this can benefit both projects as you can focus on other more important details of this project.