xml-comp/xml-comp

Compare ANY markup documents.

Menu

1. What is XML-Comp?
2. Features
3. Installing
4. Running
5. How this works?
6. Comparing any kind of document
7. Contributing
8. To Do
9. Using only the comparer package

What is XML-Comp

This is a command line tool and a package that together they provide the capability of comparing two directories and appending to files the differences between the directories, also creates possible files or folders that are missing. It was made to help RimWorld's community translators(1) to know what was modified on the last XML updates and to let them keep in track of what they need to add/remove from what has been done.

(1) and maybe other indie games that uses XML

Features

• Compare two given directories
• Append missing tags to the respective file
• Creates missing files on the compared directory
• Creates missing folders on the compared directory
• Compare one directory with multiple directories
• Compare automatically the game's english version with the given translation
• Translate tag content
• Expose untracked files
• Expose untracked directories
• Expose untracked tags

Installing

$ go get github.com/XML-Comp/XML-Comp

Running

$ XML-Comp -translation /path/to/language/translation

How this works?

You need the path that is called "translation", which are described bellow:

• "translation": Full path directory of your RimWorld Language folder cloned from GitHub.

My "translation" path: /Users/arthur/Github/RimWorld-PortugueseBrazilian

With this path in hand, running xml-comp -translation your/path/to/translation will let you know in every file of your project what is missing by adding lines to it with what is needed to translate! That simple!

[RIMWORLD not installed in standard path]

If by any reason you did not install the game on Steam's standard path or want to use a different one, It's possible to use the original flag that exposes your customized path to the game as shows:

• "original": Full path directory of your RimWorld English folder (optional). My "original" path (optional): /Users/arthur/Library/Application Support/Steam/steamapps/common/RimWorld/RimWorldMac.app/Mods/Core/Languages/English

In order to use this flag you'll need to parse it with the translation flag as in: xml-comp -original path/... -translation path/...

Comparing any kind of document

To compare any kind of files, all you need is to use the flag -doc <type name>, eg -doc html. This will use the paths that you gave only to compare the specified type of document. Another example:

$ XML-Comp -doc html -original path/to/It -translation path/to/It

OBS: This is not required, by default It's comparing all .xml files that are encountered.

Contributing

Join our Gitter

To Do - Check our Issues & Milestones

Using only the comparer package

// Import the package
import "github.com/XML-Comp/XML-Comp/comparer"
// Set document type variable to the desired document
// without the "." | eg: "xml" or "html"
comparer.DocType = "html"
// Start the main function with the full paths to compare
// the firstPath is always what will be used as model
func main() {
    err := comparer.Compare(firstPath, comparingPath)
    if err != nil {
        log.Fatal(err)
    }
}

xml-comp's Issues

How to use with binary

diff with maps

Maps in go are much more efficient than slices, read this article for more detailed info. Right now we're comparing the tags using slices, that makes our program much slower, than If we had It with maps.

Using maps, the diff function would be something like:

func diff(origin, translation map[string]string) []string {
	missing := []string{}
	for k := range origin {
		if _, ok := translation[k]; !ok {
			missing = append(missing, k)
		}
	}
	return missing
}

This requires changes on how the file is stored while reading It.

Create Same folder 'missingFiles.txt'

Files & Folders Navigator

To keep up with the project we need a func that READs and stores all Dir & Files. It might be a Map of Maps:

// Output:
map[
   "Dir": [
      "dir1": [
         "dir1Dirs": [
            "dir1DirsD": [],
            "dir1DirsFiles": [],
         ], 
         "dir1Files": [],
      ], 
      "dir2": [...], 
      "dir3": [...], 
      "dir4": [...], 
      ],
   Files: [
      "a",
      "b",
      "c",
   ],
]

• What we need is nested slices (or map) that gets It's folders and append them into the original folders;
  Example (pythonic):

slice_of_folders = [
[folderA, [folderAa, [more folders], folderAb, [...]]],
[folderB, [...]],
.
.
.
]

Helpful links:
http://stackoverflow.com/questions/20115327/golang-rename-the-directory-and-partial-file-renaming
https://golang.org/pkg/path/filepath/#Walk

expose untracked tags

Exposing untracked tags from a file might help to remove possible unwanted lines on .xml files. This was introduced in 9fc5002

bug: reading not .xml files

Compare stored Files on 'same' folder

missingFiles: Create newFile with all new tags

Might be a good functionality to instead of generating multiple missingTags files, creating one main missingTags file with the names of missing tags, like bellow:

<--MissingTags in names.xml-->
<personal.Name>
<Surname>

<--MissingTags in background.xml-->
<floor></floor>
<water>(maybe with english content?)</water>

(...)

test/support windows

it would be nice to have more testers on windows to debbug why the tool isnt working properly on it

bug: creating empty missing Files

After analyzing some files It's still creating empty files, even when they are equal

expose untracked files

while comparing we want the projects to be as similar as possible, that's why we create the files that are missing. To accomplish more of that we need to also point out If any file is not on the originaldirectory, meaning that It can be removed.

There's two approaches to this, we can set a flag to enforce this delete on runtime or without the flag write in some kind of file the path of the untracked files.

Read 'equal' files on same folder

Question

Is it just more or if an object is a directory it gets added to both dir variable and file variable? Why?

https://github.com/ArxdSilva/XML-Comp/blob/master/comparer/compareFolders.go#L49-#L52

More tests

Improve coverage

Easy Installer

This installer needs to provide people without programming skills a way of installing It without having to install Go first.

We might need some kind of requirements.txt file or vendor folder to help It or some kind of MAKEFILE

Compare fileB to fileA to get old tags

CLI flags

It would be nice to have some flags like:

• '-v' for version
• '-fd' for only folder verification
• '-fl' for file verifications

stop panicing errors

Panicing errors is a bad idea, we should only return It

Support Google Translation API

It would be awesome that while content is appended into the missingTags.txt we offer a translation from Google Translate using their API in the next line.

Exemple (fileMissingTags.txt):

Original:
<missingTag>English version of a missing tag</missingTag>
Google Translation to ~Portuguese:
<missingTag>Versão em inglês de uma tag em falta</missingTag>

Add Travis config (CI layer)

contributing guide

it would be nice to have a contributing.md file showing a basic setup of the project and how to work on it to contribute. Important things like 'coding style' (no empty lines inside functions/methods! :D), formatting and testing should be displayed.

Turn this project CLI viable

To make this viable we need the implementation of:

$ xml-comp PathA PathB

comparing func should use only the first word as 'tag' to compare

In xml docs It's possible to have more than one marker on each line, and only the first marker is needed to compare. e.g.:

<tag marker=""><...>

It should remove '<' and '>' and strip the markers to get only the first one

Include in 'missingTags.txt' in folder

not matching tags must start with <--Filename Not Matching Tags-->

xml file cant be opened

Just got a email about this behavior on windows.

Folders & file navigation

This is related to #5, #6, #7 and #8;
Tool needs the skill to navigate throught your file & solve the related issues.

compare lines instead of just tags

It would be useful to enable line comparison, so in files like .txt or any other non markup extension, It'll be possible to work as well.

make -original optional

It would be nice to have 'default' paths for each OS, this way we could make -original optional and the user would only use If he/she installed the game in a custom location.

reader: make It recursive

Docs: add How to use comparer

It'll be nice to have a how to on readme.md on how to use the comparer package, so new users can plug It into their apps.

Store files of each path on separated variables

Create 'missingTags.txt' starting with <--Filename Missing Tags-->

How to use on windows

Ending of tags / comments

While parsing the file's tags the new branch recursive is getting in trouble in two situations:

1. When a tag is a comment as <!--...-->
2. When a tag starts on the beginning of a file and ends at the end.

Right now It's only checking the next tag, not where is the closing one, I'm not sure If 2 is really a bug, since It only needs to verify If It exists, but comments should be recognized by the app and not referred as a missing tag.

Instead of writing into the file as a missing Tag, It should only be present there as a missing comment, without the Add your translation here commentary.

Rename Import after move

https://github.com/XML-Comp/XML-Comp/blob/master/main.go#L8

Should probably rename the import after the move

GUI: use binary with JS

It's needed to figure out how to use a binary from this project with JS so It can be executed on the GUI.

1:N comparison

It would be nice to enable a flag to make possible comparing a given original language with many others. It'd be something like:

 if (flag){
    folders := lookup(GivenRootFolderOfFolders)
    for _, f := range folders {
        compare(english,join(GivenRootFolderOfFolders,'/',f))
    }
}

This will enable contribution to N repositories with a single command.

Enable tag indenting through maps[tag]{tags}

Maybe by checking tag inline(/next lines) closing might help identify the last identation from tags.

Maybe by checking the next line + if previous are empty

GUI

EDIT: Initial GUI idea;

OBS: I'm getting some help from a designer friend and I must update this design shortly
XML, not XMP

call compare again when file is created

now when the translation is got a file missing, It'll create the new file and move on, It'd be better to compare the files again after this creation, in order to generate new tags.

Now to make this comparison happen, the user has to run the binary again.