primait/posexional

A library to manage positional files in Elixir

Posexional

Posexional is a library to manage positional files in Elixir.

Installation

Add posexional to your list of dependencies in mix.exs:

def deps do
  [{:posexional, "~> 0.4"}]
end

Usage

Positional files are the most bad and terrifying file format you ever want to work with, believe me, it's utter crap. Still some web services use this format to expose data services, and if you are here reading this, probably you are in luck like us!

A positional file has a format specification like this:

field name	offset	length	notes
code	0	5	request code fill with 0 and align on the right
prog	0	5	progressive number (fill with spaces)
type	21	2	AA
number	24	2	01
name	27	10	person name (fill with -)
future use	38	2	leave empty
end	41	1	!

The expected results should be something like this:

000B1    1AA01george----  !
000B2    2AA01john------  !
000B3    3AA01ringo-----  !
000B4    4AA01paul------  !

Cool uh?

With Posexional you can produce this file by defining a module that use Posexional.

defmodule BeatlesFile do
  use PosexionalFile

  @separator "\\n"

  row :beatles do
    value :code, 5, filler: ?0, alignment: :right
    progressive_number :code, 5
    fixed_value "AA"
    fixed_value "01"
    value :name, 10, filler: ?-
    empty 2
    fixed_value "!"
  end
end

And then use it in your code.

BeatlesFile.write([
  beatles: [code: "B1", name: "george"],
  beatles: [code: "B2", name: "john"],
  beatles: [code: "B2", name: "ringo"],
  beatles: [code: "B2", name: "paul"]
])

In the first part we define the structure inside a module. We are not saying what the content or the number of rows there will be, we are just saying that there is a row called :beatles with the structure declared by the fields.

Then we can call BeatlesFile.write/1, we pass the data that should be written in the fields. Just the relevant data, The empty fields, the fixed values or the progressive number is managed by the library itself.

The write/1 function accept a keyword list with the row name as key, and a keyword list of {field name, field value} of data. If some data is bigger than the field size an error is thrown

With the same exact module, we can even read a positional file by calling read/1 and passing a binary string of the file content.

There is only one thing to notice, when we write a file we can be declarative and say what row we want to write, as well as the data we want in it. On the other hand, while reading a positional file, we don't know which rows we are reading, so we need to tell in some way to every row how it is recognized, so that the parser is able to do its job.

Since we only have a row type (the beatles one) we can just say to the module to always match the beatles row, as simple as:

defmodule BeatlesFile do
  use PosexionalFile

  @separator "\\n"

  row :beatles, :always do # add :always here to always match this row while reading
    value :code, 5, filler: ?0, alignment: :right
    progressive_number :code, 5
    fixed_value "AA"
    fixed_value "01"
    value :name, 10, filler: ?-
    empty 2
    fixed_value "!"
  end
end

:always is a special type that always match, you could also pass :never (not so useful!) and, to gain total control over the choice, a function with a single argument(the full row content) to match some data inside of it.

Now we are able to parse a positional file.

"000B1    1AA01george----  !\n000B2    2AA01john------  !\n000B2    3AA01ringo-----  !\n000B2    4AA01paul------  !"
|> BeatlesFile.read
|> IO.puts

This is the output.

[beatles: [code: "B1", code: 1, fixed_value: "AA", fixed_value: "01",
  name: "george", fixed_value: "!"],
 beatles: [code: "B2", code: 2, fixed_value: "AA", fixed_value: "01",
  name: "john", fixed_value: "!"],
 beatles: [code: "B2", code: 3, fixed_value: "AA", fixed_value: "01",
  name: "ringo", fixed_value: "!"],
 beatles: [code: "B2", code: 4, fixed_value: "AA", fixed_value: "01",
  name: "paul", fixed_value: "!"]]

Positional files cool again!!! Well no...they still sucks...but a little less.

Contributing

Thank your for considering helping with this project. Please see CONTRIBUTING.md file for contributing to this project.

License

MIT License. Copyright (c) 2015-2020 Prima.it

posexional's Issues

Add option to trim strings

Introduce option to get structs instead of keyword lists when reading

It would be very useful to have the possibility to decide whether posexional should return a keyword list representing the parsed row or a struct representing the parsed row.

Error with progressive numbers that use non 0 fillers

The Problem

If you follow the Beatles example, you have positional file with a field that is a progressive number with a white space filler.
If you try to read from it, you get an :erlang.tuple_to_list(:error).

Why it happens

The only treatment a progressive number gets in terms of filler is checking if he is made only of filler in the Posexional.Field.depositionalize/2,

  def depositionalize(content, %{filler: filler}) do
    content
    |> nil_if_empty(filler)
  end

and after that, something like " 1" gets passed to Integer.parse here in the Posexional.Field.ProgressiveNumber module.

defimpl Posexional.Protocol.FieldRead, for: Posexional.Field.ProgressiveNumber do
  def read(field, content) do
    content
    |> Posexional.Field.depositionalize(field)
    |> Integer.parse
    |> Tuple.to_list
    |> hd
  end
end

parsing "01" is ok, parsing " 1" not that much.

Solving it

• Remove the filler from the progressive number before parsing.
• Correct the docs, you should be able to read and write the same row, but right now you write with spaces and read with zeroes.

Parsing file with different rows' structure

Hey guys,

I'm trying to use this lib to parse a positional file with different rows' structure.

For instance:

000B1    1AA01george----  !
001B102251943Liverpool
002B1voice
002B1guitar
002B1sitar
000B2    2AA01john------  !
001B210091940Liverpool
002B2voice
002B2electricguitar
002B2piano
002B2harmonica

Can this be defined with a multi row PosexionalFile?

Breaking change

Hi people!

Recent updates broke the default behaviour of the library. This line changed the "trim" behaviour.

That is not an issue to us, but I guess it should be mentioned in a changelog or something.

The difference is that now it trims everything and not only removes white spaces. We had some validations on size and performed trimming on our side so that is ok for us, but it was a bit misleading the change.

Elixir 1.5

warning: variable "deps" does not exist and is being expanded to "deps()", please use parentheses to remove the ambiguity or change the variable name
mix.exs:11

warning: variable "description" does not exist and is being expanded to "description()", please use parentheses to remove the ambiguity or change the variable name
mix.exs:12

warning: variable "package" does not exist and is being expanded to "package()", please use parentheses to remove the ambiguity or change the variable name
mix.exs:13

==> posexional
Compiling 11 files (.ex)
warning: String.ljust/3 is deprecated, use String.pad_trailing/3 with a binary padding
lib/posexional/field.ex:35

warning: String.rjust/3 is deprecated, use String.pad_leading/3 with a binary padding
lib/posexional/field.ex:40

warning: String.to_char_list/1 is deprecated, use String.to_charlist/1
lib/posexional/field.ex:66

Please add a license file

https://stackoverflow.com/questions/31639059/how-to-add-license-to-an-existing-github-project

Wrong typespecs

Hi! Thanks for your nice library.

I have an issue with dialyzer with the read functions. They may return [binary()] other than Keyword.t().

I have to add an exception to dialyzer because of this.