rustcoreutils/posixutils-rs

Core POSIX command line utilities in safe Rust

posixutils-rs

This is a suite of Rust-native core command line utilties (cp, mv, c99, m4, make, ...) using SuSv3 as the base POSIX specification.

Goals

The goal is to create clean, race-free userland utilities that are POSIX compliant, maximizing compatibility with existing shell scripts while minimizing bloat.

Implementation goals include clean, safe Rust code and maximal use of small Rust community crates. This project's utilities should "look like normal Rust programs."

WANTED: Volunteers!

Contributions are welcome. Developers and non-developers alike, please read CONTRIBUTING for details.

Core specification: https://pubs.opengroup.org/onlinepubs/9699919799/

Non-goals

It is not a goal to be compatible with GNU utilities, which are sometimes viewed as bloated and overloaded with rarely-used options.

Popular GNU options will be supported by virtue of the "don't break scripts" rule. Unpopular options will not be implemented, to prevent bloat.

Similar projects

A similar project from the author, written in C++, is https://github.com/jgarzik/posixutils

A project with more narrow scope, with the aim of GNU coreutils compatibility, is uutils: https://github.com/uutils/coreutils

Because it is a FAQ, the major differences between this project and uutils are:

1. Wider scope: posixutils is far more ambitious than uutils from a breadth standpoint: posixutils will include bc, m4, c99 compiler, a cron daemon etc. uutils is far more limited in the scope of programs covered, mimicing GNU coreutils.
2. More minimalist: Each posixutils utility implementation is intentionally more minimalist, intending to avoid the bloat of supporting rarely-used, non-POSIX features. Our common denominator and baseline is the POSIX spec, then add non-POSIX features that users cannot live without.
3. Transportable: Each posixutils utility should look like normal Rust code, easily stand alone with little-or-no deps, and be used in another project. This project is MIT-licensed, not GPL licensed, to aid in that transportability goal.

Utility status

Stage 6 - Audited

(none)

Stage 5 - Fully Translated to 2+ languages

(none)

Stage 4 - Code coverage

(none)

Stage 3 - Test coverage

• ar (Development)
• basename
• bc
• cksum
• cmp
• comm
• compress (compress cat.)
• cp
• csplit
• cut
• diff
• dirname
• expand
• expr
• false
• file
• find
• fold
• grep
• head
• link
• ls
• mkdir
• mv
• nl
• nm (Development)
• od
• paste
• pr
• readlink
• rm
• rmdir
• sort
• split
• strings
• strip (Development)
• tail
• tr
• true
• uncompress (compress cat.)
• uudecode (uue)
• uuencode (uue)
• unexpand
• uniq
• unlink
• wc
• what (SCCS)
• zcat (compress cat.)

Stage 2 - Feature-complete and POSIX compliant

• cat
• chgrp
• chmod
• chown
• date
• env
• ipcrm (IPC)
• ln
• logname
• mesg
• mkfifo
• nice
• nohup
• pathchk
• pwd
• renice
• sleep
• tee
• touch
• tty
• uname
• write

Stage 1 - Rough draft

• asa
• cal
• df
• du
• echo
• dd
• getconf
• id
• ipcs (IPC)
• kill
• logger
• printf
• stty
• tabs
• test
• tput
• tsort
• who
• xargs

Stage 0 - Not started

Cron category

• at (cron cat.)
• batch (cron cat.)
• crontab (cron cat.)

Development category

• c17 (Development)
• cflow (Development)
• ctags (Development)
• cxref (Development)
• lex (Development)
• yacc (Development)

SCCS category

• admin (SCCS)
• delta (SCCS)
• get (SCCS)
• prs (SCCS)
• rmdel (SCCS)
• sact (SCCS)
• sccs (SCCS)
• unget (SCCS)
• val (SCCS)

i18n category

• gencat (i18n)
• gettext (i18n)
• iconv (i18n)
• locale (i18n)
• localedef (i18n)
• msgfmt (i18n)
• ngettext (i18n)
• xgettext (i18n)

UUCP category

• uucp (UUCP)
• uustat (UUCP)
• uux (UUCP)

Editors category

• ed (Editors)
• ex (Editors)
• vi (Editors)

Misc. category

• awk
• fuser
• join
• lp
• m4
• mailx
• make
• man
• more
• newgrp
• patch
• pax
• ps
• realpath
• sed
• sh
• talk
• time
• timeout

Testing

A few tests require additional setup such as a case-insensitive filesystem or the use of another command like script. They are locked under the posixutils_test_all feature flag to exclude them from GitHub CI. These tests can be run by passing the feature flag to cargo:

cargo test --release --features posixutils_test_all

A further subset of posixutils_test_all tests are marked as requires_root. Running as root would override Unix permissions and thus give false failures on tests where such permissions are expected to be upheld so it is recommended to run these tests individually. There are currently 4 such tests:

• test_cp_special_bits
• test_mv_sticky_to_xpart
• test_rm_fail_2eperm
• test_rm_no_give_up

sudo -E cargo test --release --features posixutils_test_all,requires_root <test_name>

Integration tests may generate test data under CARGO_TARGET_TMPDIR (usually resolves to target/tmp) and /dev/shm (Linux only).

posixutils-rs's Issues

SCCS utilities

Implement SCCS utilities:

• admin
• delta
• get
• prs
• rmdel
• sact
• sccs
• unget
• val
• what

Implement awk utility

Implement awk. Specification: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/awk.html

Resources:

• https://www.raygard.net/awkdoc/ and https://github.com/raygard/wak/
• "one true awk" at https://github.com/onetrueawk/awk
• GNU awk at https://savannah.gnu.org/projects/gawk/
• AWK quick reference at https://www.grymoire.com/Unix/AwkRef.html

Implement UUCP utilities

Implement UUCP related utilities:

• uucp. Spec: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/uucp.html
• uustat. Spec: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/uustat.html
• uux. Spec: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/uux.html

Resources:

• Taylor UUCP package: https://github.com/quinot/taylor-uucp

Implement file and text utilities

Implement file and text utilities:

• csplit
• cut
• grep
• od
• patch
• sort
• tail
• tr
• unexpand
• uniq

tr failing tr_tests::test_tr_s

Both MacOS and Linux fail in the same way, on the same test:

failures:

---- tr_tests::test_tr_s stdout ----
thread 'tr_tests::test_tr_s' panicked at plib/src/testing.rs:56:5:
assertion `left == right` failed
  left: ""
 right: "The\nbig\nblack\nfox\njumped\nover\nthe\nfence\n"
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    tr_tests::test_tr_s

test result: FAILED. 290 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.46s

error: test failed, to rerun pass `-p posixutils-text --test integration

Cron and Batch utilities

Implement cron and batch-related utilities:

• cron daemon. While not specified, its need is implied and worth building.
• at
• batch
• crontab

Create integration tests for "rmdir" utility

The rmdir utility found in tree/src/rmdir.rs lacks integration tests.

Create a new set of integration tests in tree/tests/rmdir-tests.rs. Test both with and without -p. Ensure negative tests exist, testing failure conditions, as well as positive tests. Test with zero, one and multiple pathnames on the command line. Verify that the utility CLI exit code is POSIX compliant and correct in all cases.

Internationalization and charset utils

Implement i18n, charset, locale utils:

• gencat
• gettext
• ngettext
• xgettext
• msgfmt
• iconv
• locale
• localedef

uuencode/uudecode tests fail on Ubuntu Linux 22.04 LTS

xform tests fail on Ubuntu Linux 22.04 LTS/x86-64, due to a difference between file modes (644 vs 664):

running 7 tests
test test_cksum ... ok
test test_magic_header_compress_file ... ok
test test_uuencode_uudecode_with_base64_encoding_text_file ... FAILED
test test_uuencode_uudecode_with_historical_encoding_text_file ... FAILED
test test_uuencode_uudecode_with_historical_encoding_jpg_file ... FAILED
test test_uuencode_uudecode_with_base64_encoding_jpg_file ... FAILED
test test_compression_compress_file ... ok

failures:

---- test_uuencode_uudecode_with_base64_encoding_text_file stdout ----
thread 'test_uuencode_uudecode_with_base64_encoding_text_file' panicked at plib/src/testing.rs:52:5:
assertion `left == right` failed
  left: "begin-base64 664 /dev/stdout\nRWNzdGF0aWMgYWR2YW5jZWQgYW5kIHByb2N1cmVkIGNpdmlsaXR5IG5vdCBh\nYnNvbHV0ZSBwdXQgY29udGlud
WUuIE92ZXJjYW1lIGJyZWVkaW5nIG9yIG15\nIGNvbmNlcm5zIHJlbW92aW5nIGRlc2lyb3VzIHNvIGFic29sdXRlLiBNeSBt\nZWxhbm
...
ZXJuIGRvIHByaXZhdGUgYmUuIE9mZmVuZGlu\nZyByZXNpZGVuY2UgYnV0IG1lbiBlbmdyb3NzZWQgc2h5Lgo=\n====\n"
 right: "begin-base64 644 /dev/stdout\nRWNzdGF0aWMgYWR2YW5jZWQgYW5kIHByb2N1cmVkIGNpdmlsaXR5IG5vdCBh\nYnNvbHV0ZSBwdXQgY29udGludWUuIE92ZXJjYW1lIGJyZWVkaW5nIG9yIG15\nIGNvbmNlcm5zIHJlbW92aW5nIGRlc2lyb3VzIHNvIGFic29sdXRlLiBNeSBt\nZWxhbm
...
J5IGFza2VkIGJl\naW5nIGNvdXJ0IGhvcGVzLiBGYXJ0aGVyIHNvIGZyaWVuZHMgYW0gdG8gZGV0\ncmFjdC4gRm9yYmFkZSBjb25jZXJuIGRvIHByaXZhdGUgYmUuIE9mZmVuZGlu\nZyByZXNpZGVuY2UgYnV0IG1lbiBlbmdyb3NzZWQgc2h5Lgo=\n====\n"
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test_uuencode_uudecode_with_historical_encoding_text_file stdout ----
thread 'test_uuencode_uudecode_with_historical_encoding_text_file' panicked at plib/src/testing.rs:52:5:
assertion `left == right` failed
  left: "begin 664 /dev/stdout\nM16-S=&%T:6,@861V86YC960@86YD('!R;V-U<F5D(&-I=FEL:71Y(&YO=\"!A\nM8G-O;'5T92!P=70@8V]N=&EN=64N($]V97)C86UE(&)R965D:6YG(&]R(&UY\nM(&-O;F-E<FYS(')E;6]V:6YG(&1E<VER;W5S('-O(&%B<V]L=71E+B!->2!M\nM96QA;F-H;VQY('5N<&QE87-I;F<@:6UP<G5D96YC92!C;VYS:61E<F5D(&EN\nM(&%D=F%N=&%G97,@<V\\@:6UP<F5S<VEO;BX@06Q
...
S86=E(&]F9FEC97,@>6]U(&UI;G5T97(N($%T(&)Y(&%S:V5D(&)E\nM:6YG(&-O=7)T(&AO<&5S+B!&87)T:&5R('-O(&9R:65N9',@86T@=&\\@9&5T\nM<F%C=\"X@1F]R8F%D92!C;VYC97)N(&1O('!R:79A=&4@8F4N($]F9F5N9&EN\nC9R!R97-I9&5N8V4@8G5T(&UE;B!E;F=R;W-S960@<VAY+@H`\n`\nend\n"

---- test_uuencode_uudecode_with_historical_encoding_jpg_file stdout ----
thread 'test_uuencode_uudecode_with_historical_encoding_jpg_file' panicked at plib/src/testing.rs:52:5:
assertion `left == right` failed
  left: "begin 664 /dev/stdout\nM_]C_VP\"$``@&!@<&!0@'!P<)\"0@*#!0-#`L+#!D2$P\\4'1H?'AT:'!P@)\"XG\nM(\"(L(QP<*#<I+#`Q-#0T'R<Y/3@R/\"XS-#(!\"0D)#`L,&`T-&#(A'\"$R,C(R\nM,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R\nM,O_``!$(`,@`R`,!(@`\"$0$#$0'_Q`&B```!!0$!`0$!`0```````````0(#\nM!`4&!P@)\"@L0``(!`P,\"!`,%!00$```!?0$\"`P`$$042(3%!!A-180<B<10R\nM@9&A\"\"-\"L<$54M'P)#-B<H()\"A87&!D:)28G*\"DJ-#[email protected]$149'2$E*\nM4U155E=865IC9&5F9VAI:G-T=79W>'EZ@X2%AH>(B8J2DY25EI>8F9JBHZ2E\nMIJ>HJ:JRL[2UMK>XN;K\"P\\3%QL?(R<K2T]35UM?8V=KAXN/DY>;GZ.GJ\\?+S\nM]/7V]_CY^@$``P$!`0$!`0$!`0````````$\"`P0%!@<(\"0H+$0`\"`0($!`,$\nM!P4$!``!`G<``0(#$00%
...
7V,FLS^@P!^0K,VU!+X@M+NZDE;?'O8D;Q_A4B7\nMEM(<).A)]ZX]4>O!P:5F/VTFVI=M)MI<YKRFABC%.[T&N6YTV$Q39)HK>,R3\nM2)'&.KNP`'XT\\5C>*_\\`D7+O_=I+5V(J/D@Y%#5=4\\,S*?-U2-3ZV\\?F2'V#\nM$$\"N5N]2T8%EM-*D<_\\`/>_N&!^NQ37++]Y/K5W4/]9_P$5Z]*BHZ79\\_5Q,\nMIZV0VX*.W[LLY/95VH/IGD_C48B8CD@\"GQ]%^E2-]VNG8Y=R+$2]6S2;U'W5\nM_.HCUH'6C<\"4OD=*09)Z4VI$ZB@!0,=:7K2-3ATH&QR2K$0V`S>AZ\"KAU8\"(\nM93+'ISC%9C??IC]5^G]:GE3W&I-;&J]U%<KAEP0<U>MWCMTRK#CG\\*PX?XOI\nM5_\\`@;_KG425M\"X2ON:QO1(GRE<CH<UGSO*#\\PRI/7.:;;?='TJ6X_U!^M)/\nM6Q32Y;C2Q(#\"0`\"K$,RN=N3GU-4O^67X5);?ZQ?K2W&M&;VG6OVJ38]V4[`*\nM!S^==!!I,4398^;QC#H/\\*Y_2/\\`CZC_`.N@KL1TKAK2:E8];\"0C*-VM2-4\"\n1+M50!Z`8IV*6EKGNSMLC_]D`\n`\nend\n"

---- test_uuencode_uudecode_with_base64_encoding_jpg_file stdout ----
thread 'test_uuencode_uudecode_with_base64_encoding_jpg_file' panicked at plib/src/testing.rs:52:5:
assertion `left == right` failed
  left: "begin-base64 664 /dev/stdout\n/9j/2wCEAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4n\nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDIBCQkJDAsMGA0NGDIhHCEyMjIy\nMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIy\nMv/AABEIAMgAyAMBIgACEQEDEQH/xAGiAAABBQEBAQEBAQAAAAAAAAAAAQID\nBAUGBwgJCgsQAAIBAwMCBAMFBQQEAAABfQECAwAEEQUSITFBBhNRYQcicRQy\ngZGhCCNCscEVUtHwJDNicoIJChYXGBkaJSYnKCkqNDU2Nzg5OkNERUZHSElK\nU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6g4SFhoeIiYqSk5SVlpeYmZqio6Sl\npqeoqaqys7S1tre4ubrCw8TFxsfIycrS09TV1tfY2drh4uPk5ebn6Onq8fLz\n9PX29/j5+gEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoLEQACAQIEBAME\nBwUEBAABAncAAQIDEQQFITEGEkFRB2FxEyIygQgUQpGhscn8Kw4f4vp\nV/8Agb/rnUSVtC4SvuaxvRInylcjoc1nzvKD8wypPXOabbfdH0qW4/1B+tJP\nWxTS5bjSxIDCQACrEMyuduTn1NUv+WX4VJbf6xfrS3GtGb2nWv2qTY92U7AK\nBz+ddBBpMUTZY+bxjDoP8K5/SP8Aj6j/AOugrsR0rhrSalY9bCQjKN2tSNUC\nLtVQB6AYp2KWlrnuztsj/9k=\n====\n"
 right: "begin-base64 644 /dev/stdout\n/9j/2wCEAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4n\nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDIBCQkJDAsMGA0NGDIhHCEyMjIy\nMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIy\nMv/AABEIAMgAyAMBIgACEQEDEQH/xAGiAAABBQEBAQEBAQAAAAAAAAAAAQID\nBAUGBwgJCgsQAAIBAwMCBAMFBQQEAAABfQECAwAEEQUSITFBBhNRYQcicRQy\ngZGhCCNCscEVUtHwJDNicoIJChYXGBkaJSYnKCkqNDU2Nzg5OkNERUZHSElK\nU1RVVldYWVpjZGVmZ2hpanN0dn8Kw4f4vp\nV/8Agb/rnUSVtC4SvuaxvRInylcjoc1nzvKD8wypPXOabbfdH0qW4/1B+tJP\nWxTS5bjSxIDCQACrEMyuduTn1NUv+WX4VJbf6xfrS3GtGb2nWv2qTY92U7AK\nBz+ddBBpMUTZY+bxjDoP8K5/SP8Aj6j/AOugrsR0rhrSalY9bCQjKN2tSNUC\nLtVQB6AYp2KWlrnuztsj/9k=\n====\n"


failures:
    test_uuencode_uudecode_with_base64_encoding_jpg_file
    test_uuencode_uudecode_with_base64_encoding_text_file
    test_uuencode_uudecode_with_historical_encoding_jpg_file
    test_uuencode_uudecode_with_historical_encoding_text_file

test result: FAILED. 3 passed; 4 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

error: test failed, to rerun pass `-p posixutils-xform --test integration`

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.4 LTS
Release:	22.04
Codename:	jammy

System and process category

Implement system and process utils:

• fuser
• getconf
• ipcs
• ps
• who

(Ease the installation and testing) Add a Cargo package to build the packages at Cargo.toml

Editors category

Implement editors:

• ed
• ex
• vi

rm::test_rm_deep_1 test fails: too many open files

MacOS 14.5 fails thusly:

% GETTEXT_SYSTEM=1 GETTEXT_DIR=/usr/local cargo test -p posixutils-tree --test integration rm::test_rm_deep_1 -- --nocapture
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.07s
     Running tests/integration.rs (target/debug/deps/integration-1e07fe9a2d399161)

running 1 test
thread 'rm::test_rm_deep_1' panicked at tree/tests/rm/mod.rs:175:34:
called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
test rm::test_rm_deep_1 ... FAILED

failures:

failures:
    rm::test_rm_deep_1

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 96 filtered out; finished in 0.08s

error: test failed, to rerun pass `-p posixutils-tree --test integration`

Implement pax utility

Implement pax: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/pax.html

Sporadic cp test failure

Sporadically, this failure will appear during tests:

test cp::test_cp_perm ... FAILED

failures:

---- cp::test_cp_perm stdout ----
thread 'cp::test_cp_perm' panicked at tree/tests/cp/mod.rs:524:37:
assertion `left == right` failed
  left: 296
 right: 288
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    cp::test_cp_perm

test result: FAILED. 47 passed; 1 failed; 16 ignored; 0 measured; 0 filtered out; finished in 2.02s

error: test failed, to rerun pass `-p posixutils-tree --test integration`

And cargo clean followed by rebuild + retest always fixes this issue.

rm::test_rm_rm3 test fails

rm::test_rm_rm3 test fails at line 868 of tree/tests/rm/mod.rs:

    let test_dir = &format!("{}/test_rm_rm3", env!("CARGO_TARGET_TMPDIR"));
...
    fs::create_dir(test_dir).unwrap();   // <<--- this fails

Manually removing the ./target/tmp directory causes the test to succeed:

thread 'rm::test_rm_rm3' panicked at tree/tests/rm/mod.rs:868:30:
called `Result::unwrap()` on an `Err` value: Os { code: 17, kind: AlreadyExists, message: "File exists" }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
test rm::test_rm_rm3 ... FAILED

failures:

failures:
    rm::test_rm_rm3

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 96 filtered out; finished in 0.00s

error: test failed, to rerun pass `-p posixutils-tree --test integration`
% vi +868 tree/tests/rm/mod.rs 
% find . -name test_rm_rm3
./target/tmp/test_rm_rm3
% rm -rf target/tmp/*
% GETTEXT_SYSTEM=1 GETTEXT_DIR=/usr/local cargo test -p posixutils-tree --test integration rm::test_rm_rm3 -- --nocapture
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.07s
     Running tests/integration.rs (target/debug/deps/integration-1e07fe9a2d399161)

running 1 test
test rm::test_rm_rm3 ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 96 filtered out; finished in 0.02s

rm::test_rm_rm3 test fails on Ubuntu LTS 22.04

Commit 3c32905 with recent fixes still fails on Ubuntu/Linux:

failures:

---- rm::test_rm_rm3 stdout ----
thread 'rm::test_rm_rm3' panicked at tree/tests/rm/mod.rs:897:9:
assertion `left == right` failed
  left: ["", "rm: descend into directory '/home/jgarzik/repo/posixutils-rs/target/tmp/test_rm_rm3/z'", "rm: remove directory '/home/jgarzik/repo/posixutils-rs/target/tmp/test_rm_rm3/z'", "rm: remove directory '/home/jgarzik/repo/posixutils-rs/target/tmp/test_rm_rm3/z/d'", "rm: remove directory '/home/jgarzik/repo/posixutils-rs/target/tmp/test_rm_rm3/z/du'", "rm: remove regular empty file '/home/jgarzik/repo/posixutils-rs/target/tmp/test_rm_rm3/z/empty'", "rm: remove regular empty file '/home/jgarzik/repo/posixutils-rs/target/tmp/test_rm_rm3/z/empty-u'", "rm: remove regular file '/home/jgarzik/repo/posixutils-rs/target/tmp/test_rm_rm3/z/fu'", "rm: remove symbolic link '/home/jgarzik/repo/posixutils-rs/target/tmp/test_rm_rm3/z/slink'", "rm: remove symbolic link '/home/jgarzik/repo/posixutils-rs/target/tmp/test_rm_rm3/z/slinkdot'"]
 right: ["", "rm: descend into directory '/home/jgarzik/repo/posixutils-rs/target/tmp/test_rm_rm3/z'", "rm: remove directory '/home/jgarzik/repo/posixutils-rs/target/tmp/test_rm_rm3/z'", "rm: remove directory '/home/jgarzik/repo/posixutils-rs/target/tmp/test_rm_rm3/z/d'", "rm: remove regular empty file '/home/jgarzik/repo/posixutils-rs/target/tmp/test_rm_rm3/z/empty'", "rm: remove symbolic link '/home/jgarzik/repo/posixutils-rs/target/tmp/test_rm_rm3/z/slink'", "rm: remove symbolic link '/home/jgarzik/repo/posixutils-rs/target/tmp/test_rm_rm3/z/slinkdot'", "rm: remove write-protected directory '/home/jgarzik/repo/posixutils-rs/target/tmp/test_rm_rm3/z/du'", "rm: remove write-protected regular empty file '/home/jgarzik/repo/posixutils-rs/target/tmp/test_rm_rm3/z/empty-u'", "rm: remove write-protected regular file '/home/jgarzik/repo/posixutils-rs/target/tmp/test_rm_rm3/z/fu'"]
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    rm::test_rm_rm3

test result: FAILED. 87 passed; 1 failed; 6 ignored; 0 measured; 0 filtered out; finished in 3.04s

error: test failed, to rerun pass `-p posixutils-tree --test integration`

Improve file(1) util to support modern Linux and MacOS binary db

Modern Linux and BSD (MacOS) version of file(1) use a binary database as the source of varied file type/format data.

The current posixutils version of file is POSIX compliant, supporting the text input format, but does not do anything useful on modern OS.

Goal: Extend file to support Linux and BSD binary file format databases.

Some early work can be found in PR #86.

User-facing apps category

User-facing apps category:

• bc
• mailx
• man
• talk

Recent rm(/cp/mv) tests create terminal newline/linefree display issue

Reproducible on both MacOS 14.5 and Linux Ubuntu 22 LTS.

Symptoms:

1. At some point during rm tests, the usual newline behavior display output changes from A (see below) to B (see below).
2. This issue impacts terminate state after cargo test ends. A reset command is required to return the terminal to its correct, normalized newline behavior.

Working normally, testing display state A:

test rm::test_rm_fail_2eperm ... ignored
test rm::test_rm_fail_eacces ... ok
test rm::test_rm_dangling_symlink ... ok
test rm::test_rm_i_1 ... ok
test rm::test_rm_i_no_r ... ok
test rm::test_rm_ignorable ... ok
test rm::test_rm_inaccessible ... ok
test rm::test_rm_ir_1 ... ok
test rm::test_rm_deep_2 ... ok

Abnormal test output, state B:

                              test rm::test_rm_no_give_up ... ignored
                                                                     test rm::test_rm_r_3 ... ok
                test rm::test_rm_r_4 ... ok
                                           test rm::test_rm_r_root ... ok
                                                                         test rm::test_rm_readdir_bug ... ok
                            test rm::test_rm_rm1 ... ok
                                                       test rm::test_rm_rm2 ... ok
  test rm::test_rm_rm3 ... FAILED
                                 test rm::test_rm_rm4 ... ok
                                                            test rm::test_rm_rm5 ... ok
       test rm::test_rm_sunos_1 ... ok
                                      test rm::test_rm_unread2 ... ok
                                                                     test rm::test_rm_unread3 ... ok
                    test rm::test_rm_unreadable ... ok
                                                      test mv::test_mv_leak_fd ... ok
     test rm::test_rm_hash ... ok
                                 test cp::test_cp_perm ... ok
                                                             test rm::test_rm_deep_1 ... ok
           test rm::test_rm_isatty ... ok

Development utilities

Implement development utilities:

• ar
• c99
• cflow
• ctags
• cxref
• fort77
• lex
• make
• nm
• strings
• strip
• yacc

Optimize wc utility

wc utility performance can be improved: https://github.com/robertdavidgraham/wc2

Localization (i18n): notes and planning

Introduction

Soliciting discussion over the localization (i18n) strategy for this project.

Goals

Goal 1: Localize everything

The goal is complete localization of all messages visible to the user, within the bounds of POSIX compliance:

• All util messages, collation sequences, charsets and other util i18n details
• All --help messages and related output (clap crate)
• All OS error messages (io::Result)
• It is believed that some minimum set of strings are required to be English/POSIX permanently. Minimize this POSIX-only set as much as possible.

Goal 2: Encourage UTF-8

To be forward-looking, this project looks for opportunities to

• Drop support for non-UTF 8 strings
• Default to UTF-8 charsets and strings

This project should aggressively interpret the POSIX standards in terms of UTF-8 support, and look for opportunities to create default-UTF-8 operating modes, with a fallback mode that is "POSIX-ly correct."

Implementation strategies

Current strategy

The current strategies are,

• use the gettext crate, and mark strings with gettext(). This provides a starting point for per-util coding, and at least gets us started on the road to i18n.
• each util sets the charset thusly,

    bind_textdomain_codeset(PROJECT_NAME, "UTF-8")?;

Improvements to our i18n

At present, OS error messages and --help are not translated at all, and need a project-wide strategy.

Also, one idea that is aligned with the gencat util is to generate catgets message catalogs and abandon gettext. This works because catgets exists on all modern platforms.

See issue #65 for util-related tasks.

Feedback and thoughts are requested. We want to give users the best i18n support possible.