amythicdev/minus

An asynchronous, runtime data feedable terminal paging library for Rust

minus

minus: A library for asynchronous terminal paging, written in Rust.

Motivation

Traditional pagers like more or less weren't made for integrating into other applications. They were meant to be standalone binaries that are executed directly by users. However most applications don't adhere to this and exploit these pagers' functionality by calling them as external programs and passing the data through the standard input. This method worked for Unix and other Unix-like OSs like Linux and MacOS because they already came with any of these pagers installed. But it wasn't this easy on Windows; it required shipping the pager binary along with the applications. Since these programs were originally designed for Unix and Unix-like OSs, distributing these binaries meant shipping an entire environment like MinGW or Cygwin so that these can run properly on Windows.

Recently, some libraries have emerged to solve this issue. They are compiled along with your application and give you a single binary to distribute. The problem with them is that they require you to feed the entire data to the pager before the pager can run, this meant that there will be no output on the terminal until the entire data is loaded by the application and passed on to the pager.

These could cause long delays before output to the terminal if the data comes from a very large file or is being downloaded from the internet.

Features

• Send data as well as configure the pager on the fly.
  This means that your data can be shown on the pager's screen as soon as it is loaded by your application. But not only that, you can also configure the minus while its running.
• Supports separate modes for dynamic and static output display
  This separation of modes allows us to do some cool tricks in static mode. For example in static mode, if the terminal has enough rows to display all the data at once then minus won't even start the pager and write all the data to the screen and quit. (Of course this behaviour can be avoided if you don't like it). Similarly, in static mode if the output is piped using the | or sent to a file using the >/>>, minus would simply pass the data as it is without starting the pager.
• Highly configurable
  You can configure terminal key/mouse mappings, line numbers, bottom prompt line and more with a simple and clean API.
• Good support for ANSI escape sequences
• Both keyboard and mouse support
  Key bindings highly inspired by Vim and other modern text editors
• Clutter free line numbering
• Horizontal scrolling Scroll not only up or down but also left and right.
  NOTE: ANSI escape codes are broken when scrolling horizontally which means as you scroll along the axis, you may see broken colors, emphasis etc. This is not a minus-specific problem but rather its how terminals behave and is inherently limited because of their design
• Follow output mode
  This feature ensures that you always see the last line as the data is being pushed onto the pager's buffer.
• Full regex based searching.
  Which also fully takes care of escape sequences. Also supports incremental searching of text as you type.
• Tries to be very minimal on dependencies.
• Is designed to be used with tokio, async-std or native threads as you like.

Usage

Add minus as a dependency in your Cargo.toml file and enable features as you like.

• If you only want a pager to display static data, enable the static_output feature

• If you want a pager to display dynamic data and be configurable at runtime, enable the dynamic_output feature

• If you want search support inside the pager, you need to enable the search feature

[dependencies.minus]
version = "5.6"
features = [
    # Enable features you want. For example
    "dynamic_output",
    "search",
]

Examples

You can try the provided examples in the examples directory by using cargo:

cargo run --example <example name> --features=<required-features>

# for example to try the `dyn_tokio` example
cargo run --example dyn_tokio --features=dynamic_output,search

See the docs for a summary of examples.

Standard keyboard and mouse bindings

Can be seen in the docs.

MSRV

The latest version of minus requires Rust >= 1.67 to build correctly.

License

Unless explicitly stated, all works to minus are dual licensed under the MIT License and Apache License 2.0.

Contributing

Issues and pull requests are more than welcome. See CONTRIBUTING.md on how to contribute to minus.

Thanks

minus would never have been this without the ❤️ from these kind people

And the help from these projects:-

• crossterm: An amazing library for working with terminals.
• textwrap: Support for text wrapping.
• thiserror: Helps in defining custom errors types.
• regex: Regex support when searching.
• crossbeam-channel: MPMC channel
• parking_lot: Improved atomic storage types
• once_cell: Provides one-time initialization types.
• tokio: Provides runtime for async examples.

Get in touch

We are open to discussion and thoughts om improving minus. Join us at Discord or Matrix.

minus's Issues

Tracking issue for performance improvments

The following commits address performance issues throughout various sections of minus

Test appending

The benchmarks have been taken to create a 1GB text buffer in minus. The (...) tells the time that it took to complete the task.

• 4a015a8 + c0475fd: Massive performance boosts (~ 2.72s)
• 00af2ec: 8.4% Code speedup (~ 2.51s)
• 67772fc: 5% Code speedup (~ 2.39s)
• 4611a13: 50% Code speedup (~ 1.59s)
• 63e4c2d + 9ce5d7a: 57% Code speedup (~ 1.01s)

Switch to a runtime agnostic Mutex

Calling page_all() twice panics

Describe the bug
It is currently not possible to show a second pager after the first (static) one finished.

To Reproduce

use minus::error::MinusError;

fn main() -> Result<(), MinusError> {
    for _ in 0..2 {
        let output = minus::Pager::new();

        output.set_exit_strategy(minus::ExitStrategy::PagerQuit)?;

        for i in 0..=100 {
            output.push_str(&format!("{}\n", i))?;
        }

        minus::page_all(output)?;
        }
    Ok(())
}

Expected behavior
It is possible to show many pagers sequentially.

Desktop:

• OS: Debian 11 on WSL 1 on win11
• Terminal: IntelliJ and MobaXterm
• Shell: none and bash
• 5.0.5 and main branch

Additional context
The changes from #82 are not complete. There is still one break each in the static/dynamic match which doesn't reset RUNMODE.

On the main branch, adding the missing reset leads to a deadlock. I think the dereferenced MutexGuard from the match is kept until the end of the match block. I didn't continue here because I ran into scrolling issues (#86).

Is there a reason why you need a Mutex? Otherwise atomics (for example crossbeam_utils::atomicAtomicCell) would be much easier to use for this.

Regression: "Q" key does not close the pager any more

Describe the bug
Pressing "Q" does not exit the pager any more.

I've bisected the versions and the regression occurs between versions 4.0.1 and 4.0.2.

Expected behavior
"Q" key exits as documented

Desktop:

• OS: Arch Linux
• Terminal: gnome-terminal
• Shell: zsh
• minus Version: 4.0.3 (latest)

Thank you for this library!

async runtime segfaults when working with large strings

Please refer to the segfault example in this branch:

https://github.com/rezural/minus/tree/segfault

run with:

cargo run --example segfault --features=async_std_lib

if you uncomment the line with prints the bytes_written, you can see that it segfaults semi randomly. sometimes at 50000 bytes_written, sometimes at 4 million.

I am actually hitting these segfaults when paging the output of ls */ (or find) in a folder with a large number of files.

Regression: "q" to close broken in v5.0.4

Describe the bug
When upgrading from 5.0.3 to 5.0.4 I see that pressing "q" no longer closes the paginator.

Also something that broke pre 5.0.3 - when pager quits it no longer clears the output.

To Reproduce
My code is along the lines of:

let mut pager = minus::Pager::new();
pager.set_exit_strategy(minus::ExitStrategy::PagerQuit).unwrap();
...
minus::page_all(pager).unwrap();

Expected behavior
"q" closes the pager and clears the displayed output (same behavior as git log pagination).

Desktop:

• OS: Arch Linux 5.18.12-3-MANJARO
• Terminal: gnome-terminal
• Shell: zsh
• minus Version: 5.0.4

`search` input is duplicated on Windows

Describe the bug

Key input in search is duplicated by non-press events reported by crossterm (e.g., Release events). This only affects Windows, as far as I know, since it's the only platform that has KeyboardEnhancementFlags::REPORT_EVENT_TYPES always set. This feature was introduced with the consumption of crossterm 0.26.1, viz., 39224c3, where key repeat and release events were added but not handled in minus code.

To Reproduce

1. Enter search functionality with some input suitable for testing.
2. Begin typing a search query, i.e., cat. Observe that letters are repeated, i.e., ccaatt, cacatt, etc.

I was able to reproduce this using this file layout in a new Cargo project:

• Cargo.toml:

  [package]
  name = "tmp-xz0anm"
  version = "0.1.0"
  edition = "2021"
  
  [dependencies]
  lipsum = "0.9.1"
  minus = { version = "5.6.1", features = ["dynamic_output", "search"] }

• src/main.rs:

  use std::{fmt::Write, thread, time::Duration};
  
  fn main() {
      let mut pager = minus::Pager::new();
      let pager_thread = thread::spawn({
          let pager = pager.clone();
          move || minus::dynamic_paging(pager)
      });
      for word in lipsum::lipsum(2_000).split_whitespace() {
          writeln!(pager, "{word}").unwrap();
          thread::sleep(Duration::from_millis(10));
      }
      pager_thread.join().unwrap().unwrap();
  }

Expected behavior

Screenshots

I don't think this is necessary, but can provide screenshots on request.

Environment:

• OS: Windows
• Terminal: Wezterm
• Shell: Nushell
• minus Version: 5.4.0 is the youngest release that does not reproduce this issue. 5.5.0 and onwards (ATOW, 5.6.1 is the latest) reproduce it, being broken by 39224c3.

Additional context

Noticed as the root cause of martinvonz/jj#3448.

crossterm upstream issue: crossterm-rs/crossterm#797

crossterm upstream is planning on making the KeyboardEnhancementFlags::REPORT_EVENT_TYPES feature opt-in with crossterm-rs/crossterm#778. However, this change still seems desirable, since it's more robust and correct overall.

A couple of issues using the static output mode

Hello! I would like to report a few issues I have encountered while using minus version 5.3.0 for paginating static output. Here is a sample program to illustrate the problems:

fn page_output(data: &str) -> Result<(), minus::error::MinusError> {
    let pager = minus::Pager::new();
    pager.set_exit_strategy(minus::ExitStrategy::PagerQuit)?;
    //pager.set_run_no_overflow(true)?;
    pager.set_text(data)?;
    minus::page_all(pager)?;

    Ok(())
}

fn main() {
    page_output("Lorem ipsum dolor sit amet, consectetur adipiscing elit.").unwrap();
    page_output("Morbi enim purus, maximus eget justo vel, efficitur pretium arcu.").unwrap();
}

The issues I've encountered are:

1 - Selecting paginated text using the mouse doesn't work. This is a regression that occurred between 5.0.1 and 5.0.2
2 - If I uncomment the set_run_no_overflow(true) line, the program crashes with the following output:

Lorem ipsum dolor sit amet, consectetur adipiscing elit.
thread 'main' panicked at 'Failed to set the RUNMODE. This is caused probably bcause another instance of minus is already running', /home/renato/.cargo/registry/src/index.crates.io-6f17d22bba15001f/minus-5.3.0/src/static_pager.rs:29:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

3 - The documentation of Pager::set_run_no_overflow says the following:

Setting this to true will cause a full pager to start and display the data even if there is less number of lines to display than available rows.

However, the actual behavior is the opposite, and passing false will cause a full pager to start no matter how much data is provided.

Thank you for your hard work on minus, and I appreciate any help you can provide to resolve these issues.

Write docs for InputHandler and DefaultInputHandler

Initially outputting a string more than the terminal height, leaves the screen blank until a key is pressed

I have a branch which demonstrates this bug:

https://github.com/rezural/minus/tree/large-page-bug

cargo run --example large_page_bug --features='async_std_lib'

you will see that the initial output is blank. also, uncomment
https://github.com/rezural/minus/blob/large-page-bug/examples/large_page_bug.rs#L17

there is a similar bug that doesnt output the last chunk of string that goes over the height.

I'm debugging now, but you may have a quick fix.

Thanks for the library!

Panic at runtime: minus-5.5.2/src/core/init.rs:121:9 - Failed to set the RUNMODE when using minus 5.5.2

Hello!

Describe the bug
When compiling with 5.5.2 and running using dynamic_output, I get a panic at runtime, which can be recreated using the less-rs example. This does not happen when compiling with 5.5.1.

Full stack trace:

thread '<unnamed>' panicked at /home/mark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/minus-5.5.2/src/core/init.rs:121:9:
Failed to set the RUNMODE. This is caused probably because another instance of minus is already running
stack backtrace:
   0: std::panicking::begin_panic
   1: minus::minus_core::init::init_core
   2: minus::dynamic_pager::dynamic_paging
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
thread 'main' panicked at src/main.rs:28:17:
called `Result::unwrap()` on an `Err` value: Any { .. }
stack backtrace:
   0: rust_begin_unwind
             at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/std/src/panicking.rs:597:5
   1: core::panicking::panic_fmt
             at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/core/src/panicking.rs:72:14
   2: core::result::unwrap_failed
             at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/core/src/result.rs:1652:5
   3: fatcat::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

To Reproduce
I can recreate this error with the less-rs example and minus 5.5.2.

Expected behaviour
Contents of file paged to terminal.

Desktop:

• ZorinOS 16
• rustc 1.74.1
• minus 5.5.2

Please let me know if I missed any information, I can workaround this by using 5.5.1.

Flashing when mouse moving

Describe the bug
when I move my mouse on the terminal, pager is flashing.

To Reproduce
code as follows:

use std::fmt::Write;
use minus::{self, Pager};

fn main() -> Result<(), minus::MinusError> {
	let mut out = Pager::new();

	write!(out, "{}\n", "hello world ".repeat(1000))?;

	minus::page_all(out)?;

	Ok(())
}

and config on Cargo.toml is:

minus = { version = "5.5.1", default-features = false, features = [
	"static_output",
] }

run cargo run and move the mouse on terminal.

Expected behavior
it should not be flashing

Screenshots

Desktop:

• WSL, Ubuntu 22.04.2 LTS
• Windows Terminal
• zsh

Something else
I feel like I'm going blind...

Horizontal scrolling

Is it possible to add horizontal support?

Feature Request: add support for an EOF callback

Feature description

I'd like to use minus to display paginated data that is pulled from an API. Ideally, the data would be paginated as the user scrolls through the logs, in a similar manner to an infinite scroll component.

Proposed Solution

An additional config on the PagerState, similar to the exit callback vectors. The callback should be fired when the user reaches the end of the currently available data.

Alternative

There's no clear alternative — while minus supports streaming data in, there's no obvious way to fetch more data at the rate that the user is consuming.

I am happy to contribute the changes to include this in the library.

Inconsistant formatting with wrapped text

Describe the bug
First of all, great performance improvements in the latest update - actually made me revist a programme I was working on. However I have come across a bug regarding textwrapping and being redrawn to the terminal.

Essentially when a string is formatted (for example with a colour) and that text is wrapped, if you scroll past it (so it isnt rendered) and then scroll back up (to re-render it) the wrapped portion of the string loses its formatting.

The loss of formatting occurs when scrolling with mouse wheel or using the arrow keys for navigation. However, when using the Page Up and Page Down keys, the formatting is preserved. The behavior with the scroll wheel is inconsistent - sometimes the formatting remains intact, while other times it gets lost. In contrast, the issue consistently arises when using the arrow keys.

In addition the formatting seems inconsistant when using the arrow keys even without reloading - occasionally the wrapped text wont have the formatting applied at all, again using PgUp/PgDown doesnt have this issue. I couldnt work out a cause for this.

To Reproduce
An example to reproduce below (ensure terminal is narrow enough to wrap the highlighted text)

use std::thread::spawn;
use minus::{Pager, dynamic_paging};
use crossterm::style::{StyledContent, Stylize};

fn main() {
    let mut pager = Pager::new();
    let pager2 = pager.clone();
    let pager_thread = spawn(move || dynamic_paging(pager2));

    // Create formatted string
    let string_to_print = format!("{}{}{}", "1".repeat(110), "2".repeat(50), "\n\n");
    let search = "1111111112222";
    let styled_sequence = string_to_print.replace(search, &search.dark_yellow().to_string());

    // It also happens using terminal escape codes (see below)
    //let styled_sequence = string_to_print.replace(search, &format!("\x1b[33m{}\x1b[0m", search));

    pager.push_str(styled_sequence.repeat(100)).unwrap();
    pager_thread.join().unwrap();
}

Expected behavior
Formatting is preserved when wrapped and reloaded after scrolling past.

Screenshots
As you can see the first 4 rows have been scrolled past, and reloaded and they lose the yellow formatting:

Desktop:

• WSL2 Ubuntu 22:04
• zsh
• 5.6.1

Additional context
I have also replicated this issue on native Ubuntu 23

Minus does not seem to work on macOS

Hello,

I've been looking for a terminal paging library for Rust and have come across Minus. I've tried to run the examples in both zsh and bash on macOS. However, when I do this:

cargo run --example=static --features=static_output

I always get this error:

Error: Paging(Setup(InvalidTerminal))

Is macOS not supported yet? If not, what's the problem here? Thanks.

Smol support

I'd like to add support for the smol runtime. I'm opening this issue to make sure you wouldn't have an objection to this feature before I go the trouble of implementing it. Thanks.

General performance improvements

I'm aware that it's an edge case, but I regularly work with files that are 100,000 + lines long with rager and minus can be fairly slow when working with them. This issue is simply requesting significant general performance improvements when paging large files.

I've ran rager through a profiler a few times when paging these massive files to determine where the bottlenecks are, and now I know where they are and have a few ideas for what can be done to eliminate them:

1. I have a fork with general performance improvements (it affects static paging with no line numbers most significantly, but I think it affects all paging at least somewhat), and I can file a PR with my changes if you would like. If you'd like me to file this PR, I'd still have more changes that I think should be made, since the changes in this PR are more like a band-aid than a fix.
2. Performance could be significantly improved by rewriting how the process of flattening + adding line numbers + search result highlighting works. Instead of doing this to all of the text every time you scroll at all (as happens in annotate_line_numbers), you could store, within the Pager struct, a vector of all the lines of text, already flattened with the lines numbers and search results highlighted. Then, whenever the terminal is resized, the search term changes, or line numbers are toggled, this vector could be regenerated. This would allow, then, for basically no lag when someone is only scrolling (as opposed to the major lag that can happen right now).

I'd be happy to put some work into helping to implement either one of these options, if you'd like. Before putting any more work into this or filing a PR, though, I'd appreciate feedback on your thoughts and what direction should be taken for this issue (if any).

Thank you so much!

unused `Results`

There are quite a few unused Results in this library.

Normally the compiler would complain about this, but they've been assigned to anonymous variables (let _ = result). This keeps the compiler happy, but doesn't actually address the issue. These are all functions which can fail, and with this pattern they will fail silently, which in my view is even worse than sprinkling unwraps everywhere.

I count 22 instances of fallible functions where the error case is not handled

Ability to select and copy paged text

Is your feature request related to a problem? Please describe.
When paging through a file with more or viewing git log you can mouse-drag over or double-click the text to select and copy it. Text selection currently doesn't work with minus, which is quite limiting.

Describe the solution you'd like
Text selection should work similarly to other paginators

Additional context
Not sure if this a bug or missing feature - got same results under bash and zsh.

Changes regarding searching and others

Although searching has been implemented, there is one major bug and a missing feature

1. If search is involved twice with the same query, the search results are duplicated
2. Backward searching is yet to be implemented

Also dynamic_output function has probably grown too much. Clippy is already warning about it. Therefore we should probably break it into smaller functions

Scrolling is broken on main branch

Describe the bug
Scrolling down shows junk lines. Scrolling back up skips lines.

To Reproduce
cargo run --features="static_output" --example static

Expected behavior
The screen always shows the numbers in the correct sequence.

Screenshots
Scrolling downwards:

Scrolling back up:

Desktop:

• OS: Debian 11 on WSL 1 on win11
• Terminal: IntelliJ and MobaXterm
• Shell: none and bash
• main branch

Additional context
The cause is a dbg!() in src/core/display/mod.rs:63. I will create a PR to remove it.

Static (page_all) output is breaking zsh

Thank you for your work on this crate. I just gave 4.0.0-alpha1 version a try.

Pagination works as expected with a few minor (unrelated) issues:

• I expected space bar to be same as PgDn just like in less
• Search does not seem to navigate to the first/next occurence
• Highlighting of search result seems to break colors of the text being highlighted

But the main issue is that after I exit my program every mouse movement in the terminal window is causing some garbage output:

It's quite hard to recover the terminal session after this.

Happy to help to test potential fixes or help narrow down what's causing this.

Resizing does not update the view

Hi,
this is already really nice.

However I recognized, that when I start with a small window and enlarge this, the view/output does not get updated.

Hardy

Peessing `[n]G` takes to nth row not the nth line

Describe the bug
Pressing [n]G jumps to the nth row not the nth line. This occurs because in the definition of the key binding, it simply asks minus to move the upper mark to the 50th row.

high use of CPU

With a large text the CPU is overheating.

Native input definitions

This major feature lets end-applications to define key/mouse bindings for minus by neither requiring to copy and paste the default input definitions and changing the parts that they want nor needing to explicitly depend on crossterm.

When this feature is incorporated, you can for example define a key binding like this

map.add_key_events(["enter"], |ev, ps| {
// ....
});

Do note that by introducing this new method for input definition, we aren't deprecating the legacy method of copy-pasting so your legacy code will run fine even when this feature is introduced.

PagerState.message is private

Describe the bug

I want to add a bit change to DefaultInputClassifier to customize keybinding.
So I copied DefaultInputClassifier to my CustomInputClassifier.
But it can't be compiled.

To Reproduce

The error message is below:

error[E0616]: field `message` of struct `PagerState` is private
  --> src/custom_input_classifier.rs:46:23
   |
46 |                 if ps.message.is_some() {
   |                       ^^^^^^^ private field

Expected behavior

Changing message to public or adding an API which has the same behavior as message.is_some().

IMPORTANT: Switching to Conventional Commits

As minus is continuously growing in size along with new contributors and users, it is essential to have proper standards for all the important aspects of the project.

minus is written in Rust, and the Rust Community has already defined
a good set of rules for codebases written in Rust and minus also blindly follows these rules.

But one such area where minus lacks a good standard is Git commit messages. Although the CONTRIBUTING.md file describes a format for commit messages, it isn't thorough enough to be imposed as a standard.

To improve the quality of commits, minus will now officially switch to Conventional Commits v1.0.0 and all commits from now on will be strictly under Conventional Commits v1.0.0. Contributors are also required to write their commit messages that follow Conventional Commits v1.0.0.

There are some pre-requisites that should be clearly converted before this issue can be enacted upon.

• Clearly define the scopes. Also describe all the types that can be used.
• Write more guidelines in CONTRIBUTING.md

Additional Links

• Conventional Commits Specification: https://www.conventionalcommits.org/en/v1.0.0/#specification

Follow-mode

Is your feature request related to a problem? Please describe.
I'm trying to contribute to tailspin which is a pager especially for log files and it's maybe pretty neat to see the latest lines if new lines are popping up.

Describe the solution you'd like
minus provides an option like .follow(true) where minus always scrolls down to the bottom if new lines are added.

Describe alternatives you've considered
Manually scrolling in the terminal.

Additional context
I'd like to create a PR for this, since I think that this shouldn't be that hard.

Introduce `DataSource` trait

The Problem

Although minus is quite performant in its routine stuff like efficiently drawing the terminal, handling terminal events etc however one of the areas where it isn't optimized much is with processing the input data.

Currently we store two copies of the entire data, one is the original data without any formatting which is used to recreate the formatted data. The other one is of course the formatted data that is suitable for the terminal and includes stuff like line numbering and search highlights. another aspect related to this is that minus currently employs an eager model to format data which means minus will immediately try to format all the data that it received.

Although this works, there are several drawbacks to this approach:-

• Currently minus cannot page over data that exceed the size of the available RAM in the machine.
• It takes a really long time to format large data which causes the main thread to stay paused during that interval.

Proposed Solutions

Memory maps

Here are two of the most plausible solutions that I have found:

One of the proposed solutions suggested by @TornaxO7 who gave an amazing explanation in his comment was to store the data in a memory-mapped file and read through from it.

One of the main criticism against this idea that I had is that it relied on memory-maps which although work fine in all UNIX-style systems but kinda sucks on Windows. Specifically Windows doesn't have anonymous memory-maps.

DataSource

So another solution that I want to propose here which is highly inspired by the previous solution is the DataSource trait. The trait will allow applications to hook up their data into minus without them loading the data for minus. This means a simple file pager can register the file into minus without loading the the entire file buffer into memory. Similarly a network based application can hook up the socket to minus without reading the entire socket. Now as the user scrolls through the page display, minus will automatically load the data from the source into its buffer.

It will have the following signature^[*1]^

trait DataSource {
    fn reads_forward(&mut self) -> Vec<String>;
    fn reads_backward(&mut self) -> Vec<String>;
    fn readr_lines(&mut self, index: usize) -> (String, Vec<String>, Vec<String>);
}

The reads_forward() and reads_backward will ask the source the read a couple of lines in forward/backward direction. respectively The s in their names suggest that they will be called when lines needs to be read sequentially which means relative to the upper_mark.

The readr_lines will ask the source to fetch the line at index index and also some lines that are adjacent to index in both direction. In the returning tuple the first element is the actual line at index index, the 2nd element are a couple of lines after index and the third element are couple of lines before index.

The exact number of lines to read is still undecided. When we finalize on this the Vecs will by replaced by fixed sized arrays.

[1]: This is not final and it may have changes down the road.

Related to: #106
cc: @TornaxO7, @FlipB

No prompt displayed initially

Describe the bug
When starting the pager, no prompt is displayed It is only displayed when turning on line numbers or doing a search

Expected behavior
On starting the pager, the prompt should be displayed. When no search is active, no search count should be displayed but when search is activated, the search counter should be visible in the rightmost section.

Screenshots
Current:

Expected:

Desktop:

• OS: Arch Linux
• Terminal: Alacritty
• Shell: fish
• minus Version: v5.0.2

Support for scrolling more than 1 line at a time while paging

This is a feature request for scrolling more than 1 line at a time while paging, akin to typing 5k or 100j while viewing a file in vim.

I have worked on this a bit and have a few implementations that will work, but they all require breaking changes to the InputClassifier trait. This is because, before this feature, the input classifier didn't need to store any state between different calls of classify_input, but it must to implement this feature, since it requires handling multiple characters.

The three implementations I've considered (and successfully implemented) are:

1. Adding another parameter to the classify_input fn, e.g.line_count: &mut Option<String> so that if a numeric key is pressed, it can be appended to line_count, then the fn can return None, and it assume that nothing was input.
2. Changing the parameters of classify_input to take, instead of many parameters, a single struct (e.g. InputParameters) that will contain everything that was previously a parameters of classify_input. This will allow you to add extra members (and thus functionality) to InputParameters later without it being a breaking change
3. Modifying the classify_input fn to take &mut self instead of &self so that the DefaultInputHandler can store and modify state when classify_input is called. I think this is the best solution, but I also think it may take more work than the others (borrow checker gets in the way with some mutable/immutable reference rules when you try to just change &self to &mut self).

I would very much like this feature, and would like your thoughts on if you would like it as well, and what you think the best solution is. If you can think of a different way to implement this that would not require a breaking change, that would also be much appreciated and I'd be more than happy to put some work into it.

Minus 4.1

This roadmap will lay down the next release iteration of minus. It will be a minor release, having minimal breaking changes

The roadmap is as follow:-

• Use std::sync::Mutex and drop async_mutex
• Improve the documentation
• Add support for OS threads

Pager panics when static output has > `u16::get_max()` lines

Describe the bug
When the static output has more lines than u16::get_max() and the user attempts to move to a line whose number is larger than u16::get_max(), the pager panics. This is because of the unwrap on line 106 of search.rs.

To Reproduce
Statically page output with more than u16::get_max() (65535) lines of output, then jump to the bottom with G

Expected behavior
The pager should not crash.

Desktop:
macOS Big Sur, iTerm2, rustc 1.55 stable, minus version 4.0.1.

I have a fix ready to go that doesn't crash and passes all tests when running cargo test --release --all-features. If you'd like this to be fixed, I'd love to submit a PR with the fix.

Starting at the end of input

Hi there!

I'm currently this crate for a log displaying system, and I'd like to have the ability to start the pager at the very end of the log's content. That way users see the last messages first, and can then scroll up to see the older ones.

I didn't find any option to set this behaviour in the crate, so would be it possible to implement it?

Thanks in advance for your help :)

Error Handling

The error handling in this crate is now in a bit of a weird state

• Most of the error handling uses AnyHow, which really shouldn't be used in library crates
• The error::Error struct doesn't appear to be used anywhere?
• the error::Result type alias is almost exactly a restatement of anyhow::Result

I could take a look at what this would look like with concrete error types, instead of the 'type-erased' error handling that's in use currently?

Implement backpressure

Is your feature request related to a problem? Please describe.
Minus becomes unresponsive when attempting to page big files (gigabytes).

Describe the solution you'd like
It seems like the pager's buffer is unbounded and keeps growing to attempt to fit the entire input.
Instead the pager ought to limit the buffer to eg. 1000 lines, and block write_str when full.
Scrolling the output consumes the buffer, allowing new lines to be written by write_str.

Additional context
It spends virtually all time in PagerState::append_str, pegging the CPU at 100%.

Feature Request: Modern Search UX

Is your feature request related to a problem? Please describe.
I haven't found a pager that has a search feature similar to modern graphical text editors or user-friendly terminal emulators such as GNOME Terminal or Konsole. I'm currently using the Kitty terminal emulator which searches its scrollback via a pager.

Describe the solution you'd like

• additional highlighting of the currently focused result relative to other results
• display number of total results and currently focused result (#102)
• incremental search (#105)
• context-dependent deactivation of incremental search, for instance if (number of lines > X && search string length < Y). (I made this one up and would personally set it to something like X=1000 and Y=4) (#105)

Describe alternatives you've considered
zellij, less, moar, most, kitty-kitten-search

Feature Request: Text Selection

Till this date, minus doesn't allow users to select or copy text which really limited it's use case. This issue aims to implement a text selection feature using the mouse.

Refactor all text handling functions

The situation of text handling functions like append_str(), format_lines(), formatted_line() are no less than complete mess.

Most functions are tightly related to PagerState.
They depend on each other like a web which makes them really difficult to understand for newcomers. Most of them are extremely long with not so good documentation. It becomes very easy to introduce bugs like #66. Most importantly it becomes very difficult to track where things might have gone wrong.

Through this, we intent to eliminate all of these issues once and for all.

• Generalize make_append_str() to the point where it not only formats incoming text to be appended but to the entire data inside minus (Maybe give it a better name after then).
• Refactor formatted_line() and make it independent of PagerState.
• Refactor format_lines() to the point where it's just a special case of make_append_str() where the text to format is the entire data of given to minus till that point.
• Move most of the stuff to the core/utils/text.rs module.
• Add better documentation for text module.
• Fix all failing tests due to all the refactoring.

Remove . vscode

@danieleades in a recent commit, you have pushed your . vscode folder. Could you please remove it, and maybe add it in the . gitignore so that others don't repeat the same mistake

FX mode

Is your feature request related to a problem? Please describe.
I'm using minus as a pager in a program where we would like functionality similar to less -FRX where it would be nice to have the output remain on screen after it exits (less -X) and automatically exit if the entire file can be displayed on the screen (less -F). This ideally would also mean not clearing prior scroll-back when the entire file or less can be displayed.

Describe the solution you'd like

I took a first pass at -X in #128 . I'm not sure what the right approach for -F is.

Feeding minus with text from a network call results in corrupted text until resizing

Describe the bug

Hi,

I'm using reqwest and tokio to make an async network call. Utilizing the stream feature of reqwest I'm reading chunks of the response text and write! it into minus using the dynamic_paging feature.

Then the text, line numbers and so forth gets corrupted until I resize the terminal window, which (I assume) causes a refresh of the view.

To Reproduce

use bstr::ByteSlice;
use bytes::Bytes;
use minus::LineNumbers;

#[tokio::main]
async fn main() {
    use futures::StreamExt;
    use minus::{dynamic_paging, MinusError, Pager};
    use std::fmt::Write;
    use tokio::{join, task::spawn_blocking};

    let pager = Pager::new();
    pager.set_line_numbers(LineNumbers::Enabled).unwrap();

    let task = async {
        let mut pager = pager.clone();
        let client = reqwest::Client::default();
        let mut stream = client.get("https://www.google.de/")
            .send()
            .await
            .unwrap()
            .bytes_stream();

        while let Some(chunk) = stream.next().await {
            let bytes: Bytes = chunk.unwrap();
            let text = bytes.to_str_lossy().to_string();
            write!(pager, "{}", text).unwrap();
        }
        Result::<_, MinusError>::Ok(())
    };

    let pager = pager.clone();
    let (res1, res2) = join!(spawn_blocking(move || dynamic_paging(pager)), task);
    res1.unwrap().unwrap();
    res2.unwrap();

}

Cargo.toml

[package]
name = "minus-bug"
version = "0.1.0"
edition = "2021"

[dependencies]
reqwest = { version = "0.11.10", features = ["stream"] }
minus = { version = "5", features = ["dynamic_output", "search"] }
tokio = { version = "1.17.0", features = ["full"] }
futures = "0.3.21"
anyhow = "1.0.56"
bytes = "1.1.0"
bstr = "0.2.17"

Screenshots

This is the immediate result:

As you can see, the line numbering is wrong and the content ins't correctly display. However, if I now resize the terminal everything is fine:

Desktop:

• OS: macOS 12.2.1
• Terminal: Iterm and the native terminal.app
• Shell: zsh
• minus Version: 5.0.1

Additional context

When I don't stream the text-chunks but instead using the text method from reqwest, which loads the complete text into a single string with a single write! it works like a charm.

Feature Request: Hooks

Some actions inside minus need to performed on certain events regardless of the situation in which that event occured. For example, doing cleanup needs to be done anytime the user exits, regardless of whether the pager is in static or dynamic mode. Hence we need to proper system to tap into some special events and call functions on that event. Hooks are a feature that allows multiple callback functions to be registered with a event and whenever that event occur, all of those functions would be called.

This feature will also allow end-applications to add their own events. For example in a application that fetches data from the internet, whenever the user reaches the end of output, a callback function would be called to fetch more data.

Broken output displayed when scrolling with mouse

Describe the bug
The output gets distorted and broken when the user tto scroll with the mouse while simultaneously data is also inserted into the screen.

To Reproduce
Scroll using the mouse while data is incoming.

Expected behavior
The output should not be broken

Desktop:

• minus Version: v5.2.1

Implement `std::io::Write` for Pager.

Is your feature request related to a problem? Please describe.

I am trying to make function generic function that takes data and writes it to that output. Before I had it taking std::io::Write since that is what stdout implements. However this did not work when trying to add minus to my library since it only implements std::fmt::Write.

Describe the solution you'd like

std::io::Write to be an impl for Pager.

crates.io v3 minor version bump

Could there be a minor version bump in crates.io for #39 ?
This allows me to publish my crate using main/HEAD features.

Panics when searching for multi-byte characters

When searching for strings containing double width characters, the program panics at this call to String::drain, since the endpoint is not on a char boundary. I haven't figured out how to fix it.

Steps to reproduce

Search 甲州 in the pager with the following code.

Dummy text in Japanese: text.txt

main.rs

use std::fmt::Write;

const STR: &str = include_str!("./text.txt");

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut pager = minus::Pager::new();
    write!(pager.lines, "{}", STR)?;
    minus::page_all(pager)?;
    Ok(())
}

Cargo.toml

# ...
minus = { version = "3.3", features = ["search", "static_output"] }

Panic backtrace

thread 'main' panicked at 'assertion failed: self.is_char_boundary(end)', /home/akitaki/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/string.rs:1572:9
stack backtrace:
0: rust_begin_unwind
    at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/panicking.rs:493:5
1: core::panicking::panic_fmt
    at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/panicking.rs:92:14
2: core::panicking::panic
    at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/panicking.rs:50:5
3: alloc::string::String::drain
    at /home/akitaki/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/string.rs:1572:9
4: minus::search::find
    at /home/akitaki/.cargo/registry/src/github.com-1ecc6299db9ec823/minus-3.3.3/src/search.rs:194:9
5: minus::search::highlight_search
    at /home/akitaki/.cargo/registry/src/github.com-1ecc6299db9ec823/minus-3.3.3/src/search.rs:166:13
6: minus::init::static_paging
    at /home/akitaki/.cargo/registry/src/github.com-1ecc6299db9ec823/minus-3.3.3/src/init.rs:78:32
7: minus::static_pager::page_all
    at /home/akitaki/.cargo/registry/src/github.com-1ecc6299db9ec823/minus-3.3.3/src/static_pager.rs:86:13
8: indicatif_test::main::{{closure}}
    at ./src/main.rs:10:5
9: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
    at /home/akitaki/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80:19
10: tokio::park::thread::CachedParkThread::block_on::{{closure}}
    at /home/akitaki/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.6.0/src/park/thread.rs:263:54
11: tokio::coop::with_budget::{{closure}}
    at /home/akitaki/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.6.0/src/coop.rs:106:9
12: std::thread::local::LocalKey<T>::try_with
    at /home/akitaki/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/local.rs:272:16
13: std::thread::local::LocalKey<T>::with
    at /home/akitaki/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/local.rs:248:9
14: tokio::coop::with_budget
    at /home/akitaki/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.6.0/src/coop.rs:99:5
15: tokio::coop::budget
    at /home/akitaki/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.6.0/src/coop.rs:76:5
16: tokio::park::thread::CachedParkThread::block_on
    at /home/akitaki/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.6.0/src/park/thread.rs:263:31
17: tokio::runtime::enter::Enter::block_on
    at /home/akitaki/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.6.0/src/runtime/enter.rs:151:13
18: tokio::runtime::thread_pool::ThreadPool::block_on
    at /home/akitaki/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.6.0/src/runtime/thread_pool/mod.rs:71:9
19: tokio::runtime::Runtime::block_on
    at /home/akitaki/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.6.0/src/runtime/mod.rs:452:43
20: indicatif_test::main
    at ./src/main.rs:11:5
21: core::ops::function::FnOnce::call_once
    at /home/akitaki/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Fix tests

The current testing module is broken. Could you please fix it @poliorcetics , since I am currently working on refinement for the next major version. I would be really thankful to you