Text wrapping at < 79 columns?

[cybershadow]

Text wrapping at < 79 columns?

[CyberShadow]

DFeed wraps text at the recommendation of RFC 2646:

When generating Format=Flowed text, lines SHOULD be shorter than 80 characters. As suggested values, any paragraph longer than 79 characters in total length could be wrapped using lines of 72 or fewer characters. While the specific line length used is a matter of aesthetics and preference, longer lines are more likely to require rewrapping and to encounter difficulties with older mailers. It has been suggested that 66 character lines are the most readable.

[simendsjo]

66 is perhaps the most readable, but many programmers wraps text at 79 :)

[CyberShadow]

This makes all text written with manual wrapping look very bad.

I think this is the real problem - DFeed does not provide a way to post pre-wrapped text, does not warn the user if they attempt to post pre-wrapped text, and does not allow previewing the result of its own text wrapping.

[simendsjo]

How about combining the efforts of Vibenews? It has preview and the ability to use markdown when posting. It also looks better with threaded messages (not using >>>>)

[simendsjo]

.. Vibenews uses DFeed under the hood I think. Don't know how much it has changed it though.

[CyberShadow]

AFAIK, Vibenews shares no code with DFeed, even though the Vibe.d website used DFeed for a short period of time.

The purpose of Vibenews is different from the purpose of DFeed. Rich text formatting will not be added to DFeed, because it will not be accessible to users of other client software, and users of other client software may accidentally post text that unintentionally triggers formatting when viewed with DFeed.

[Llammissar]

Regardless of the RFC, the current behaviour tends to make proper quoting difficult because of how it wraps fixed-length lines without recombination.

It gives you this amazingly awkward sort of line, orphan, line,
orphan
thing where you don't want to just LEAVE it, but you also don't
know
for sure what the limits are as you (manually) reformat it.

Regardless of your perspective on the psychovisual aspect (if you must be strict about it, 72 is also within the "good" bounds according to research, and much more common in computing), I think some of the friction would dissipate if quoted paragraphs when there is a series of lines greater than 66 em were automatically rewrapped for the reply box.

EDIT: Also, some way of determining what column you're at would be super useful. Maybe if the JS isn't too heavy, make it autowrap as you type?

[CyberShadow]

Regardless of the RFC, the current behaviour tends to make proper quoting difficult because of how it wraps fixed-length lines without recombination.

I don't know what you mean by this. How exactly does it make quoting difficult? DFeed implements RFC 2646, and neatly rewraps text quoted from messages created using a RFC2646-compliant user agent. Some ancient/broken UAs (e.g. Pine or Mutt, I think) don't forrow RFC2646, so DFeed can't wrap their text neatly which generates the wrapping artifacts as in your example.

[Llammissar]

I don't know what you mean by this. How exactly does it make quoting difficult?

Apologies; to make it explicit, I'm referring specifically to behaviour when replying to a post. If I'm replying inline I have to either tolerate the quoted text to taking up a bunch of extra vertical space, I have to manually reformat the quoted text without any indication on the form if my reformat will also fall afoul of the reflowing rules. That's tedious for me or annoying for everyone else to read. This isn't some unusual or hypothetical thing, either; here's a recent example in the wild: http://forum.dlang.org/post/[email protected]

For that post, the response form wraps e.g. the second section like so:

> 1. I'm still sad there are no allocAligned() functions or 
> something of that
> type to request explicit alignment with allocations. I'm not 
> sure there is
> sufficient support for requesting alignment with allocations. 
> The set-able
> alignment property approach seems a little weird (and only 
> seemed to be
> supported on one allocator?). I guess experience will tell if 
> this is
> sufficient and/or convenient.
> I'd still like to see an allocWithAlignment() method or 
> something, which
> may be implemented efficiently by allocators that can support 
> it.

Whereas I think this would make more sense in most cases (just a quick reflow in vim with tw=64):

> 1. I'm still sad there are no allocAligned() functions or
> something of that type to request explicit alignment with
> allocations. I'm not sure there is sufficient support for
> requesting alignment with allocations. The set-able alignment
> property approach seems a little weird (and only seemed to be
> supported on one allocator?). I guess experience will tell if
> this is sufficient and/or convenient.  I'd still like to see an
> allocWithAlignment() method or something, which may be
> implemented efficiently by allocators that can support it.

I realise it's not perfect (as it consumed some newlines that were apparently intentional), but even this naive approach is substantially better for normal use. A slightly smarter way would be to check for sequences of lines longer than 66 characters and only reformat those:

> 1. I'm still sad there are no allocAligned() functions or
> something of that type to request explicit alignment with
> allocations. I'm not sure there is sufficient support for
> requesting alignment with allocations. The set-able alignment
> property approach seems a little weird (and only seemed to be
> supported on one allocator?). I guess experience will tell if
> this is
> sufficient and/or convenient.
> I'd still like to see an allocWithAlignment() method or
> something, which may be implemented efficiently by allocators
> that can support it.

Not great, but possibly a good compromise. (As an interesting aside, if I don't put those into indented blocks, github's response form reflows all three identically as quotes of a numbered list at its native response width. Intentional formatting newlines are clobbered as in the second version.)

Fundamentally, I disagree that someone's UA failing to respect RFC should make DFeed users' experience tangibly worse. I know this isn't completely possible in practice; there are always edge cases, but I think they can at least be mitigated.

DFeed implements RFC 2646, and neatly rewraps text quoted from messages created using a RFC2646-compliant user agent.

This is cool, but as a user that really doesn't mean anything to me. Does that mean if I insert a quote that's one long line...

> 1. I'm still sad there are no allocAligned() functions or something of that type to request explicit alignment with allocations. I'm not sure there is sufficient support for requesting alignment with allocations.

...it will be flowed properly as four quoted lines? I have no idea and no way of knowing short of trying it.

I hope this clarifies things?

[CyberShadow]

That message was generated by a user-agent not compliant with RFC 2646. Unfortunately, it is impossible for DFeed to rewrap messages like that, because the quoted content might be code with deliberate word breaks. It would mean that quoting a code block would turn the whole code into soup, since DFeed does not know whether each line break is a soft or hard line break. This problem was solved by RFC 2646. I'm sorry, but I did think about this before, while writing DFeed, and also could not find a good solution, and I believe the current behavior is better than others.

As I wrote above, I accept that it would be good to have preview functionality. #24 is a partial duplicate of this issue.

Another possible improvement, which I just thought of, would be to (re)wrap a paragraph only if its longest line exceeds 79 characters. That should cause text quoted from non-compliant clients to remain readable for at least a few quote levels.

[Llammissar]

the quoted content might be code with deliberate word breaks.

Three observations I have here:

1. If it runs over 66 characters it already gets a forced line break anyway. Often, this is a comment, and the next line is back under the limit. This is part of the reasoning behind my third proposal.
2. Most example code I can find has lines shorter than 66 characters. This is anecdotal from looking over D and D.learn posts made in the past week or so, but most things that make it to the NG are compact, trivial examples; for longer stuff, people use a pastebin (http://dpaste.dzfl.pl) or gist link with highlighting.
3. A lot of the example code I'm seeing in NG posts has a lot of vertical whitespace, so paragraph separations (per the RFC definition) are frequent. Most code I've seen will be completely unaffected if the sequence-of-lines heuristic is used.

I accept that it would be good to have preview functionality.

This would be very helpful regardless; thanks.

which I just thought of, would be to (re)wrap a paragraph only if its longest line exceeds 79 characters. That should cause text quoted from non-compliant clients to remain readable for at least a few quote levels.

That will work for a subset (I think it's even a small subset, but I don't have any data to back that), but not for mail from non-compliant clients that is nevertheless formatted for 80-column displays. I considered this same thing before proposing the "sequences of [at least two] lines [where each line is] longer than 66 characters". (I realise now the bracketed parts may not have been implicit.)

[CyberShadow]

OK, so could you formulate a proposed algorithm following the discussion above and some examples/tests for how it should behave?

[Llammissar]

@CyberShadow Just to drop a note, I've got a patch and I think I have it mostly working. I've been kinda busy lately, but I'm hoping to get it finished and unit-tested tonight.

[CyberShadow]

Continued in #26