Telnet and carriage return.

[Arnaud Rébillout]

Hello,

I'm wondering if there is a way to issue a carriage return with telnet.

Here is the story:
This morning I used telnet to send a mail. It's quite easy to do that,
just telnet to the right SMTP server, the procedure is quite
straight-forward.

The problem is that the SMTP protocol expects each line to be terminated
by a carriage return (CR, \r in C) followed with a line feed (LF, \n in
C). And obviously, when I hit enter in Telnet, the line issued to the
server is terminated by a simple \n.

To fix that, I simply patched telnet, and replace each \n by \r\n. It
works, but it's an ugly solution.

That's why I'm wondering for a better solution to do that. Simply, isn't
there a way to issue a carriage return character ?

Thanks

[Rich Felker]

Post by Arnaud Rébillout
Hello,
I'm wondering if there is a way to issue a carriage return with telnet.
This morning I used telnet to send a mail. It's quite easy to do
that, just telnet to the right SMTP server, the procedure is quite
straight-forward.
The problem is that the SMTP protocol expects each line to be
terminated by a carriage return (CR, \r in C) followed with a line
feed (LF, \n in C). And obviously, when I hit enter in Telnet, the
line issued to the server is terminated by a simple \n.
To fix that, I simply patched telnet, and replace each \n by \r\n.
It works, but it's an ugly solution.
That's why I'm wondering for a better solution to do that. Simply,
isn't there a way to issue a carriage return character ?

There are lots of translation modes available at the terminal level,
controllable with the stty command. There may be one to enable
translation of carriage return characters (the enter key) to CR+LF
combinations rather than just NL (LF). Otherwise, you could disable
the translation entirely and just press enter followed by ^J at the
end of each line...

Note that this only matters for interactive use, which is hopefully a
rare thing. Also, any SMTP server I've ever used accepts NL-only (with
no CR) input, so for interactive use rather than script use (wherein
the latter you would want to be fully robust against strict servers),
it probably doesn't matter anyway...

Another solution would be to use sed 's/$/^M/' | telnet ... (where ^M
is a literal carriage return in the command line) so that telnet is
reading from a pipe rather than a terminal, and sed is adding CR's for
you...

Rich

[Arnaud Rébillout]

Hi,
thanks a lot for your quick (and useful) answer Rich.

Post by Rich Felker
There are lots of translation modes available at the terminal level,
controllable with the stty command. There may be one to enable
translation of carriage return characters (the enter key) to CR+LF
combinations rather than just NL (LF). Otherwise, you could disable
the translation entirely and just press enter followed by ^J at the
end of each line...

Sounds good, I never thought about it. I think changing such a low-level
behavior just for a specific telnet usage is a little bit overkill. But
it would do the job.

Post by Rich Felker
Note that this only matters for interactive use, which is hopefully a
rare thing.

You're right, I forgot to tell that all of this happens in interactive
usage of telnet. It's just a convenient way of trying things with a SMTP
server, and understand a little bit how it works.

Post by Rich Felker
Also, any SMTP server I've ever used accepts NL-only (with
no CR) input, so for interactive use rather than script use (wherein
the latter you would want to be fully robust against strict servers),
it probably doesn't matter anyway...

I'm experimenting with a gmail server at the moment. It works well until
I enter the DATA step. At this moment, the server fails to recognize the
termination line (ie. the line containing a single dot), and I get stuck
there with no way out. I figured out it's because of the missing CR
character.

Post by Rich Felker
Another solution would be to use sed 's/$/^M/' | telnet ... (where ^M
is a literal carriage return in the command line) so that telnet is
reading from a pipe rather than a terminal, and sed is adding CR's for
you...

Yep ! That's great, exactly what I was looking for. I tried this
solution, and I had a little bit more trouble than expected, but it
worked in the end.

The first thing is that, for interactive use, you don't want sed to
buffer the data. And this feature is not implemented in busybox sed, so
you need another implementation of sed.

The second thing is that there is this nasty piece of code in busybox
telnet.c source:

else if (c == '\r')
outbuf[j++] = '\0'; /* CR -> CR NUL */

Hell ! For some reason, every CR has a NUL appended to it. So I had to
remove that, and then everything works fine. Any idea about the reason
for this stuff ? I read the comment above but it doesn't make sense to
me...

Anyway, in the end the right command is:
$ sed --unbuffered 's/$/\r/' | telnet ...


Arnaud

[Ralf Friedl]

Post by Arnaud Rébillout

Post by Rich Felker
Also, any SMTP server I've ever used accepts NL-only (with
no CR) input, so for interactive use rather than script use (wherein
the latter you would want to be fully robust against strict servers),
it probably doesn't matter anyway...

I'm experimenting with a gmail server at the moment. It works well
until I enter the DATA step. At this moment, the server fails to
recognize the termination line (ie. the line containing a single dot),
and I get stuck there with no way out. I figured out it's because of
the missing CR character.

If that is the only place where you need the CR, you can just type it
manually: Type Ctrl-V Ctrl-M, and '^M' should appear.

Post by Arnaud Rébillout
The second thing is that there is this nasty piece of code in busybox
else if (c == '\r')
outbuf[j++] = '\0'; /* CR -> CR NUL */
Hell ! For some reason, every CR has a NUL appended to it. So I had to
remove that, and then everything works fine. Any idea about the reason
for this stuff ? I read the comment above but it doesn't make sense to
me...

That seems to be normal, the standard telnet client does the same. But
one difference is that the standard telnet client sends CR-LF at the end
of the line and not LF alone.

[Denys Vlasenko]

Post by Arnaud Rébillout
Hi,
thanks a lot for your quick (and useful) answer Rich.

Post by Rich Felker
There are lots of translation modes available at the terminal level,
controllable with the stty command. There may be one to enable
translation of carriage return characters (the enter key) to CR+LF
combinations rather than just NL (LF). Otherwise, you could disable
the translation entirely and just press enter followed by ^J at the
end of each line...

Sounds good, I never thought about it. I think changing such a low-level
behavior just for a specific telnet usage is a little bit overkill. But it
would do the job.

Post by Rich Felker
Note that this only matters for interactive use, which is hopefully a
rare thing.

You're right, I forgot to tell that all of this happens in interactive usage
of telnet. It's just a convenient way of trying things with a SMTP server,
and understand a little bit how it works.

Post by Rich Felker
Also, any SMTP server I've ever used accepts NL-only (with
no CR) input, so for interactive use rather than script use (wherein
the latter you would want to be fully robust against strict servers),
it probably doesn't matter anyway...

I'm experimenting with a gmail server at the moment. It works well until I
enter the DATA step. At this moment, the server fails to recognize the
termination line (ie. the line containing a single dot), and I get stuck
there with no way out. I figured out it's because of the missing CR
character.

Post by Rich Felker
Another solution would be to use sed 's/$/^M/' | telnet ... (where ^M
is a literal carriage return in the command line) so that telnet is
reading from a pipe rather than a terminal, and sed is adding CR's for
you...

Yep ! That's great, exactly what I was looking for. I tried this solution,
and I had a little bit more trouble than expected, but it worked in the end.
The first thing is that, for interactive use, you don't want sed to buffer
the data. And this feature is not implemented in busybox sed, so you need
another implementation of sed.
The second thing is that there is this nasty piece of code in busybox
else if (c == '\r')
outbuf[j++] = '\0'; /* CR -> CR NUL */
Hell ! For some reason, every CR has a NUL appended to it. So I had to
remove that, and then everything works fine. Any idea about the reason for
this stuff ? I read the comment above but it doesn't make sense to me...

I have no idea yet. Here is when it was added:

commit 0e28e1fa0551487dd28a42f1dbeb5bf717817175
Author: Eric Andersen <andersen at codepoet.org>
Date: Fri Apr 26 07:20:47 2002 +0000

Forward port patch from Przemyslaw Czerpak <druzus at polbox.com>:

1. busybox-telnet dosn't inform server about the size of terminal screen.
In the world of xterminals and frame buffers it's rather horrible
to use fixed 80x24 region in upper-left corner of screen/window.

2. If client sends character 0x0d to the server then sends character 0x0a
the server eat the second byte (0x0a) - it's described in telnet RFC.
Client should send two bytes ( 0x0d + 0x0a or 0x0d + 0x00 ) insted of
one 0x0d byte.

3. busybox telnet implementation wasn't 8bit clean (look at 0xff byte).
I need it because I have to use binray transfer like rz/sz. So when
I resloved the problem (2) I corrected this one two.

This also contains a small cleanup patch from vodz, and some minor editing
by me.

+ int i, j;
byte * p = G.buf;
+ byte outbuf[4*DATABUFSIZE];

- for (i = len; i > 0; i--, p++)
+ for (i = len, j = 0; i > 0; i--, p++)
{
if (*p == 0x1d)
{
conescape();
return;
}
+ outbuf[j++] = *p;
if (*p == 0xff)
- *p = 0x7f;
+ outbuf[j++] = 0xff;
+ else if (*p == 0x0d)
+ outbuf[j++] = 0x00;
}
- write(G.netfd, G.buf, len);
+ if (j > 0 )
+ write(G.netfd, outbuf, j);

[Denys Vlasenko]

On Thu, Sep 13, 2012 at 12:37 PM, Denys Vlasenko

Post by Denys Vlasenko

Post by Arnaud Rébillout
The second thing is that there is this nasty piece of code in busybox
else if (c == '\r')
outbuf[j++] = '\0'; /* CR -> CR NUL */
Hell ! For some reason, every CR has a NUL appended to it. So I had to
remove that, and then everything works fine. Any idea about the reason for
this stuff ? I read the comment above but it doesn't make sense to me...

commit 0e28e1fa0551487dd28a42f1dbeb5bf717817175
Author: Eric Andersen <andersen at codepoet.org>
Date: Fri Apr 26 07:20:47 2002 +0000
1. busybox-telnet dosn't inform server about the size of terminal screen.
In the world of xterminals and frame buffers it's rather horrible
to use fixed 80x24 region in upper-left corner of screen/window.
2. If client sends character 0x0d to the server then sends character 0x0a
the server eat the second byte (0x0a) - it's described in telnet RFC.
Client should send two bytes ( 0x0d + 0x0a or 0x0d + 0x00 ) insted of
one 0x0d byte.
3. busybox telnet implementation wasn't 8bit clean (look at 0xff byte).
I need it because I have to use binray transfer like rz/sz. So when
I resloved the problem (2) I corrected this one two.
This also contains a small cleanup patch from vodz, and some minor editing
by me.
+ int i, j;
byte * p = G.buf;
+ byte outbuf[4*DATABUFSIZE];
- for (i = len; i > 0; i--, p++)
+ for (i = len, j = 0; i > 0; i--, p++)
{
if (*p == 0x1d)
{
conescape();
return;
}
+ outbuf[j++] = *p;
if (*p == 0xff)
- *p = 0x7f;
+ outbuf[j++] = 0xff;
+ else if (*p == 0x0d)
+ outbuf[j++] = 0x00;
}
- write(G.netfd, G.buf, len);
+ if (j > 0 )
+ write(G.netfd, outbuf, j);

Changed it to this in git:

else if (c == '\r')
- outbuf[j++] = '\0'; /* CR -> CR NUL */
+ /* See RFC 1123 3.3.1 Telnet End-of-Line Convention.
+ * Using CR LF instead of other allowed possibilities
+ * like CR NUL - easier to talk to HTTP/SMTP servers.
+ */
+ outbuf[j++] = '\n'; /* CR -> CR LF */

[Ralf Friedl]

Post by Arnaud Rébillout
else if (c == '\r')
- outbuf[j++] = '\0'; /* CR -> CR NUL */
+ /* See RFC 1123 3.3.1 Telnet End-of-Line Convention.
+ * Using CR LF instead of other allowed possibilities
+ * like CR NUL - easier to talk to HTTP/SMTP servers.
+ */
+ outbuf[j++] = '\n'; /* CR -> CR LF */

This sends a CR-LF End-of-Line, but as a response to Ctrl-M. In line
mode the telnet client reads Ctrl-J or LF when the user presses enter.
The right thing to do would be to revert this change and to add
+ else if (c == '\n' && G.charmode != CHM_ON) {
+ outbuf[j] = '\r'; /* LF -> CR LF */
+ outbuf[j++] = '\n';
+ }

[Denys Vlasenko]

Post by Ralf Friedl

Post by Arnaud Rébillout
else if (c == '\r')
- outbuf[j++] = '\0'; /* CR -> CR NUL */
+ /* See RFC 1123 3.3.1 Telnet End-of-Line Convention.
+ * Using CR LF instead of other allowed possibilities
+ * like CR NUL - easier to talk to HTTP/SMTP servers.
+ */
+ outbuf[j++] = '\n'; /* CR -> CR LF */

This sends a CR-LF End-of-Line, but as a response to Ctrl-M.

Is that wrong or what? I don't understand what you want to say.

Post by Ralf Friedl
In line mode
the telnet client reads Ctrl-J or LF when the user presses enter.

I am not even familiar with this line mode thingy.
Let me experiment... ok, so in line more we see '\n'
from keyboard, not '\r'.

Post by Ralf Friedl
The right thing to do would be to revert this change and to add
+ else if (c == '\n' && G.charmode != CHM_ON) {
+ outbuf[j] = '\r'; /* LF -> CR LF */
+ outbuf[j++] = '\n';
+ }

Probably...

[Ralf Friedl]

Post by Denys Vlasenko

Post by Ralf Friedl

Post by Arnaud Rébillout
else if (c == '\r')
- outbuf[j++] = '\0'; /* CR -> CR NUL */
+ /* See RFC 1123 3.3.1 Telnet End-of-Line Convention.
+ * Using CR LF instead of other allowed possibilities
+ * like CR NUL - easier to talk to HTTP/SMTP servers.
+ */
+ outbuf[j++] = '\n'; /* CR -> CR LF */

This sends a CR-LF End-of-Line, but as a response to Ctrl-M.

Is that wrong or what? I don't understand what you want to say.

This change converts CR to CR-LF. But to type CR you have to press
Ctrl-V Ctrl-M, not the enter key. The enter key is read as LF or '\n'.
This the "end-of-line" key or "Return" or "Enter" from RFC1123 section
3.3.1. According to this section, "end-of-line" or LF should be sent by
default to a telnet server and what must be sent to a non telnet server.

I looked at the behavior of the standard telnet client when talking to
an SMTP server (and probably to any non telnet server) and it is this:
CR -> CR NUL
LF -> CR LF

So a single CR is converted to CR NUL to show the server that CR or '\r'
or Ctrl-M was pressed.
A single LF is converted to CR LF to show the server that LF or '\n' or
Ctrl-J was pressed.

Post by Denys Vlasenko

Post by Ralf Friedl
In line mode
the telnet client reads Ctrl-J or LF when the user presses enter.

I am not even familiar with this line mode thingy.
Let me experiment... ok, so in line more we see '\n'
from keyboard, not '\r'.

I think busybox telnet doesn't really implement the telnet linemode, but
what is called 'old line by line' in the telnet manual. Just read a line
terminated by '\n' from the keyboard and sent it to the server. But this
mode is what I meant when I wrote line mode.

Post by Denys Vlasenko

Post by Ralf Friedl
The right thing to do would be to revert this change and to add
+ else if (c == '\n' && G.charmode != CHM_ON) {
+ outbuf[j] = '\r'; /* LF -> CR LF */
+ outbuf[j++] = '\n';
+ }

Probably...

[Denys Vlasenko]

Post by Denys Vlasenko

Post by Ralf Friedl
This sends a CR-LF End-of-Line, but as a response to Ctrl-M.

Is that wrong or what? I don't understand what you want to say.

This change converts CR to CR-LF. But to type CR you have to press Ctrl-V
Ctrl-M, not the enter key. The enter key is read as LF or '\n'.

No, in raw tty mode Enter key sends ^M = '\r' = CR.

This the "end-of-line" key or "Return" or "Enter" from RFC1123 section 3.3.1.

Parse error.
Anyway, practical considerations demand that keyboard Enter key
should be usable as end-of-line for telnet users.

According to this section, "end-of-line" or LF should be sent by default to
a telnet server and what must be sent to a non telnet server.

Really?

A User Telnet MUST be able to send any of the forms: CR LF, CR
NUL, and LF. A User Telnet on an ASCII host SHOULD have a
user-controllable mode to send either CR LF or CR NUL when the
user presses the "end-of-line" key, and CR LF SHOULD be the default.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

When a user presses the "end-of-line" key, some User
Telnet implementations send CR LF, while others send CR
NUL (based on a different interpretation of the same
sentence in RFC-854). These will be equivalent for a
correctly-implemented ASCII server host, as discussed
above. For other servers, a mode in the User Telnet is
needed.

The existence of User Telnets that send only CR NUL when
CR is pressed creates a dilemma for non-ASCII hosts: they
can either treat CR NUL as equivalent to CR LF in input,
thus precluding the possibility of entering a "bare" CR,
or else lose complete interworking.

I looked at the behavior of the standard telnet client when talking to an
CR -> CR NUL

Well, smtp isn't going to understand what this means.

LF -> CR LF
So a single CR is converted to CR NUL to show the server that CR or '\r' or
Ctrl-M was pressed.
A single LF is converted to CR LF to show the server that LF or '\n' or
Ctrl-J was pressed.

What are "single CR" and "single LF" in terms of
what keys user presses on the keyboard?

To me it looks like we should ensure that Enter key
sends something which is understood as end-of-line.

Since Enter produces CR _or_ LF (depending on mode),
how about we just always replace both of them with
CR+LF pair?

[Cathey, Jim]

Post by Rich Felker
Another solution would be to use sed 's/$/^M/' | telnet ...

you don't want sed to buffer the data...not implemented in busybox sed

The canonical cr<->lf translation is to use tr, not sed.

-- Jim