2

I've seen a lot of things about redirecting stdout to a TCP socket, but no real example of how to do it in practice, specifically when the output stream generated by the first "command" never ends.

To talk about something concrete, let's take programs like servers that typically output their log endlessly to stdout (well, as long as they run). If you redirect the output to a log file on the disk, then this file is always open (therefore not readable by others?) and grows infinitely, which eventually is going to cause problems.

This might be a nood question, but I don't know what it does or how to do it so.

  1. How to redirect the output of a command to the internal loop?
  2. I want to make sure that data is sent EVERY time something is written to stdout, and that the pipe won't wait for the command to end (never happens ideally!). Is that right?
  3. If 2 is true, is there a buffer system to send chunks of data once it reaches a certain size only?
  4. Could you give me concrete command line examples to do the above?

Thanks in advance

Jonathan H
  • 687
  • 1
    Have you had a look at nc (netcat)? If I understand your question correctly, I think you can accomplish it by doing yes | nc 123.45.56.78 1234 where yes generates the output continuously, and the nc parameters are the IP address and port as the target. Please refer to the manpage of nc. – gertvdijk Jun 28 '13 at 09:11
  • @gertvdijk I never used it, but I saw references to netcat elsewhere. I just want to make sure that it verifies point 2; it doesn't wait for yes to terminate to send the data stream, right? – Jonathan H Jun 28 '13 at 09:15
  • @gertvdijk Please post that into an answer. nc is the way to go – ignis Jun 28 '13 at 09:31
  • Writing an answer. – gertvdijk Jun 28 '13 at 09:32
  • 1
    For completeness: In general, you can find solutions to question 3 here via unbuffer, stdbuf, etc. However, they are not needed if you use nc. – ignis Jun 28 '13 at 09:33
  • @ignis Thank you :) It was a plus anyway, the important thing is that I understand the basics right now. I'll wait for the answer of gertvdijk. – Jonathan H Jun 28 '13 at 09:41
  • An alternative to nc is explained here (with Bash native syntax) – ignis Jun 28 '13 at 09:41

1 Answers1

4

Use netcat

  1. Install netcat-traditional Install netcat-traditional or netcat-openbsd Install netcat-openbsd .

  2. On the receiver end, open up a listening socket:

    nc -l 1234

    where 1234 is the port number to listen on. By default netcat uses a TCP socket. This will run continuously until it receives an EOF. If you need to turn that off, add the option -q -1 (negative wait timeout - see manpage).

    Optionally, redirect the output to a file or pipe it, e.g.:

    nc -l 1234 > mylogfile.log

  3. On the sender end, fire up the command and pipe the stdout to netcat:

    mycommand | nc 123.45.67.89 1234

    where 123.45.67.89 is the IP address of the listening end and 1234 is the port number again.

    This will run continuously (streaming the data) until the mycommand outputs an EOF.

Discussion

How to redirect the output of a command to the internal loop?

I'm not sure what you mean by the "internal loop", but redirecting output is a very general question for your shell.

I want to make sure that data is sent EVERY time something is written to stdout, and that the pipe won't wait for the command to end (never happens ideally!). Is that right?

Pipes in the shell won't wait for an EOF to pass the data, as far as I know.

If 2 is true, is there a buffer system to send chunks of data once it reaches a certain size only?

Yes, the general buffers in the pipes as described in the suggestion of ignis in his comment: Question on UL: Turn off buffering in pipe.

TCP will handle buffers on the TCP level.

Community
  • 1
gertvdijk
  • 69,607
  • Thanks a lot for this detailed post, this goes beyond what I expected, and I understand perfectly the solution :) Thanks again! – Jonathan H Jun 28 '13 at 12:40
  • I've been researching something like this. My situation was requiring the ability of having a log capturing daemon. This was a process that listened on a TCP port for logs. Then I wanted to run a command and redirect the STDOUT and STDERR to the TCP port rather than saving it to a log file. This allows me to centralise my logs, and then send it to some third party for further processing. Would net cat help with that? Or can I use some native bash to achieve my goals? – CMCDragonkai Jan 11 '14 at 15:17
  • @CMCDragonkai Ask a NEW question please. :) – gertvdijk Jan 11 '14 at 18:25