Reading data from socket and send it again

[amuessig]

Hey,

I am trying to build up a middlebox device which is reading packets from a socket, check some stuff like IP address and (if the check matches) send the packet out through the socket.
Later, I will be migrating the socket, thus I need to use raw sockets.
My code is as follow:

init({Parent, Port}) ->
    register(?MODULE,self()),
    proc_lib:init_ack(Parent, {ok, self()}),
    {ok, ListenSock} = procket:open(Port, [{protocol, 16#0008}, {type, raw}, {family, packet}]),
    P = erlang:open_port({fd, ListenSock, ListenSock}, [binary, stream]),
    accept(P).

How can I bind the opened socket to a interface?
I tried this:
{ok, ListenSock} = procket:open(Port, [{protocol, 16#0008}, {type, raw}, {family, packet}, {interface, "vnf-eth0"}]),
but it didn't work.

Furthermore, trying the listening method, I am not sure how to write back the data..:

accept(ListenSock) ->
    case gen_tcp:accept(ListenSock) of
        {ok, Client} ->
            io:format("foo~n"),
            Tmp = rule_database:lookup_rule_table(Client),
            if Tmp ->
                %if the lookup is positive, send it back
                {ok, _} = procket:sendTo(ListenSock);
            true ->
                % otherwise don't send it
                gen_tcp:close(Client)
                %erlang:port_close(Client)
            end,
            accept(ListenSock);
        _Error ->
            ok
    end.

Any suggestions/help? Thanks!

[msantos]

On Thu, Jan 21, 2016 at 09:37:21AM -0800, amuessig wrote:

Hey,

I am trying to build up a middlebox device which is reading packets from a socket, check some stuff like IP address and (if the check matches) send the packet out through the socket
Later, I will be migrating the socket, thus I need to use raw sockets
My code is as follow:

init({Parent, Port}) ->
    register(?MODULE,self()),
    proc_lib:init_ack(Parent, {ok, self()}),
    {ok, ListenSock} = procket:open(Port, [{protocol, 16#0008}, {type, raw}, {family, packet}]),
    P = erlang:open_port({fd, ListenSock, ListenSock}, [binary, stream]),
    accept(P)

How can I bind the opened socket to a interface?
I tried this:
{ok, ListenSock} = procket:open(Port, [{protocol, 16#0008}, {type, raw}, {family, packet}, {interface, "vnf-eth0"}]),
but it didn't work

procket calls setsockopt(SO_BINDTODEVICE) for the interface option.
According to socket(7):

SO_BINDTODEVICE

    Note that this works only for  some socket types, particularly
    AF_INET sockets.  It is not supported for packet sockets (use normal
    bind(2) there).

So we need to use bind(2). There's a version of bind() in the procket
PF_PACKET module that works with packet sockets:

{ok, ListenSock} = procket:open(Port, [{protocol, 16#0008}, {type, raw}, {family, packet}]),
IfIndex = packet:ifindex(ListenSock, "vnf-eth0"),
ok = packet:bind(ListenSock, IfIndex)

Furthermore, trying the listening method, I am not sure how to write back the data:
accept(ListenSock) ->
case gen_tcp:accept(ListenSock) of
{ok, Client} ->
io:format("foo~n"),
Tmp = rule_database:lookup_rule_table(Client),
if Tmp ->
%if the lookup is positive, send it back
{ok, _} = procket:sendTo(ListenSock);
true ->
% otherwise don't send it
gen_tcp:close(Client)
%erlang:port_close(Client)
end,
accept(ListenSock);
_Error ->
ok
end

Any suggestions/help? Thanks!

Because we're using PF_PACKET sockets, we can't call gen_tcp:accept/1. The
ethernet frame will be sent to our process as port data. The frame can
be parsed using pkt (https://github.com/msantos/pkt).

For sending the frame, use packet:send/3.

Something like this:

accept(ListenSock, IfIndex, Port) ->
    receive
        {Port, {data, Data}} ->
             Frame = pkt:decapsulate(Data),
             Tmp = rule_database:lookup_rule_table(Frame),
             if Tmp ->
                 %if the lookup is positive, send it back
                 % XXX will return {ok, Size} on partial write
                 ok = packet:send(ListenSock, Ifindex, Data);
             true ->
                 % otherwise don't send it

                 % Could generate a RST here
                 %gen_tcp:close(Client)
                 %erlang:port_close(Client)
             end,
             accept(ListenSocket, IfIndex, Port);
        _Error ->
            ok
    end.

If the device is in the middle between the client and server, instead of
closing the socket you could generate a RST in both directions to shut down
the connection.

If any of that isn't clear or if you have any questions, please let me
know!

[amuessig]

Thanks for your helpful hints!
Sending RST will be implemented later on. First, there should be a working prototype.
I don't know why, but I don't get any data send back, I think they are not even received.
This is the code I have figured out for now:

init({Parent, Port}) ->
register(?MODULE,self()),
proc_lib:init_ack(Parent, {ok, self()}),
{ok, ListenSock} = procket:open(Port, [{protocol, 16#0008}, {type, raw}, {family, packet}]),
IfIndex = packet:ifindex(ListenSock, "vnf-eth0"),
ok = packet:bind(ListenSock, IfIndex),
accept(ListenSock, IfIndex, Port).

accept(ListenSock, IfIndex, Port) ->
    io:fwrite("Method accept is called\n"),
    receive
    {Port, {data, Data}} -> 
        Frame = pkt:decapsulate(Data),
        io:fwrite("Frame:", Frame, "\n"),
        Tmp = rule_database:lookup_rule_table(Frame),
        if Tmp ->
            {ok, Size} = packet:send(ListenSock, IfIndex, Data);
        true ->
            erlang:port_close(IfIndex)
        end,
        accept(ListenSock, IfIndex, Port);
    _Error ->
        ok
end.

I am getting the output "Method accept is called", but never "Frame: [...]".
Did I misunderstand something?
I was sending pings to the vm, which was working.
When calling the server by opening a TCP connection to download data, these data are not put back again. I do not even get the message that something was received.

Thanks!

EDIT: Yes, the interface is run in promisc. mode.

[msantos]

Here's a working example:

-module(resend).

-export([init/1]).

init(Dev) ->
    {ok, ListenSock} = procket:open(0, [{protocol, 16#0008}, {type, raw}, {family, packet}]),
    IfIndex = packet:ifindex(ListenSock, Dev),
    ok = packet:bind(ListenSock, IfIndex),
    Port = erlang:open_port({fd, ListenSock, ListenSock}, [binary,stream]),
    accept(ListenSock, IfIndex, Port).

accept(ListenSock, IfIndex, Port) ->
    io:fwrite("Method accept is called\n"),
    receive
        {Port, {data, Data}} ->
            Frame = pkt:decapsulate(Data),
            error_logger:info_report(Frame),
            ok = packet:send(ListenSock, IfIndex, Data),
            accept(ListenSock, IfIndex, Port);
        _Error ->
            ok
    end.

A few things to note:

• erlang uses the term "port" for both TCP/UDP ports and the mechanism for external I/O

  In this case, a PF_PACKET socket does not have a IP port, so we can set it to 0 in procket/open/2.

  The other port is an Erlang port. We use a port to poll our socket file descriptor using the erlang event loop.

• packet:send/3 will return 'ok' on a complete write. On a partial write, it'll return the amount written ({ok, Size})

• depending on what you're doing, you may to re-write the frame, for example, substituting the MAC address of the system for the sender

Hope that clears things up a bit, feel free to post here if you run into any other problems.

[amuessig]

Thanks a lot, works perfectly.
When sending a RST, do I have to build my frame manually or is your framework offering this? I only found RST for TCP/UDP sockets.

Appreciating your support! 👍

[msantos]

There's an old example here: https://gist.github.com/msantos/446057#file-rst-erl
And some explanation: http://blog.listincomprehension.com/2010/06/fun-with-raw-sockets-in-erlang-abusing.html

You'll probably need to swap the MAC address of your MITM host as the source MAC address when sending the RSTs.

[amuessig]

It's working very well.

however, I couldn't find anything how to forward frames/packets/segments to the applications of the host where my Erlang is running.
In example, when I am using a distributed Erlang, I want to be able to set it up from remote. Currently I only send out the frames or I drop them. But this also drops the packets for starting erlang processes remotely.

[msantos]

It's working very well.

Great!

however, I couldn't find anything how to forward frames/packets/segments to the applications of the
host where my Erlang is running.

In example, when I am using a distributed Erlang, I want to be able to set it up from remote. Currently I only send out the frames or I drop them. But this also drops the packets for starting erlang processes remotely.

Probably the simplest way is to match any packets with the source or destination set to the IP address of the MITM host and ignore them. Maybe something like this:

https://github.com/msantos/herp/blob/master/src/herp.erl#L127

[amuessig]

Yeah, it worked perfectly.
Do you have any idea how to send ARP requests?
In your packet file, ( https://github.com/msantos/procket/blob/master/src/packet.erl ), you show up a way for looking up the mac address in the ARP table of the host, but I cannot find any way to get the MAC address if it is not at the host's 'ARP' cache.
Any idea of that or do I have to send ARP packets on my own?

[msantos]

Sure, it is pretty simple. Construct an ARP packet then write it to a PF_PACKET or BPF socket:
https://github.com/msantos/farp/blob/master/src/farp.erl#L192

To populate the ARP cache, open any network connection to the host. For example, send a UDP packet to some random port on the host.

If you want more control, you could do an ARP probe or gratuitous ARP using the target host's IP address then sniff the reply from the host.