A way to handle both websocket and regular HTTP requests

[zoggy]

Hello,

I'd like to build a server able to act as a regular HTTP server but also as a websocket server, depending on the path of the initial HTTP request. For example, querying http://myserver/ws will make a webserver connection, while querying any other path will make a regular HTTP connection (retrieving pages etc.).
I could not find a way to do so, the only function being available is establish_server. Could there be either a function to switch from a cohttp connection to a websocket connection, or either a parameter to establish_server allowing to stay in regular HTTP on a given condition ?

[vbmithr]

I'll think about it ASAP.

[vbmithr]

I think we could do something like this: https://github.com/mirleft/ocaml-tls/blob/master/lwt/tls_lwt.mli

When I wrote this code it was more with a client use-case in mind. But I agree that the server functionality is not very flexible as-is. I'll try to mimic the interface of Tls_lwt, soon.

[zoggy]

Thanks. No urgency ;)

[zoggy]

Any news about this ?

[vbmithr]

On 09/10/2014 16:10, Zoggy wrote:

Any news about this ?

I started rewriting a websocket "decoder". I'll try to finish it asap.
Vincent

[zoggy]

ping ! :)

[vbmithr]

On 04/12/2014 16:53, Zoggy wrote:

ping ! :)

Now that I did ocaml-scid, I master the non-blocking streaming technique
:) Stay tuned!

Vincent

[avsm]

I could use this as well -- happy to put in patches to Cohttp to support Upgrade handoff if it helps.

[vbmithr]

On 05/02/2015 11:54, Anil Madhavapeddy wrote:

I could use this as well -- happy to put in patches to Cohttp to support
|Upgrade| handoff if it helps.

Yeah, I definitely need to do this but fails to find the time for it.
It's still in my focus though!!

Vincent

[lostman]

Here's my attempt at upgrade from cohttp:

val upgrade_connection :
    Cohttp.Request.t ->
    Conduit_lwt_unix.flow * Cohttp.Connection.t ->
    (Frame.t option -> unit) ->
    (Cohttp.Response.t * Cohttp.Body.t * (Frame.t option -> unit)) Lwt.t

let upgrade_connection request (conn, conn_id) frames_in_fn =
    let resp =
        let headers = C.Request.headers request in
        let key = Opt.run_exc @@ C.Header.get headers "sec-websocket-key" in
        let hash = key ^ websocket_uuid |> b64_encoded_sha1sum in
        let response_headers =
            C.Header.of_list
                ["Upgrade", "websocket";
                 "Connection", "Upgrade";
                 "Sec-WebSocket-Accept", hash]
        in
        Cohttp_lwt_unix.Response.make
            ~status:`Switching_protocols
            ~encoding:Cohttp.Transfer.Unknown
            ~headers:response_headers
            ~flush:true
            ()
    in

    let frames_out_stream, frames_out_fn = Lwt_stream.create () in

    Lwt.async (
        fun () ->
            let open Conduit_lwt_unix in
            match conn with
                | TCP (tcp : tcp_flow) ->
                    let ic = Lwt_io.make ~mode:Lwt_io.input (Lwt_bytes.read tcp.fd) in
                    let oc = Lwt_io.make ~mode:Lwt_io.output (Lwt_bytes.write tcp.fd) in
                    Lwt.join [
                        (* data in *)
                        read_frames (ic,oc) frames_in_fn frames_out_fn;
                        (* data out *)
                        send_frames ~masked:false frames_out_stream (ic,oc)
                    ]
                | _ -> Lwt.fail_with "expected a TCP Websocket connection"
    );
    Lwt.return (resp, Cohttp.Body.empty, frames_out_fn)

And upgrading a /ws route:

  | "/ws" ->
    let print_frames = function
        | None -> Printf.printf "None\n%!"
        | Some f ->
            match Websocket.Frame.content f with
            | None -> Printf.printf "None\n%!"
            | Some c -> Printf.printf "received: %s\n%!" c
    in
    Websocket.upgrade_connection req (ch, conn) print_frames
    >>= fun (resp, body, frames_out_fn) ->
    (* send a text frame back to the client every 5 seconds *)
    let _ =
        let rec go n =
            Lwt_unix.sleep 5.0 >>= fun () ->
            Lwt.wrap1
                frames_out_fn
                (Some (Websocket.Frame.of_string
                        ~content:(Printf.sprintf "Ping! %d" n) ())) >>= fun () ->
            if n > 0 then go (n-1) else Lwt.return_unit
        in
        go 1000
    in
    Lwt.return (resp, (body :> Cohttp_lwt_body.t))

Sadly, it doesn't work.

Connection gets established but every 2nd frame I send to the server is somehow lost and after a while the connection just drops. Messages sent from the server to the client seem to fare a little better -- all of them get delivered.

Any ideas why this fails?

[vbmithr]

On 13/02/2015 03:51, Maciej Woś wrote:

Any ideas why this fails?

I'll have a look perhaps. Do you answer pings ?

Vincent

[lostman]

I thought I don't have to do it explicitly. It looks like read_frames is taking care of that:

push_to_remote (Some (Frame.of_bytes ~opcode:Opcode.Pong ~extension ~final ~content ()));

[vbmithr]

On 13/02/2015 10:42, Maciej Woś wrote:

I thought I don't have to do it explicitly. It looks like |read_frames|
is taking care of that:

push_to_remote (Some (Frame.of_bytes ~opcode:Opcode.Pong ~extension ~final ~content ()));

Ah, right, yes.

Vincent

[lostman]

It seems something continues to read from the fd even after reading the whole (empty) request body. I'm not sure where the remaining data goes.

When I start another Lwt thread that reads from the same fd half of the frames go to first reader and the other half to the second.

I came up with this hacky solution:

let open Conduit_lwt_unix in
match conn with
    | TCP (tcp : tcp_flow) ->
        let dup = Lwt_unix.dup tcp.fd in
        Lwt_unix.close tcp.fd >>= fun () ->
        Cohttp_lwt_unix.Server.Response.write_header
            resp
            (Lwt_io.of_fd ~mode:Lwt_io.output dup)
        >>= fun () ->
        Lwt.join [
            (* data in *)
            read_frames
                (Lwt_io.of_fd ~mode:Lwt_io.input dup)
                frames_in_fn
                frames_out_fn;
            (* data out *)
            send_frames
                ~masked:false (* server never masks the frames *)
                frames_out_stream
                (Lwt_io.of_fd ~mode:Lwt_io.output dup)
        ]
    | _ -> Lwt.fail_with "expected a TCP Websocket connection"

For what it's worth, it seems to work. I can establish the connection and send/receive frames.

Note: I've changed my local version of read_frames to only take the input_channel and send_frames to only take the output_channel.

Maybe Cohttp should stop reading after getting the request body?

[avsm]

Cohttp keeps reading in order to look for the next pipelined request. It could possibly stop reading if there were a Connection: close in the request.

On 18 Feb 2015, at 09:03, Maciej Woś [email protected] wrote:

It seems something continues to read from the fd even after reading the whole (empty) request body. I'm not sure where the remaining data goes.

When I start another Lwt thread that reads from the same fd half of the frames go to first reader and the other half to the second.

I came up with this hacky solution:

let open Conduit_lwt_unix in
match conn with
| TCP (tcp : tcp_flow) ->
let dup = Lwt_unix.dup tcp.fd in
Lwt_unix.close tcp.fd >>= fun () ->
Cohttp_lwt_unix.Server.Response.write_header
resp
(Lwt_io.of_fd ~mode:Lwt_io.output dup)
>>= fun () ->
Lwt.join [
(* data in )
read_frames
(Lwt_io.of_fd ~mode:Lwt_io.input dup)
frames_in_fn
frames_out_fn;
( data out )
send_frames
~masked:false ( server never masks the frames *)
frames_out_stream
(Lwt_io.of_fd ~mode:Lwt_io.output dup)
]
| _ -> Lwt.fail_with "expected a TCP Websocket connection"
For what it's worth, it seems to work. I can establish the connection and send/receive frames.

Note: I've changed my local version of read_frames to only take the input_channel and send_frames to only take the output_channel.

Maybe Cohttp should stop reading after getting the request body?

—
Reply to this email directly or view it on GitHub #18 (comment).

[sgrove]

We've been trying to figure this out as well, would love to figure something out here.

[vbmithr]

@zoggy : Could you try this? Are you happy with this solution?
If yes, please close the ticket, I'm preparing a release.

[zoggy]

Thanks. Looking at the exampe code in tests/upgrade_connection.ml: could Cohttp_lwt_body.drain_body body be done by Websocket_cohttp_lwt.upgrade_connection ?

I can't give it a try right now but it seems ok to me.

By the way, compilation still seems to require async to be installed:

ocamlfind ocamldep -package containers -package core -package async -package ppx_deriving.std -package uri -package cohttp.async -package nocrypto.unix -modules tests/wscat_async.ml > tests/wscat_async.ml.depends
+ ocamlfind ocamldep -package containers -package core -package async -package ppx_deriving.std -package uri -package cohttp.async -package nocrypto.unix -modules tests/wscat_async.ml > tests/wscat_async.ml.depends
ocamlfind: Package `async' not found

[vbmithr]

@zoggy : Should we close this ticket?

[zoggy]

I did not find time yet to try, but Cohttp_lwt_body.drain_body is not called in Websocket_cohttp_lwt.upgrade_connection. I think it should be moved here to prevent forgetting to call it from "user"'s code.