-
-
Notifications
You must be signed in to change notification settings - Fork 654
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Haxe-in-Haxe notes #6843
Comments
I have added |
Since Haxe currently support C# would F# provide a simpler path. |
I have compiled F# mono graphics examples on mac so it's not just windows although doubt it would provide the same speed as Ocaml initially? |
I also wonder about using hxcpp but I suppose libraries like |
@nanjizal I don't think F# produces as fast code as OCaml do, as it's built on top of DotNet |
@nanjizal anyway I have little interest in targeting F#, and this requires extra profiling and extensive changes as the article explains |
@nanjizal let's not diverge the focus here. Porting from the compiler from OCaml to Haxe is a big task already. Let keep this post focused on this single aspect. Using other target language is beyond the scope of the post as said in OP already:
|
Awesome, Keep up the good work guys! I think this will improve Haxe a lot! |
@ncannasse what about performance? I mean, if feature X will develop in Haxe->Ocamel it will be slower than if written in Ocamel directly? the difference is negligible? |
It would be very interesting if there was an example of what Haxe code might look like for targeting Ocaml so a snippet showing a small aspect of functionality. I am not even sure on what support Haxe has for immutable code at the moment ( there is now final, but I have not seen it used ). |
@mikicho the idea is to have 0 loss of performance in the process, so we are looking for something almost 1-to-1 conversion |
So I started a little ocaml project for converting ml to hx here https://github.com/nadako/ml2hx. It's really rough but the foundation is there to improve upon. It works by invoking ocaml as a lib and processing the typed AST to generate Haxe code. Currently requires 4.06 and probably won't compile with earlier versions, because the compiler API incompatibilities, but oh well. |
I'm a little surprised this didn't get mentioned: https://github.com/elnabo/haxe_in_haxe ^^ Either way, this is very exciting news. I really hope something will come out of it ;) |
@nadako hi very interesting, obviously my first thought was to try the code on itself so I can understand the Ocaml code better: $ ./main main.ml Should this work? ( By the way could the make file work without ocamlfind and use opam instead, it's not in homebrew and not needed for make haxe compiler? Well I have installed ocamlfind here, but for other mac users might be better to avoid? I don't know? ) |
@nanjizal Please discuss about his awesome project in its repo. |
Could someone shed some light on https://github.com/elnabo/haxe_in_haxe? Is it a coincidence that the creation of this repo somehow matches the creation of this issue? I guess the idea is to first automatically transpile the OCaml codebase to Haxe using https://github.com/nadako/ml2hx. and then fix / improve manually as we go? That said, those are really great news! 🎉 |
It's not, it was published when discussions started about Haxe in Haxe :) I'm not sure about the fate of this repo though, since porting this amount of code by hand is not really an option.
Yeah, that's the idea at the moment. Thanks to the fact Haxe is actually quite similar to ocaml (enums, pattern matching, var shadowing, everything as expression, etc.), it's pretty straightforward to generate readable-ish haxe code from ocaml. |
I started to try and do a copy of the compiler in Haxe a month ago. And when the issue of writing Haxe in Haxe arose, @ibilon told me to make my code public. I don't know how long, I'll continue it, because it's likely other options will succeed first. But for the moment, it help me understand a bit better the compiler. |
@elnabo well it may turn out that for hxcpp this approach is better, we can't assume that ocaml/functional output will provide the best results, but I can imagine that an automated initial port that nadako has started should provide a solution that more easily covers all the edge cases currently. But you can never be sure where such projects leads I certainly urge you to continue sometimes working from different angles also brings a better overall solution and new opportunities. Can you compile the JS target with your code? |
I tried to compile with hxnodejs but I get weird error But as I try to be as similar to the source as possible. Me and nadako should end with similar looking code. Except I'm likely to have bugs hidden somewhere, as I think I currently have. |
RantYou're crazyI just read through this issue properly for the first time and think y'all are crazy. Porting the Haxe sources to Haxe only to then compile them back to OCaml is insanity. Let's look at what's bad about OCaml:
Which of these problems would be alleviated by having the Haxe sources in Haxe and compiling them to OCaml? Well, we would still be stuck with the same crappy toolkit, except that we now have another layer on top of it which makes most of the issues even worse:
Furthermore, all that is assuming that we would even make it that far in the first place. I don't share your optimism that the code base could be converted to maintainable Haxe and be generated back to efficient OCaml. There's no way that you could generate idiomatic Haxe to efficient OCaml in general, so we would have to gimp ourselves by essentially writing OCaml-idiomatic code in Haxe. AlternativesSo in summary, I think this entire idea of going through OCaml should be dropped. Which obviously raises the question of what we should do instead. Given that we want to write Haxe-in-Haxe, I can see two possibilities:
Given that 1. requires an additional step, it would seem that 2. is the best option. At this point, hxcpp is pretty battle-hardened. And guess what, it already has a generational GC, which is the reason why Nicolas clings to OCaml. I can't speak to the efficiency of it, but in my experience all it takes to get Hugh on a job is a benchmark showing a problem. What's more, hxcpp supports parallelism. Even if its GC doesn't beat OCaml's (which I expect to be the case), I'd wager that the usage of multithreading would make up for that. And yes I'm implying that we should rewrite Haxe instead of trying to awkwardly port it. I know people think it's not an option, but I disagree. We're currently sitting at just above 100kloc, which is not much. I can easily write you a parser + typer + optimizer + JS generator in one month of focused effort if I have a clear vision of what to go for. I'd have to think a bit about how to port eval, but that can also be figured out. The analyzer would require some redesign that doesn't depend on functors. Porting generators requires some manpower, but there's nothing inherently difficult about it either. IMO this is what we should go for with Haxe 5. There's a million questions still to be answered obviously, but I really want to get the point across that targeting OCaml is a huge strategic mistake in 2019. |
I'm not sure how much different the rewrite would look like from the "OCaml-idiomatic code", especially if we talk about multi-threading. I'm pretty sure it'll still be a bunch of recursive pattern matching functions and persistent data structures, unless "we" want to go for the OOP visitor pattern madness (I can see different array handling and some proper non-exception based control flow though:)). That's why I think that automated port is technically viable, even though with all the extra punctuation that Haxe will bring, our single-letter names will look even more messy :) Personally I have no problems with OCaml nowadays, and it's tooling is slowly getting better, but I could of course work on Haxe in Haxe. In some situations Haxe would be actually nicer (e.g. reification and/or macros for expr building). Regarding the run-time, I was recently writing a compiler for AS3 in a functional style, simiilar to how Haxe is written, and I can say this requires a really really good GC, because even V8 (node.js) chokes badly with a lot of persistent data object allocations. I had to add quite some boilerplate checks to do less allocations so it doesn't die. HXCPP seemed to perform better, but I didn't do deeper benchmarking, but I feel like at this point "we" have a lot of experince to improve GC :) |
I partly agree with the rant against OCaml and its "difficulties" to setup and use. I think an interesting question would be ask ourselves why OCaml performs so good. I think it boils down to two things: a) good code analysis / auto inlining / tail call optimization, resulting in functional code being entirely lifted into more imperative constructors I think we could provide the (a) as part of some of our native targets (HxCPP and/or HL) |
Regarding multithreading, I there it could work well for some very specific cases (pure map, some truly parallel stateless filters, etc.) but most of the rest of the compiler code has to remain single threaded for our own sanity. |
I think it would be a lot nicer to skip the OCaml layer if we could use something like HashLink or HxCPP instead. I really like how tiny HashLink is and how easy it is to compile. Reducing the need for OCaml tooling to just needing a C/C++ compiler would go a long way in making it easier to get setup for development. |
This is quite interesting on lack of parrallel Ocaml, but I expect you have read it, but there are many interesting aspects if you read all the comments. |
Yes, forget OCaml as a target initially. Forget adding AST components. Forget changing the haxe compiler at all - these are just distractions. The is a project written in haxe, not ocaml. No new "switch" syntax - just use if/else - after all, you are generating code so you can be as verbose as you like. So
becomes:
and recursion would be something like:
where l.afterHead creates a new list view starting at the next element No changes to the haxe compiler are required for this. You just need to start with a haxe-readable ocaml ast dump of some sort. |
What is the current status of supporting OCaml as a target language? |
cc @ncannasse who wrote the first protototype. I would love to help if needed (I am an OCaml programmer). |
I think that it's a good idea to first have a Haxe in Haxe like the one I started here HaxeFoundation/code-cookbook#160 |
Also would be nice to have a EBNF grammar for HAXE before going forward, otherwise the code is the specification will make it hard. |
I have this crude HAXE grammar that I expect to publish here https://github.com/mingodad/plgh it can be visualized here https://www.bottlecaps.de/rr/ui , help to improve it is welcome !
|
I just extended the crude Haxe grammar and published it here https://github.com/mingodad/plgh/blob/main/haxe.ebnf again any help to improve it is welcome ! |
I think for haxe 5 it would be wise to make it haxe in haxe. Based off just some generic internet benchmarks C++ G++ and C GCC are technically faster, although I don't know if this holds up for haxe compiled code (especially 🤢 windows compiled code, considering this uses gcc and g++). Between HL/C and C++ it seems its a tossup depending on the task (which emphasizes the importance of what you are doing), but C++ wins for more complex tasks (or tasks that take longer in general) and C wins for shorter tasks. However, both of them seem to beat out ocaml (C i'm not too sure about, as there is no direct comparison), which for our purposes is all that matters. Personally, if I were rewritting/porting the compiler to haxe, my target would be C++ as C++ is nicer to work with and is lax with its' types (unlike a certain OTHER HAXE VM), and it also runs faster for more complex tasks. Porting to haxe should be done not just because performance - although it's a nice benefit - but because the HAXE community wants to hack onto the compiler. Currently, I would like to edit how the compiler handles java/jvm jar interface importing. Haxe doesn't have default interfaces and makes you do them yourself (which imo is a bit silly but that's another thing). I downloaded haxe repository, downloaded ocaml (which by the way was a pain to do on windows) and looked at the code and said "What the heck is this language, what does it mean?" OCaml, at least to a person who has only written in C#, Javascript, Java, and Haxe, is very unreadable and has a steep learning curve. I would have been probably completely done with the Haxe4Java project by now if it was written in haxe. Rewriting the compiler to haxe would make it easier to recruit new contributors who know what they're doing, because it's written in the language that you want to help with. TL;DR; C++ and C are faster than OCaml, C++ is generally faster, OCaml has a steep learning curve, and if haxe was rewritten in haxe the project could be maintained by more people and more easily. |
Compiler looks like it makes a big difference for C, and I assume the same applies to C++. So Hxcpp will probably make a big difference, same with HL/C. OCaml may have consistent speed (which some would say is better than just being fast). this invalidates the first 2 paragraphs of my mini-essay a bit but I think the 3rd point still stands. However, we don't have benchmarks for it. When I get home and finish my work I'll rewrite some of those programs in haxe. |
You won't like hearing this, but Haxe is (like all compilers) complicated and people mostly just use OCaml as an excuse once they realize that. ... Haxe should still eventually move away from OCaml, but this "more contributors" argument has always been nonsense. The only way to prove me wrong is by making substantial contributions to haxelib. :| |
I don't think speed is a great issue. I'm sure that if some auto-port is done, it will be way slower initially and this will be about the memory usage patterns rather than raw language speed. It would then be a matter of profiling and optimizing, which can be an ongoing process. Ultimately, I think the speed gains would come from multi-threading. |
The OCaml installation is our hazing ritual. |
Hey, I half understand haxelib. I hacked onto it with silk and added commands to it (my favorite being the why command, which required writing new functions that iterate thru the libs installed). I'm sure if I tried I would be able to add it to the real haxelib. And just to be petty I may do that today lmao |
Do we know what the major use-cases are for hacking on the compiler? My main use is adding new targets and other code generation on the back-end. Context.onGenerate works well for this. Improving the performance of interp would make this use better. It might be nice to bridge the internal haxe.macro.Type api to HashLink so that generators written in Haxe could be compiled to HL C and incorporated in a custom compiler build. |
Going to just leave this here for anyone feeling adventurous |
@hughsando would ocaml-hxcpp be possible in a similar manor? |
Main use case for me would be adding targets, and adding new fully supported features (like default implementations). Macros technically work for this however if I hack on the compiler I can add new keywords |
I could see many advantages for having Haxe in Haxe:
|
I'm not sure if useful, but it would grant the ability to compile client side in a browser. |
making hscript be actually haxescript would be amazing |
hscript is already very powerful; if we wrote the compiler in haxe we could include it as a lib. recommendations: make a class that just calls interp/make a “haxe” library |
Recently I have been working with some concepts of transpilers for academic purposes, and Haxe would be a good framework to test it (instead of creating yet another transpiler). Maybe this concept is the same of genml (GenOCAML) in the beginning of this thread. It is easier to show the concept with examples. For instance, GenSwift.hx could be an implementation for Swift target.
The file GenSwift.hx could be implemented with something like this:
If this concept is correct, the Ast wrapper could replace these steps:
|
Revisiting, I do think that ocaml is probably a better language. After learning haskell it is much more understandable - and for language parsing, the more functional is usually better. Perhaps what we really want is more powerful access to haxe compiler. I know JS already has something like this, but a good way to write in haxe a target would be amazing. I also noticed that haxe plugins exist, but I am unsure of the power they provide. A haxe in haxe may be cool, as it would allow easy installation and building of new haxe compilers via haxe (would surely be a boon for e.g. lix). |
any progress on haxe/ocaml target ? |
Apparently one can create compilation targets using Haxe Macros now: https://github.com/RobertBorghese/reflaxe |
Good news or bad news first? Good:Haxe lang is able to do similar compile to C/C++ and JavaScript that Nim is able, but Also Open-FL(ASH) library, which allows migrating most Only God knows why Haxe has 3 times less stars (6K at time of writting) than Nim (16K). Bad:Haxe-in-Haxe should absolutely have nothing to do with OCaml-target. I mean:
Improve C/C++ target, so that it will be able to handle any task that OCaml-target is able to, and don't waste time adding more targets if your existing ones can't handle a simple Haxe-in-Haxe ;-) Edit: even now "only God knows", however, if you allow a guess, it seems the majority of the world uses Windows. |
The first goal is to have the compiler written in Haxe and generating the corresponding OCaml code, allowing for a more familiar syntax for potential contributors and compiler team. At longer term, this will allow the compiler to run on other platforms although this is not something that will be possible or that we should focus at first.
The following needs to be resolved to have a working Haxe-in-Haxe implementation.
Note to Haxe users : this is an exploration of the possibility, we are not sure yet we will have something working in the end.
OCaml Haxe target (Haxe-to-OCaml) (@ncannasse) :
::
as proper operator with type inference, correctly infer::
in pattern matchingAutomated Haxe compiler OCaml-to-Haxe ML2HX (@nadako) :
Merge of both ML target and ML2HX :
The text was updated successfully, but these errors were encountered: