Replies: 9 comments 23 replies
-
|
Thanks for making an effort to show a concrete alternative to #40537! Would this approach also help with the OrdinaryDiffEq stacktraces? Example: Here the issue is not that the stack trace is too long, but rather that due to generous use of type parameters for performance/dispatching reasons, single stacks can become >10 KiB long, and the overall stacktrace is ~78 KiB. This for example, was my motivation to jump on the #40537 thread in the first place, since this is not just a question of "this is confusing", but rather "this is literally unusable". |
Beta Was this translation helpful? Give feedback.
-
|
It doesn't really help it at all as this problem because our problem is that most of the type parameters are meaningless. The majority of type parameters are for specialization and have nothing to do with "dispatch" beyond being used by the compiler internally. I talked with @JeffBezanson about this and a solution was coded up in #48444. However, you can see in that discussion that it stalled. While I agree that having some fancy system etc. etc. could be useful in theory, in practice, nobody cares. I have talked with literally hundreds of people about this, and absolutely nobody cares to be able to get the Because of this, we have rolled out a system in https://github.com/SciML/TruncatedStacktraces.jl and are now pirating all stacktraces to say:
Basically, it's not hard. All that needs to happen is that you're given a simple stack trace by default, and a message that tells you how to expand it. It's perfectly fine to have to re-run the code to get the bigger stack trace. |
Beta Was this translation helpful? Give feedback.
-
Note that's not what you get on latest releases. It looks like you're on a pre-TruncatedStacktraces version. If you update most of that goes away, but you'll also want to truncate stacktraces on your types.
Yes exactly, which is why to the disgust of many Base Julia devs we are keeping TruncatedStacktraces until there's a solution merged into Base. And I'll point out again, that there is literally a solution already coded and ready to go for v1.9 as experimental in #48444 and packages like OrdinaryDiffEq.jl are already setup to use it. I hate that we have to do this because it pirates all stacktraces and introduces invalidations, but this is literally the biggest reported problem that people have with the language, and since we have found a solution we just have to use it, even though we know it is something that requires some changes in Base Julia in order to actually not be jank. Literally all that's needed to end this is to take #48444 and add to any stack trace that has a silenced type a similar message to what we say from TruncatedStacktraces and the problem is gone, so this problem really infuriates me because at this point it's one of the biggest problems with language adoption but also a manufactured problem with a solution that already exists and just not merged for package devs to use. |
Beta Was this translation helpful? Give feedback.
-
|
I recognize the frustration, but the solution #48444 (comment) is hopefully not so far off in the future. I'm hoping it will land in Julia 1.10. If you can be patient until then, I think we'll have the best of all worlds. The complex types issue affects more than just "clearly-useless" parameters: we also get some pretty awful printing of types in cases where they are useless 95% of the time and useful 5% of the time. The problem with 48444 is that it does nothing to address this case, and forces us into making a hard-wall choice. |
Beta Was this translation helpful? Give feedback.
-
|
Julia 1.10 would be a clear timeline, so yes, for me that would make it worthwhile to wait for a proper solution 🙂 |
Beta Was this translation helpful? Give feedback.
-
|
To be clear, simplifying type-printing in stack traces (#48444) is totally separate from the issue of shortening the number of frames in the stacktrace (this discussion and #40537). Simplifying type-printing is much higher priority and much easier to implement, which is why I think it will land in Julia 1.10. Abbreviating the stackframes as proposed here is both much harder and less pressing, and it almost surely won't land in 1.10. |
Beta Was this translation helpful? Give feedback.
-
So in cases like: julia> g(x; y=2) = error()
julia> g(1)
ERROR:
Stacktrace:
[1] error()
@ Base ./error.jl:44
[2] g(x::Int64; y::Int64)
@ Main ./REPL[2]:1
[3] g(x::Int64)
@ Main ./REPL[2]:1
[4] top-level scope
@ REPL[3]:1We probably want to hide frame [3]. But in cases like julia> f(g, a; kw...) = error();
julia> @inline f(a; kw...) = f(identity, a; kw...);
julia> f(1)
ERROR:
Stacktrace:
[1] error()
@ Base ./error.jl:44
[2] f(g::Function, a::Int64; kw::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ Main ./REPL[1]:1
[3] f(g::Function, a::Int64)
@ Main ./REPL[1]:1
[4] #f#4
@ ./REPL[2]:1 [inlined]
[5] f(a::Int64)
@ Main ./REPL[2]:1
[6] top-level scope
@ REPL[3]:1we want to hide [4] (and [2]). I think the point is that default positional arguments wrappers inject themselves before the function of interest, while the default kw forwarder injects itself after. |
Beta Was this translation helpful? Give feedback.
-
|
Some more possible incremental improvements: Stacktrace:
[1] zero(#unused#::Type{Any})
@ Base ./missing.jl:106
[2] reduce_empty(#unused#::typeof(+), #unused#::Type{Any})
@ Base ./reduce.jl:343
[3] reduce_empty(#unused#::typeof(Base.add_sum), #unused#::Type{Any})Remove printing In [2] g(; kw::Base.Pairs{Symbol, Int64, Tuple{Symbol, Symbol}, NamedTuple{(:x, :y), Tuple{Int64, Int64}}})
@ Main ./REPL[1]:1Print the NamedTuple with |
Beta Was this translation helpful? Give feedback.
-
|
Sample size of 1, but |
Beta Was this translation helpful? Give feedback.
-
+1 🙂 A |
Beta Was this translation helpful? Give feedback.
-
You can only invalidate code that uses the function you're extending. So if the only place |
Beta Was this translation helpful? Give feedback.
-
|
@timholy is there already a tracking issue for |
Beta Was this translation helpful? Give feedback.
-
|
I don't know of one. Would be happy to have someone add it. |
Beta Was this translation helpful? Give feedback.
-
|
I appreciate this proposal, glad to have spurred further thought and discussion on it. Thinking about this example I've been using, how would these rules apply? Same file and line number rule would remove one frame, 15. Frame 13 just dispatches to Frame 12, which just dispatches to Frame 11, so I suppose 13 and 12 would be removed. Frame 9 and 10 seem like possible candidates for being folded into Frame 11. (I imagine it would also be a good idea to only do the above folding when the frames are part of the same module.) Certainly an improvement. Is this as far as the above rule set could go? |
Beta Was this translation helpful? Give feedback.
-
|
Responding preliminarily now, mostly to say I don't know: as you surely discovered, this analysis takes a bit of time and I haven't (yet) put that in for this example you raised. That would be a good thing to do before getting serious about this effort, but I'm in the middle of other stuff right now. If it does turn out that I/we can't come up with enough ways to simplify the stacktrace, at that point an opt-in AST-like approach would likely become more palatable to me, because we would have already exhausted other "lossless" options. (Analogy: if you want to compress images, try PNG first, but if that's insufficient go to JPG.) |
Beta Was this translation helpful? Give feedback.
-
A difference here to that analogy is that makes no sense to first do a PNG compression and then do a JPG compression on that output but for stacktraces it do makes sense to first to the lossless part and after that tack on the lossy stuff. |
Beta Was this translation helpful? Give feedback.
-
|
Here is another case where things can probably be better: Stacktrace 1 and 2, as well as 3 and 4 are identical. Why are they printed twice? |
Beta Was this translation helpful? Give feedback.
-
|
They aren't identical though. This looks like a case where a task is spawned and then waited for at almost the same place in the code, which will not necessarily happen. |
Beta Was this translation helpful? Give feedback.
-
|
Hm, okay, but nonetheless, it's a scary example for someone who makes a typo in a standard DataFrame call. Maybe DataFrames could do better though. |
Beta Was this translation helpful? Give feedback.
-
|
Repeatedly, I have thought it would be useful to remove common prefixes from stacktraces (including relative to the place it is being printed) |
Beta Was this translation helpful? Give feedback.
-
|
Another trace for inspection, from AWS/HTTP requests that fail. This is an important example because there's very little in the stack trace that is relevant to the error - it's just an HTTP request that produces an error response from the server. All the relevant information is in the HTTP response. Maybe some frames up through the response submission are relevant. A big part of the problem is excessive type information. I'd also argue that this is a case where the exception is being emitted too deep in the stack. But Julia encourages failing early, so I don't think that should be considered in the design here. I don't see any same-line sequences of frames. 39 and 40 could be combined because it's just adding a default kwarg. And a lot of excess type information. Let's just try that out: |
Beta Was this translation helpful? Give feedback.
-
|
I haven't looked in detail to see if it can follow the rules above, but let's just assume we can eliminate the unnamed function frames: Still have 25 stack frames if that checks out. |
Beta Was this translation helpful? Give feedback.
-
|
I really don't think Julia can escape the need for something like AbbreviatedStackTraces, regardless of how many frames can be "safely" dropped. |
Beta Was this translation helpful? Give feedback.
-
|
So much nicer: |
Beta Was this translation helpful? Give feedback.
-
|
I value a conscious, evident based approach to implement solutions, and I also agree that changing twice, like by adopting one solution first, and then soon after changing a slightly different solution, would certainly cause confusion. We all understand the implications of the current state, as many others have already pointed out, how lackluster the current solution is. I feel like we are giving a lot of attention to the details, and miss the one critical aspect: According to some of the voices of both experts and newbies a like, the status quo is reason enough to ditch the language entirely. I mean, if that is not ringing a bell, I don't know what is. We somehow missed the feature freeze for 1.10, so this will delay at least to 1.11. I think this is concerning enough, particularly since it is already like that for years. AbbreviatedStackTraces is a solution that can be described as an already practical improvement that many people like, and I don't see why we should stay with a critical issue for some more years just because another solution could potentially be a little bit better. So I would suggest collecting data, and verifiably proof that this approach yields objectively better results. Otherwise, I suggest including the established solution in the next release. The biggest threat to that, would be that we change it to an even better solution, some day. The value proposition between those two solutions seems to be just significantly far apart, and growing by each day, that we spend with the current solution. Hope for lively and happy responses. 😃 |
Beta Was this translation helpful? Give feedback.
-
|
Another year has passed, and I haven't seen momentum on the piecewise approach, which is what I was afraid of. Has anything been happening out of my view? It's clearly not a high priority for the core devs, which is understandable, but it means a good enough solution is sitting around collecting dust while we wait for time to be freed up for those resisting to come up with a perfect solution. /gripe |
Beta Was this translation helpful? Give feedback.
-
|
My impression is that a lot of progress is being made in one of the prerequisites that Tim has listed above. Namely JuliaSyntax has been merged in Julia and now progress is being made on JuliaLowering https://github.com/c42f/JuliaLowering.jl, so that provenance can be tracked well. That is already a pretty enormous amount of work, and it will take some time for Claire to finish it (I think she is the only JuliaLowering dev right now, and not full-time on it). |
Beta Was this translation helpful? Give feedback.
-
|
Sure, so we're maybe still 1-3 years out from getting from that foundational work to shortening stacktraces? And which even as imagined in the OP, still would be pretty long for some calls. |
Beta Was this translation helpful? Give feedback.
-
Particularly with #40537 and the arrival of AbbreviatedStackTraces, there's been growing interest/pressure in getting Julia to simplify its error-reporting. From my own experience teaching, I'd agree this is an exceptionally important problem. However, in #40537 I've argued that we shouldn't just adopt the AbbreviatedStackTraces approach. Several folks in the community don't seem to like that answer, and seem to imagine there probably isn't anything better on the horizon. So, here's a concrete proposal for something better.
Let's take the infamous stacktrace produced from
sum([]). With inlining off (so you can see all the types), the stacktrace is as follows:Previously I've pointed out one (partial) path towards improvement, which is to drop frames that fill in default argument values from the stacktrace. In a stacktrace, these stand out as successive entries that have the same file & line number. Doing so would eliminate 4 of the 18 entries (it consolidates 16:17, 14:15, 12:13, and 10:11). I'd argue that essentially takes care of all of 10:18, leaving just 1:9 to consider.
Counting backwards from 10, what do those dispatches look like?
2 performs the "fatal" call
zero(Any).It's pretty obvious upon inspection that 8 is the only "real" method among these; the rest just use dispatch to do control flow. (Julia has a tendency to encourage people to solve lots of things as dispatch problems; that has strengths but also directly contributes to the complexity of our stacktraces.) Consequently I think we could report
10:-1:9asi.e., eliding 9. Likewise,
7:-1:2is justThis analysis suggests the following solution: we should report stacktraces where some frame-ranges are represented as ficticious methods, generated by inlining the source-text to correspond to the result from type-inference & optimization. In fact, with inlining on that stacktrace looks like
i.e., the pattern of inlining directly suggests a subset of the solution I'm proposing.
So in aggregate, these approaches would allow a stacktrace which might look something like this:
Is this science fiction? Today, yes. But in a remarkable coincidence of timing, the brand-new TypedSyntax package (which I developed to make Cthulhu better, not to make stacktraces better) provides an approximate way to go from type-inferred code back to source text. I think it's too approximate in its current state, but there is a path to making it perfect: move JuliaSyntax into Base, and then expand it to rewrite lowering so that it too keeps track of "provenance".
So a concrete proposal would be:
f(x, y=2)andg(x; y=2)methods). That seems straightforward and achievable today, and would already yield benefits.Beta Was this translation helpful? Give feedback.
All reactions