Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speeding up the phase linking processing for large, single stacks #265

Open
scottstanie opened this issue Mar 12, 2024 · 0 comments
Open
Labels
enhancement New feature or request run time Related to workflow processing speed

Comments

@scottstanie
Copy link
Collaborator

scottstanie commented Mar 12, 2024

For NISAR processing, we'll need to figure out how to get similar data parallelism for the phase linking loop as sentinel-1 bursts.

There's discussion, but not consensus on the best way to speed up Jax on CPUs
(See also jax-ml/jax#5506 and jax-ml/jax#10180)

While we use jax.vmap on the eigenvalue computation, this is used to SIMD vectorize JAX code. It does not actually do CPU-parallelization. This is why the CPU usage during much of the wrapped phase code is rather low.

This isn't a problem for Sentinel-1 where we'll process multiple bursts at the same time with n_parallel_bursts; but we'll need to think about better parallelization strategies for large, single-stack processing like with NISAR/ALOS.

@scottstanie scottstanie added enhancement New feature or request run time Related to workflow processing speed labels Mar 12, 2024
@scottstanie scottstanie changed the title pmap may be able to speed up the EMI loop Speeding up the phase linking processing for large, single stacks Jun 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request run time Related to workflow processing speed
Projects
None yet
Development

No branches or pull requests

1 participant