[Help]: Problems with PRM creation

[anaferreira97]

Hello everyone,

I'm trying to do a pygmtsar processing in Portugal, i'm using python 3.11.6 and pygmtsar 2024.8.2.post3. When I run the compute_align function, it gives me this error and I can't understand why:

---------------------------------------------------------------------------
_RemoteTraceback                          Traceback (most recent call last)
_RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/opt/conda/lib/python3.11/site-packages/joblib/externals/loky/process_executor.py", line 463, in _process_worker
    r = call_item()
        ^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/joblib/externals/loky/process_executor.py", line 291, in __call__
    return self.fn(*self.args, **self.kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/joblib/parallel.py", line 589, in __call__
    return [func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/joblib/parallel.py", line 589, in <listcomp>
    return [func(*args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/home/jovyan/.local/lib/python3.11/site-packages/pygmtsar/Stack_align.py", line 295, in _align_rep_subswath
    PRM.from_file(mstem_prm).set(PRM.fitoffset(3, 3, par_tmp)).update()
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jovyan/.local/lib/python3.11/site-packages/pygmtsar/PRM.py", line 691, in fitoffset
    raise Exception(f'FAILED - not enough points to estimate parameters, try lower SNR ({rng.shape[0]} < 8)')
Exception: FAILED - not enough points to estimate parameters, try lower SNR (0 < 8)
"""

The above exception was the direct cause of the following exception:

Exception                                 Traceback (most recent call last)
Cell In[25], line 1
----> 1 sbas.compute_align()

File ~/.local/lib/python3.11/site-packages/pygmtsar/Stack_align.py:869, in Stack_align.compute_align(self, geometry, dates, n_jobs, degrees, joblib_aligning_backend, debug)
    866 # prepare secondary images
    867 with self.tqdm_joblib(tqdm(desc='Aligning Repeat', total=len(dates_rep)*len(subswaths))) as progress_bar:
    868     # threading backend is the only one working inside Docker container to run multiple binaries in parallel
--> 869     joblib.Parallel(n_jobs=n_jobs, backend=joblib_aligning_backend)(joblib.delayed(self._align_rep_subswath)(subswath, date, degrees=degrees, debug=debug) \
    870                                    for date in dates_rep for subswath in subswaths)
    872 if len(subswaths) > 1:
    873     # calculate the offsets and also create merged reference PRM
    874     offsets = self.get_subswaths_offsets(self.reference, debug=debug)

File /opt/conda/lib/python3.11/site-packages/joblib/parallel.py:1952, in Parallel.__call__(self, iterable)
   1946 # The first item from the output is blank, but it makes the interpreter
   1947 # progress until it enters the Try/Except block of the generator and
   1948 # reach the first `yield` statement. This starts the aynchronous
   1949 # dispatch of the tasks to the workers.
   1950 next(output)
-> 1952 return output if self.return_generator else list(output)

File /opt/conda/lib/python3.11/site-packages/joblib/parallel.py:1595, in Parallel._get_outputs(self, iterator, pre_dispatch)
   1592     yield
   1594     with self._backend.retrieval_context():
-> 1595         yield from self._retrieve()
   1597 except GeneratorExit:
   1598     # The generator has been garbage collected before being fully
   1599     # consumed. This aborts the remaining tasks if possible and warn
   1600     # the user if necessary.
   1601     self._exception = True

File /opt/conda/lib/python3.11/site-packages/joblib/parallel.py:1699, in Parallel._retrieve(self)
   1692 while self._wait_retrieval():
   1693 
   1694     # If the callback thread of a worker has signaled that its task
   1695     # triggered an exception, or if the retrieval loop has raised an
   1696     # exception (e.g. `GeneratorExit`), exit the loop and surface the
   1697     # worker traceback.
   1698     if self._aborting:
-> 1699         self._raise_error_fast()
   1700         break
   1702     # If the next job is not ready for retrieval yet, we just wait for
   1703     # async callbacks to progress.

File /opt/conda/lib/python3.11/site-packages/joblib/parallel.py:1734, in Parallel._raise_error_fast(self)
   1730 # If this error job exists, immediatly raise the error by
   1731 # calling get_result. This job might not exists if abort has been
   1732 # called directly or if the generator is gc'ed.
   1733 if error_job is not None:
-> 1734     error_job.get_result(self.timeout)

File /opt/conda/lib/python3.11/site-packages/joblib/parallel.py:736, in BatchCompletionCallBack.get_result(self, timeout)
    730 backend = self.parallel._backend
    732 if backend.supports_retrieve_callback:
    733     # We assume that the result has already been retrieved by the
    734     # callback thread, and is stored internally. It's just waiting to
    735     # be returned.
--> 736     return self._return_or_raise()
    738 # For other backends, the main thread needs to run the retrieval step.
    739 try:

File /opt/conda/lib/python3.11/site-packages/joblib/parallel.py:754, in BatchCompletionCallBack._return_or_raise(self)
    752 try:
    753     if self.status == TASK_ERROR:
--> 754         raise self._result
    755     return self._result
    756 finally:

Exception: FAILED - not enough points to estimate parameters, try lower SNR (0 < 8)

I've tried a larger area and it still gives me the same error, can anyone help me?

[AlexeyPechnikov]

The key lines are:

  File "/home/jovyan/.local/lib/python3.11/site-packages/pygmtsar/PRM.py", line 691, in fitoffset
    raise Exception(f'FAILED - not enough points to estimate parameters, try lower SNR ({rng.shape[0]} < 8)')
Exception: FAILED - not enough points to estimate parameters, try lower SNR (0 < 8)

This suggests that your DEM is incomplete or that the images are too noisy to successfully align.

[sarvagya112]

The DEM is being downloaded automatically, so how can we modify that?

[AlexeyPechnikov]

You can check how well it is covering the area. Some tiles may not have been downloaded or might be completely missing. In such cases, you can use another DEM source.