Debugging Aid (#165)

* fix: Issues creating and handling

- Fixed reporting non-imported (`save()` failed) instances to DiffSync
- Added `created` and `updated` to Nautobot model stats

* feat: Issues tracing

- Added `--trace-issues` argument to `nautobot-server import_netbox` command.

* chore: Build test fixtures

* chore: Added changelog fragments

* fix: Re-introduce removed `invoke import-netbox`

* fix: docs

* chore: Improved docstring for add_issue

changes/165.added

@@ -0,0 +1,2 @@
+Added `--trace-issues` argument to `nautobot-server import_netbox` command to log exception trace-backs.
+Added `created` and `updated` to Nautobot model stats.

changes/165.fixed

@@ -0,0 +1 @@
+Fixed reporting non-imported (`save()` failed) instances to DiffSync library.

development/nautobot_config.py

@@ -113,12 +113,14 @@
             },
         },
         "loggers": {
-            "django": {"handlers": ["normal_console"], "level": "INFO"},
+            "django": {"handlers": ["normal_console"], "level": "INFO", "propagate": False},
             "nautobot": {
                 "handlers": ["verbose_console" if DEBUG else "normal_console"],
                 "level": LOG_LEVEL,
+                "propagate": False,
             },
         },
+        "root": {"handlers": ["normal_console"], "level": "INFO"},
     }
 
     if DEBUG:

docs/dev/demo.md

@@ -161,7 +161,7 @@ The initial output of the import process will resemble the following:
 
 ```
 Running docker compose command "ps --services --filter status=running"
-Running docker compose command "exec nautobot nautobot-server import_netbox --save-json-summary-path=generated-mappings.json --bypass-data-validation --dry-run --field-mapping  --sitegroup-parent-always-region --summary  --no-color https://raw.githubusercontent.com/netbox-community/netbox-demo-data/master/json/netbox-demo-v3.6.json"
+Running docker compose command "exec nautobot nautobot-server import_netbox --save-json-summary-path=generated-mappings.json --bypass-data-validation --dry-run --sitegroup-parent-always-region --print-summary  --no-color https://raw.githubusercontent.com/netbox-community/netbox-demo-data/master/json/netbox-demo-v3.6.json"
 11:01:05.550 DEBUG   nautobot.core.celery __init__.py        import_jobs_as_celery_tasks() :
   Importing system Jobs
 11:01:05.552 DEBUG   nautobot.core.celery __init__.py                      register_jobs() :

docs/user/app_use_cases.md

@@ -7,11 +7,10 @@ This document describes common use-cases and scenarios for this App.
 This app provides `import_netbox` management command to import data from NetBox with the following options:
 
 ```bash
-nautobot-server import_netbox --help
-
+nautobot-server help import_netbox
 usage: nautobot-server import_netbox [-h] [--dry-run] [--update-paths] [--bypass-data-validation] [--sitegroup-parent-always-region] [--fix-powerfeed-locations] [--print-summary]
-                                     [--no-unrack-zero-uheight-devices] [--save-json-summary-path SAVE_JSON_SUMMARY_PATH] [--save-text-summary-path SAVE_TEXT_SUMMARY_PATH] [--version] [-v {0,1,2,3}]
-                                     [--settings SETTINGS] [--pythonpath PYTHONPATH] [--traceback] [--no-color] [--force-color] [--skip-checks]
+                                     [--no-unrack-zero-uheight-devices] [--save-json-summary-path SAVE_JSON_SUMMARY_PATH] [--save-text-summary-path SAVE_TEXT_SUMMARY_PATH] [--trace-issues] [--version]
+                                     [-v {0,1,2,3}] [--settings SETTINGS] [--pythonpath PYTHONPATH] [--traceback] [--no-color] [--force-color] [--skip-checks]
                                      json_file
 
 Import a NetBox JSON data dump into Nautobot's database
@@ -38,6 +37,7 @@ options:
                         File path to write the JSON summary to.
   --save-text-summary-path SAVE_TEXT_SUMMARY_PATH
                         File path to write the text summary to.
+  --trace-issues        Show a detailed trace of issues originated from any `Exception` found during the import.
   --version             show program's version number and exit
   -v {0,1,2,3}, --verbosity {0,1,2,3}
                         Verbosity level; 0=minimal output, 1=normal output, 2=verbose output, 3=very verbose output

nautobot_netbox_importer/diffsync/adapters/netbox.py

@@ -60,6 +60,7 @@ class NetBoxImporterOptions(NamedTuple):
     unrack_zero_uheight_devices: bool = True
     save_json_summary_path: str = ""
     save_text_summary_path: str = ""
+    trace_issues: bool = False
 
 
 AdapterSetupFunction = Callable[[SourceAdapter], None]
@@ -71,7 +72,13 @@ class NetBoxAdapter(SourceAdapter):
     # pylint: disable=keyword-arg-before-vararg
     def __init__(self, input_ref: _FileRef, options: NetBoxImporterOptions, job=None, sync=None, *args, **kwargs):
         """Initialize NetBox Source Adapter."""
-        super().__init__(name="NetBox", get_source_data=_get_reader(input_ref), *args, **kwargs)
+        super().__init__(
+            name="NetBox",
+            *args,
+            get_source_data=_get_reader(input_ref),
+            trace_issues=options.trace_issues,
+            **kwargs,
+        )
         self.job = job
         self.sync = sync