Export Template Containing Values From Objects Not In Queryset

[flippind]

I'm trying to adjust a DCIM\Cable export template I had in place for the old (pre 3.x) cable model. Specifically, I'm having issue getting the serial of the devices of [ab]_terminations. The old code was simply {% if cable.b_terminations.object.device.serial %}({{cable.b_terminations.object.device.serial[-4:]}}){% endif %} to add the last 4 digits of the serial number wrapped in parenthesis (xxxx). However, since the new model does not include the serial in the object itself anymore, I need to make a fresh query to the device to access that field. It should be trivial to get the output I want after having access to that data, but getting the data in the first place is the hard part. The pkid of the device is already in the data as should be the device URL, though I can't actually get that to print in the template for some reason (string wrapping of characters or escaping the whole string possibly? I'm not familiar enough with Jinja2 to bother spending time on it). I just can't figure out how to query that device object itself within the template code. Is performing a subquery even supported and, if so, how would I go about doing so?

EDIT:
Looking back on the original template, I'll need to include the interface label from the terminations as well. Oddly, the cable already has the display property (at least according to the API) which is exactly the value I want, but - like the URL - I can't get that into the template for some reason

A bad case of the Fridays invalidated my initial use case, as @candlerb helped determine. However, the root question seems to be open still - Is performing a subquery within an export template supported, and how would one go about doing so?

Using a cable's attached device serial as an example (even though it's easily accessible directly). Even ruling out attempting to use a variable for the device PKID, any cased combination of dcim.device.objects.get(pk=n).serial or just straight up device.objects.get(pk=n).serial always complains about the master object (either DCIM or Device in that casing or all lower) being undefined, with or without DCIM | Device in the assigned models, if the original object type to be exported is not a Device (in this case, a Cable). According to nbshell Device.objects.get(pk=n).serial should be all that's necessary, but there appears to be something within how the Export request is associated with another model initially that prevents it. It appears that functionality was provided for at least Config Templates in #12814 but does not seem functional within Export Templates.

[candlerb]

I'm trying to adjust a DCIM\Cable export template I had in place for the old (pre 3.x) cable model.

I can't see how your code ever worked with Netbox 2.x, since the array of terminations wasn't added until 3.3

Specifically, I'm having issue getting the serial of the devices of [ab]_terminations. The old code was simply {% if cable.b_terminations.object.device.serial %}({{cable.b_terminations.object.device.serial[-4:]}}){% endif %} to add the last 4 digits of the serial number wrapped in parenthesis (xxxx).

I don't understand which object you're talking about, or even what the .object accessor did or does. cable.b_terminations returns an array of terminations, each of which is a component (e.g. interface, frontport etc), and to get the device which those are on, you use the device accessor.

Use nbshell to try this out interactively:

$ /opt/netbox/venv/bin/python /opt/netbox/netbox/manage.py nbshell
### NetBox interactive shell (netbox4)
### Python 3.12.3 | Django 5.0.7 | NetBox 4.0.8
### lsmodels() will show available models. Use help(<model>) for more info.
>>> c = Cable.objects.first()
>>> c.a_terminations
[<FrontPort: F16>]
>>> c.a_terminations[0]
<FrontPort: F16>
>>> c.a_terminations[0].device
<Device: pp1>

Remember of course that terminations is a list: if there are multiple terminations, either you just pick the first one (as above), or you must iterate over them all.

However, since the new model does not include the serial in the object itself anymore

What do you mean? Which model are you talking about? serial is still an attribute of Device, as it always was.

>>> c.a_terminations[0].device.serial
''

, I need to make a fresh query to the device to access that field. It should be trivial to get the output I want after having access to that data, but getting the data in the first place is the hard part. The pkid of the device is already in the data as should be the device URL, though I can't actually get that to print in the template for some reason (string wrapping of characters or escaping the whole string possibly? I'm not familiar enough with Jinja2 to bother spending time on it). I just can't figure out how to query that device object itself within the template code. Is performing a subquery even supported and, if so, how would I go about doing so?

I don't really understand any of that, except that you can start a new query within a template by using the top-level model classes passed in, such as dcim.Device. This was introduced in v3.6.0:

• #12814 - Expose NetBox models within ConfigTemplate rendering context

Example:

You have {{ dcim.Device.objects.count() }} devices

EDIT: Looking back on the original template, I'll need to include the interface label from the terminations as well. Oddly, the cable already has the display property (at least according to the API) which is exactly the value I want, but - like the URL - I can't get that into the template for some reason

You're right that it's not obvious how the REST API generates the display attribute, but it appears to be simply the object's str() method (look for method get_display, which is overridden in a few subclasses but not any dcim ones)

Therefore, I think you render the cable (or device etc) as a string:

>>> c
<Cable: #19>
>>> str(c)
'#19'
>>> print(c)
#19
>>>

This will happen by default anyway if you render the object in a template, e.g. {{ cable }}.

[flippind]

I can't see how your code ever worked with Netbox 2.x, since the array of terminations wasn't added until 3.3
I don't understand which object you're talking about, or even what the .object accessor did or does.

The {% if cable.b_terminations.object.device.serial %}({{cable.b_terminations.object.device.serial[-4:]}}){% endif %} code is that of the original (now non-operational) template, as displayed in the current v3.7.8 instance. I didn't see anything in the patch notes between our previous version and current version regarding template updates, so I assume that's the same original code. I created that template several years ago, so your guess is as good as mine, but it most definitely did work in 2.7 - I used the export just a couple of days prior to upgrading. Below is the code I'm using in the updated template, where the queryset is against DCIM\Cable. I used the conversations in #11762 and #12018 as starting points for the new template.

{%- for c in queryset %}
{%- set adev = c.a_terminations[0].device %}
{%- set bdev = c.b_terminations[0].device %}

What do you mean? Which model are you talking about? serial is still an attribute of Device, as it always was.

That's exactly what I was attempting to do original. You would expect adev.serial and bdev.serial to return the appropriate values, but they always come up as empty strings. To validate I just made an API call for the cable I'm using to build the template and the serial field is displayed for neither device, so I assume the cable termination model does not use that property in its reference. Hence, the assumption that another query against the particular device is necessary to get the serial from it.

 "id": 1659,
    "url": "https://netbox.fqdn/api/dcim/cables/1659/",
    "display": "240801",
    "type": "cat6",
    "a_terminations": [
        {
            "object_type": "dcim.interface",
            "object_id": 9544,
            "object": {
                "id": 9544,
                "url": "https://netbox.fqdn/api/dcim/interfaces/9544/",
                "display": "LAN (Network Icon)",
                "device": {
                    "id": 724,
                    "url": "https://netbox.fqdn/api/dcim/devices/724/",
                    "display": "Fusion",
                    "name": "Fusion"
                },
                "name": "LAN",
                "cable": 1659,
                "_occupied": true
            }
        }
    ],
    "b_terminations": [
        {
            "object_type": "dcim.interface",
            "object_id": 251,
            "object": {
                "id": 251,
                "url": "https://netbox.fqdn/api/dcim/interfaces/251/",
                "display": "eth1/28 (28)",
                "device": {
                    "id": 6,
                    "url": "https://netbox.fqdn/api/dcim/devices/6/",
                    "display": "SW01",
                    "name": "SW01"
                },
                "name": "eth1/28",
                "cable": 1659,
                "_occupied": true
            }
        }
    ],

Well what the heck? I must've had a bad case of the Fridays, because adev.serial definitely does work. I wonder if I did something non-obvious like adev.sn, adev.serialnumber, or adev.serialno instead.

This will happen by default anyway if you render the object in a template, e.g. {{ cable }}.

That appears to be the case. {{ c.a_terminations[0] }} does return the same string as the API defines as display

Thanks for all the input - that helps me resolve my actual task requirements. But it might be helpful following the root question - how to perform subqueries - all the way through for future users (if ever a use case comes along).

Using a cable's attached device serial as an example (even though it's easily accessible). Even ruling out attempting to use a variable for the device PKID, any cased combination of dcim.device.objects.get(pk=n).serial or just straight up device.objects.get(pk=n).serial always complains about the master object (either DCIM or Device in that casing or all lower) being undefined, with or without DCIM | Device in the assigned models, if the original object type to be exported is not a Device (in this case, a Cable). According to nbshell Device.objects.get(pk=n).serial should be all that's necessary, but there appears to be something within how the Export request is associated with another model initially that prevents it.

As an aside, the templating system really leaves a lot to be desired from an end-user perspective. Even just replicating an easily displayed table into a template wants the user to be familiar with Jinja2, Django, have administrative access to the backend venv environment, Python (to activate said venv), and use a tool (nbshell) which is explicitly mentioned should only be used by knowledgeable users. I'm no dev (I just know enough to be dangerous in many languages) and this is really my first foray into Django and Jinja2, so I can't really contribute much more than that opinion.

[candlerb]

The confusion is probably between export templates and config templates.

The following config template (under Provisioning top-level menu) works for me:

{{ dcim.Device.objects.count() }}

But an export template (under Customization top-level menu) is not the same thing, and I have not tested that. If it doesn't work there, you could raise a feature request.

[flippind]

I don't know if a feature request is the correct path, since it's currently just a hypothetical. I'll leave this unanswered for now and clarify the original to see if anyone else can chime in.

Thanks again for your help at least getting me through my initial task.