cannot remove protocols #234

modularTaco · 2021-03-03T16:53:05Z

Hey,

I tried removing removing iotas from iotam, but this does not seem to be possible. Neither in the latest released version (1.13.0) nor in the latest dev version (latest tag on docker hub).

The registered iotas are as follows:

➜ curl -s localhost:8082/iot/protocols | jq .
{
  "count": 3,
  "protocols": [
    {
      "_id": "5ea0a27a5e27f20010e5fe6a",
      "description": "Ultralight 2.0 IoT Agent (Node.js version) supports HTTP/MQTT/AMQP protocols",
      "protocol": "UL",
      "resource": "/iot/ul",
      "iotagent": "http://ul-iot-agent:4041/config/iot",
      "__v": 0
    },
    {
      "_id": "5ea0a2815e27f20010e5fe6b",
      "description": "JSON IoT Agent (Node.js version) supports HTTP/MQTT/AMQP protocols",
      "protocol": "JSON",
      "resource": "/iot/json",
      "iotagent": "http://json-iot-agent:4041/config/iot",
      "__v": 0
    },
    {
      "_id": "5ef9f548d16bb5000fcdc97f",
      "description": "LORAWAN IoT Agent, supports TTN and Loraserver.io",
      "protocol": "LORAWAN",
      "resource": "/iot/lorawan",
      "iotagent": "http://lorawan-iot-agent:4041/config/iot",
      "__v": 0
    }
  ]

I tried deleting a registered iota with the following command:

➜ curl -X DELETE localhost:8082/iot/protocols/5ea0a2815e27f20010e5fe6b

With 1.13.0 the following appears in the log:

iot-agent-manager-iot-agent-manager-67d79966cb-lv5g9 iot-agent-manager time=2021-03-03T16:35:53.192Z | lvl=DEBUG | corr=5c5b4cc7-7e62-479a-aa86-7554294ed64e | trans=5c5b4cc7-7e62-479a-aa86-7554294ed64e | op=IoTAgentNGSI.GenericMiddlewares | from=n/a | srv=n/a | subsrv=n/a | msg=Request for path [/iot/protocols/5ea0a2815e27f20010e5fe6b] from [localhost:8082] | comp=IoTAgentManager
iot-agent-manager-iot-agent-manager-67d79966cb-lv5g9 iot-agent-manager time=2021-03-03T16:35:53.194Z | lvl=DEBUG | corr=5c5b4cc7-7e62-479a-aa86-7554294ed64e | trans=5c5b4cc7-7e62-479a-aa86-7554294ed64e | op=IoTAManager.ProtocolAPI | from=n/a | srv=n/a | subsrv=n/a | msg=Removing protocol with id [5ea0a2815e27f20010e5fe6b] | comp=IoTAgentManager
iot-agent-manager-iot-agent-manager-67d79966cb-lv5g9 iot-agent-manager (node:1) DeprecationWarning: collection.remove is deprecated. Use deleteOne, deleteMany, or bulkWrite instead.
iot-agent-manager-iot-agent-manager-67d79966cb-lv5g9 iot-agent-manager time=2021-03-03T16:35:53.208Z | lvl=DEBUG | corr=50e91161-79f9-4b69-ac84-78b4d795c3e5 | trans=50e91161-79f9-4b69-ac84-78b4d795c3e5 | op=IoTAManager.ProtocolAPI | from=n/a | srv=n/a | subsrv=n/a | msg=Protocol [5ea0a2815e27f20010e5fe6b] successfully removed. | comp=IoTAgentManager

Looks good, but... the iota is still present when querying /protocols

With the latest docker image the following error appears:

iot-agent-manager-iot-agent-manager-549547b78-2r9jr iot-agent-manager time=2021-03-03T16:44:11.094Z | lvl=DEBUG | corr=af12e7d2-2b1f-49ba-8943-51ee5459d40f | trans=af12e7d2-2b1f-49ba-8943-51ee5459d40f | op=IoTAgentNGSI.GenericMiddlewares | from=n/a | srv=n/a | subsrv=n/a | msg=Request for path [/iot/protocols/5ea0a2815e27f20010e5fe6b] from [localhost:8082] | comp=IoTAgentManager
iot-agent-manager-iot-agent-manager-549547b78-2r9jr iot-agent-manager time=2021-03-03T16:44:41.105Z | lvl=DEBUG | corr=af12e7d2-2b1f-49ba-8943-51ee5459d40f | trans=af12e7d2-2b1f-49ba-8943-51ee5459d40f | op=IoTAManager.ProtocolAPI | from=n/a | srv=n/a | subsrv=n/a | msg=Internal MongoDB Error getting device: MongooseServerSelectionError: Server selection timed out after 30000 ms
iot-agent-manager-iot-agent-manager-549547b78-2r9jr iot-agent-manager     at Function.Model.$wrapCallback (/opt/iotaManager/node_modules/mongoose/lib/model.js:4878:32)
iot-agent-manager-iot-agent-manager-549547b78-2r9jr iot-agent-manager     at /opt/iotaManager/node_modules/mongoose/lib/query.js:4372:21
iot-agent-manager-iot-agent-manager-549547b78-2r9jr iot-agent-manager     at promiseOrCallback (/opt/iotaManager/node_modules/mongoose/lib/helpers/promiseOrCallback.js:9:12)
iot-agent-manager-iot-agent-manager-549547b78-2r9jr iot-agent-manager     at model.Query.exec (/opt/iotaManager/node_modules/mongoose/lib/query.js:4371:10)
iot-agent-manager-iot-agent-manager-549547b78-2r9jr iot-agent-manager     at model.Query.Query.deleteOne (/opt/iotaManager/node_modules/mongoose/lib/query.js:2714:13)
iot-agent-manager-iot-agent-manager-549547b78-2r9jr iot-agent-manager     at Function.deleteOne (/opt/iotaManager/node_modules/mongoose/lib/model.js:1946:13)
iot-agent-manager-iot-agent-manager-549547b78-2r9jr iot-agent-manager     at Function.removeProtocol (/opt/iotaManager/lib/services/protocolData.js:165:20)
iot-agent-manager-iot-agent-manager-549547b78-2r9jr iot-agent-manager     at Object.remove (/opt/iotaManager/node_modules/iotagent-node-lib/lib/services/common/alarmManagement.js:116:25)
iot-agent-manager-iot-agent-manager-549547b78-2r9jr iot-agent-manager     at deleteProtocol (/opt/iotaManager/lib/services/protocols.js:83:15)
iot-agent-manager-iot-agent-manager-549547b78-2r9jr iot-agent-manager     at Layer.handle [as handle_request] (/opt/iotaManager/node_modules/express/lib/router/layer.js:95:5)
iot-agent-manager-iot-agent-manager-549547b78-2r9jr iot-agent-manager     at next (/opt/iotaManager/node_modules/express/lib/router/route.js:137:13)
iot-agent-manager-iot-agent-manager-549547b78-2r9jr iot-agent-manager     at Route.dispatch (/opt/iotaManager/node_modules/express/lib/router/route.js:112:3)
iot-agent-manager-iot-agent-manager-549547b78-2r9jr iot-agent-manager     at Layer.handle [as handle_request] (/opt/iotaManager/node_modules/express/lib/router/layer.js:95:5)
iot-agent-manager-iot-agent-manager-549547b78-2r9jr iot-agent-manager     at /opt/iotaManager/node_modules/express/lib/router/index.js:281:22
iot-agent-manager-iot-agent-manager-549547b78-2r9jr iot-agent-manager     at param (/opt/iotaManager/node_modules/express/lib/router/index.js:354:14)
iot-agent-manager-iot-agent-manager-549547b78-2r9jr iot-agent-manager     at param (/opt/iotaManager/node_modules/express/lib/router/index.js:365:14) {
iot-agent-manager-iot-agent-manager-549547b78-2r9jr iot-agent-manager   reason: [TopologyDescription]
iot-agent-manager-iot-agent-manager-549547b78-2r9jr iot-agent-manager } | comp=IoTAgentManager
iot-agent-manager-iot-agent-manager-549547b78-2r9jr iot-agent-manager time=2021-03-03T16:44:41.112Z | lvl=ERROR | corr=af12e7d2-2b1f-49ba-8943-51ee5459d40f | trans=af12e7d2-2b1f-49ba-8943-51ee5459d40f | op=IoTAgentNGSI.Alarms | from=n/a | srv=n/a | subsrv=n/a | msg=Raising [MONGO-ALARM]: {"name":"INTERNAL_DB_ERROR","message":"An internal DB Error happened: MongooseServerSelectionError: Server selection timed out after 30000 ms","code":500} | comp=IoTAgentManager
iot-agent-manager-iot-agent-manager-549547b78-2r9jr iot-agent-manager (node:1) [DEP0097] DeprecationWarning: Using a domain property in MakeCallback is deprecated. Use the async_context variant of MakeCallback or the AsyncResource class instead.

The MongoDb setup is a replicaset with three nodes, it worked fine with previous versions of iotam.

Even deleting non-existant iotas from iotam (1.13.0) does not return an error.
Command:

➜ curl -X DELETE localhost:8082/iot/protocols/5ea0a2815e27f20010e5fe6bAAAAAAAAA

Log:

iot-agent-manager-iot-agent-manager-67d79966cb-lv5g9 iot-agent-manager time=2021-03-03T16:37:32.231Z | lvl=DEBUG | corr=ca427141-c3a9-49a5-8d2e-76387ee0ea1e | trans=ca427141-c3a9-49a5-8d2e-76387ee0ea1e | op=IoTAgentNGSI.GenericMiddlewares | from=n/a | srv=n/a | subsrv=n/a | msg=Request for path [/iot/protocols/5ea0a2815e27f20010e5fe6bAAAAAAAAA] from [localhost:8082] | comp=IoTAgentManager
iot-agent-manager-iot-agent-manager-67d79966cb-lv5g9 iot-agent-manager time=2021-03-03T16:37:32.237Z | lvl=DEBUG | corr=ca427141-c3a9-49a5-8d2e-76387ee0ea1e | trans=ca427141-c3a9-49a5-8d2e-76387ee0ea1e | op=IoTAManager.ProtocolAPI | from=n/a | srv=n/a | subsrv=n/a | msg=Removing protocol with id [5ea0a2815e27f20010e5fe6bAAAAAAAAA] | comp=IoTAgentManager
iot-agent-manager-iot-agent-manager-67d79966cb-lv5g9 iot-agent-manager time=2021-03-03T16:37:32.241Z | lvl=DEBUG | corr=50e91161-79f9-4b69-ac84-78b4d795c3e5 | trans=50e91161-79f9-4b69-ac84-78b4d795c3e5 | op=IoTAManager.ProtocolAPI | from=n/a | srv=n/a | subsrv=n/a | msg=Protocol [5ea0a2815e27f20010e5fe6bAAAAAAAAA] successfully removed. | comp=IoTAgentManager

The text was updated successfully, but these errors were encountered:

AlvaroVega · 2021-03-04T10:20:01Z

IMHO there is few sense in delete a protocol (an IoTAgent), maybe it is better do not register that iotagent into iotagenet-manager.

But if you want it then you should provide protocol string (in your example JSON) instead internal mongo _id.

Iotagent-manager is trying to delete from Protocol schema

const Protocol = new Schema({
    iotagent: String,
    resource: String,
    protocol: String,
    description: String
});

something which maches with this condition:

 const condition = {
        protocol: id
    };

AlvaroVega · 2021-03-04T10:22:31Z

Anyway, response obtained trying delete a protocol which is not found is not proper, its a bug.
Thanks for your report @modularTaco

modularTaco · 2021-03-04T10:37:09Z

IMHO there is few sense in delete a protocol (an IoTAgent), maybe it is better do not register that iotagent into iotagenet-manager.

Hey, thanks for the quick answer!

We were using the UL and JSON IOTAgent a while ago for testing purposes, but now we decided to shut it down.
After shutting down the agents all queries to the IOTAgent-Manager were broken if no explicit protocl was specified:

➜ curl localhost:8082/iot/devices
{"name":"TARGET_SERVER_ERROR","message":"There was an error redirecting the request to the target server: Error: getaddrinfo ENOTFOUND ul-iot-agent ul-iot-agent:4041"}%

So I thought i'd need to remove the protocols from the IOTAgent-Manager

So with curl -X DELETE localhost:8082/iot/protocols/JSON it should've worked?

AlvaroVega · 2021-03-04T10:52:10Z

Yes, curl -X DELETE localhost:8082/iot/protocols/JSON will work!

fgalan · 2021-03-04T11:31:24Z

The problems with not found protocol is going to be solved in PR #235. The problem with MongoDB errors has been moved to its own issue: #226

fgalan · 2021-03-04T12:10:00Z

Merged PR #235

@modularTaco a new image will be built in dockerhub in a matter of minutes. Could test latest again to confirm the problem has been solved, please? Thanks!

modularTaco · 2021-03-04T12:27:46Z

With the current latest tag i'm not able to retrieve the protocls from IOTAM:

➜ curl -s localhost:8082/iot/protocols | jq .
{
  "name": "MongoError",
  "message": "not master and slaveOk=false"
}

iot-agent-manager-iot-agent-manager-7d78c99476-hvjj5 iot-agent-manager time=2021-03-04T12:20:58.882Z | lvl=ERROR | corr=92e2c259-92cc-4003-9219-07490093ec5a | trans=92e2c259-92cc-4003-9219-07490093ec5a | op=IoTAManager.ProtocolAPI | from=n/a | srv=n/a | subsrv=n/a | msg=error: {"topologyVersion":{"processId":"603e1b43d6bd39fad1a50d85","counter":14},"operationTime":"6935772841828679681","ok":0,"code":13435,"codeName":"NotPrimaryNoSecondaryOk","$clusterTime":{"clusterTime":"6935772841828679681","signature":{"hash":"AAAAAAAAAAAAAAAAAAAAAAAAAAA=","keyId":0}},"name":"MongoError"} | comp=IoTAgentManager

with the following config (working fine with 1.13.0):

    var config = { 
        server: {
            port: 8082
        },
        mongodb: {
            host: "mongodb-headless.platform.svc.cluster.local",
            port: 27017,
            db: "iot-agent-manager",
            replicaSet: "rs",
            retries: 5,
            retryTime: 5
        },
        logLevel: "INFO"
    };  

    module.exports = config;

When the mongodb hosts points directly to the primary node (host: "mongodb-0.mongodb-headless.platform.svc.cluster.local") it works and an error is reported if i try deleting a non-existent iota:

➜ curl -X DELETE localhost:8082/iot/protocols/JSON    
{"name":"PROTOCOL_NOT_FOUND","message":"Protocol not found for resource [] and protocol [undefined]","code":404}%

fgalan · 2021-03-04T12:29:24Z

With the current latest tag i'm not able to retrieve the protocls from IOTAM:

Dockerhub may take some time to build once the PR has been merged. Are you sure the image you are testing is the current latests and not the old one?

Just to discard a possible cause of problem...

modularTaco · 2021-03-04T12:44:54Z

With the current latest tag i'm not able to retrieve the protocls from IOTAM:

Dockerhub may take some time to build once the PR has been merged. Are you sure the image you are testing is the current latests and not the old one?

Just to discard a possible cause of problem...

it was pushed 5 min ago when it pulled, so yes, i think so

fgalan · 2021-03-04T15:07:42Z

Anyway, this error

{
  "name": "MongoError",
  "message": "not master and slaveOk=false"
}

seems more related with #236 than with #234 after the issue scope reduction.

Could you test with a MongoDB standalone instance, please.

modularTaco · 2021-03-05T07:31:19Z

Could you test with a MongoDB standalone instance, please.

I've done this already

When the mongodb hosts points directly to the primary node (host: "mongodb-0.mongodb-headless.platform.svc.cluster.local") it works and an error is reported if i try deleting a non-existent iota:
➜ curl -X DELETE localhost:8082/iot/protocols/JSON    
{"name":"PROTOCOL_NOT_FOUND","message":"Protocol not found for resource [] and protocol [undefined]","code":404}%   

AlvaroVega · 2021-03-05T08:04:33Z

I've test it several times with a non replicaset mongo and delete protocol works properly (does it when exists and report error when no)

fgalan · 2021-03-05T08:43:00Z

Related PR: #237, fixing logs.

AlvaroVega mentioned this issue Mar 4, 2021

Fix/remove non found protocol #235

Merged

fgalan mentioned this issue Mar 4, 2021

Errors in MongoDB access possible due to recent upgrade in mongoose #236

Open

AlvaroVega closed this as completed Sep 3, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cannot remove protocols #234

cannot remove protocols #234

modularTaco commented Mar 3, 2021

AlvaroVega commented Mar 4, 2021

AlvaroVega commented Mar 4, 2021 •

edited

Loading

modularTaco commented Mar 4, 2021

AlvaroVega commented Mar 4, 2021

fgalan commented Mar 4, 2021

fgalan commented Mar 4, 2021

modularTaco commented Mar 4, 2021

fgalan commented Mar 4, 2021

modularTaco commented Mar 4, 2021

fgalan commented Mar 4, 2021

modularTaco commented Mar 5, 2021

AlvaroVega commented Mar 5, 2021

fgalan commented Mar 5, 2021

cannot remove protocols #234

cannot remove protocols #234

Comments

modularTaco commented Mar 3, 2021

AlvaroVega commented Mar 4, 2021

AlvaroVega commented Mar 4, 2021 • edited Loading

modularTaco commented Mar 4, 2021

AlvaroVega commented Mar 4, 2021

fgalan commented Mar 4, 2021

fgalan commented Mar 4, 2021

modularTaco commented Mar 4, 2021

fgalan commented Mar 4, 2021

modularTaco commented Mar 4, 2021

fgalan commented Mar 4, 2021

modularTaco commented Mar 5, 2021

AlvaroVega commented Mar 5, 2021

fgalan commented Mar 5, 2021

AlvaroVega commented Mar 4, 2021 •

edited

Loading