Skip to content

Debugger Tutorial

Jump to bottom Edit New page
David Nichols edited this page Jan 24, 2018 · 27 revisions

Introduction

Qore is a multithreaded scripting language designed for logic embedding in the Program class with sandboxing controls. The Qore debugger supports the n:m relationship between threads and Program containers, and allows programmers to control and inspect data in the container by stepping through code, setting breakpoints, running code, handling exceptions, inspecting the call stack, and reading and modifying variables among other things.

The Qore debugger solution has a client-server architecture, which allows the debugging of Qore programs over the network as well as debugging programs run locally.

Because all Qore debugging programs use a consistent command-line client implementation in the DebugCmdLine module, debugging commands are identical whether performing local or remote debugging.

Local debugging

Use the qdbg program which provides both the debug client and server and takes the script to be debugged as an argument. All following arguments belong to that program. The debug command-line client runs in a separate thread.

The syntax is:

    qdbg [args] qore_script [arg1 arg2 ...]

For example:

qdbg -pno-database test/qore/debug/test-debug-script.q arg1 test2

Remote debugging

Remote debugging is supported using the qdbg-server and qdbq-remote programs.

  • qdbg-server provides the debug server
  • qdbg-remote is the command line client to the debug server

Communication is over HTTP using the WebSocket protocol. The WebSocket protocol supports multiple clients connected to the same server, so multiple debug clients can be connected to the same debug server at any one time.

The server (qdbg-server) requires the listener socket address (ip:port) and URI path plus the Qore source to be run as arguments.

The client (qdbg-remote) requires the listener socket address (ip:port) and URI path (or optionally a full URL using the ws:// or wss:// scheme) to the debug server as an argument.

The syntax is:

    qdbg-server [args] listener script [arg1 arg2 ...]

    qdbg-remote [args] url

For example:

qdbg-server -l 127.0.0.1:40000 test/qore/debug/test-debug-script.q arg1 test2
qdbg-remote 127.0.0.1:40000

If an SSL-enabled debug server has been implemented in your own program, then the wss:// scheme must be used in the URL as in the following example:

dbg-remote wss://127.0.0.1:40001

Debugger shell

The simple shell supports autocomplete via the TAB key, command-line history, help etc.

NOTE: the Qore linenoise module is required for debugger command-line functionality

Abbreviations can be used for any command as long as the abbreviation is unique.

Type help to get a list of valid commands, session to get information about the current debugging session.

Quick Start

Start the local debugging session (qdbg) or connect to the remote server (qdbg-remote); the following table contains a subset of common commands:

Command Description
pgm list list Program containers
pgm id set Program context
pgm add set current Program as a debug target
pgm stop tells the debug server to transfer control to the debug client when the current Program executes
bt show thread backtrace for the current Program and thread
bt all show thread backtraces for all Program containers
frame id set a new stack frame location in the backtrace
thread list show all currently active threads
thread local show local variables in the current stack frame
thread local var show value of given local variable
thread local var value set new value for local variable
go release control back to program
help show help text

Program context

To get list of available Program objects use pgm list command. The result shows pgmId, the script file name, number of threads and info if the Program may be debugged, and if it is currently being debugged. Note that Program objects are by default allowed to be debugged; to disable this (for security reasons for example), use the %no-debugging parse directive (equivalent to PO_NO_DEBUGGING).

qdbg> pgm list
  1 : qdbg                   thr:  2 dbg: 0/0
  2 : DebugCmdLine.qm        thr:  1 dbg: 0/0
  3 : DebugUtil.qm           thr:  0 dbg: 0/0
  4 : Util.qm                thr:  0 dbg: 1/0
  5 : DebugProgramControl.qm thr:  2 dbg: 0/0
  6*: test-debug-script.q    thr:  1 dbg: 1/0

NOTE: As you can see from the above output, each user module has its own Program container, as does the local debugger (qdbg) and the program being debugged (test-debug-script.q).

You can also use pgm search to search for a particular Program.

For future work with a particular Program object, we need know the pgmId to set up the correct Program context.

qdbg> pgm set 6
remote program context set to "6"

To display current Program context, use

qdbg> pgm
  6 :
    scriptName : "test-debug-script.q"
    scriptPath : "test/qore/debug/test-debug-script.q"
    threads : [2]
    debugging : True
    debugged : True
    interrupted : [2]

or

qdbg> pgm get
  6

We can inspect parse directives:

qdbg> pgm options
  PO_ALLOW_BARE_REFS
  PO_ASSUME_LOCAL
  PO_NEW_STYLE

NOTE: if PO_NO_DEBUGGING is set, then the debugger cannot attach to the Program container for debugging.

qdbg> pgm set 1
remote program context set to "1"
qdbg> pgm options
  PO_NO_TOP_LEVEL_STATEMENTS
  PO_ALLOW_BARE_REFS
  PO_ASSUME_LOCAL
  PO_NEW_STYLE
  PO_NO_DEBUGGING
  PO_ALLOW_DEBUGGER

We can also get %define values

qdbg> pgm defines
  HAVE_ATOMIC_OPERATIONS : True
  HAVE_CLOSE_ALL_FD : True
  HAVE_DETERMINISTIC_GC : True
  HAVE_FILE_LOCKING : True
  HAVE_FORK : True
  HAVE_GETPPID : True
  HAVE_IS_EXECUTABLE : True
  ...

The Program context also provides info about global variables:

qdbg> pgm global ARGV
list ARGV = [{type: "string", value: "arg1"}, {type: "string", value: "test2"}]

Global variables can be modified as in the following example:

qdbg> pgm global globalString
nothing globalString = null
qdbg> pgm global globalString NEW
  ok
qdbg> pgm global globalString
string globalString = "NEW"

parse_to_qore_value() is used to parse non trivial values.

qdbg> pgm global globalAny "(a=(b=(c=x, 2, 3)))"
  ok
qdbg> pgm global globalAny                      
hash globalAny = {a: {b: [{c: "x"}, 2, 3]}}

Currently it is impossible to get/set particular values from hash or list; the entire data structure must be set instead.

Go and stop

The run command starts the Program running again until the next event is triggered, see Runstate.

qdbg> go step
2017-12-05 22:11:54.553140: run/step, pgm: 6, tid: 2
2017-12-05 22:11:54.553834: interrupt/step, pgm: 6, tid: 2, (test/qore/debug/test-debug-script.q:47, stmtid: 9)
qdbg> go step 
2017-12-05 22:11:56.469166: run/step, pgm: 6, tid: 2
2017-12-05 22:11:56.472726: interrupt/funcEnter, pgm: 6, tid: 2, (test/qore/debug/test-debug-script.q:19, stmtid: 1)
qdbg> go untilreturn
2017-12-05 22:12:18.914171: run/untilreturn, pgm: 6, tid: 2: cmd "run": action null: run "untilreturn"
2017-12-05 22:12:18.914850: interrupt/funcExit, pgm: 6, tid: 2, (test/qore/debug/test-debug-script.q:19, stmtid: 1)

If the Program is running, then it can be interrupted manually as follows:

qdbg> program stop 
ok
qdbg> program stop 2
ok

Thread context

The user must set the thread to be controlled manually as in the following example:

qdbg> thread set 2
remote thread context set to TID 2

To get the current thread context:

qdbg> thread
TID context: 2

The stack for the current thread context can be inspected as follows:

qdbg> stack
TID 2 call stack:
 - 0: TestClass::log() (test/qore/debug/test-debug-script.q:46-46 user)
 - 1: main() (test/qore/debug/test-debug-script.q:60-60 user)
 - 2: Program::run() (bin/qdbg:126-126 builtin)
 - 3: DebugWrapper::background operator() (bin/qdbg:126-126 new-thread)

The debugger can set a particular stack frame and get or set local variable values:

qdbg> frame
1
qdbg> thread local fmt
string fmt = "TestClass::log() %s
"
qdbg> frame 1
frameId set to "1"
qdbg> thread local i        
integer i = 0
qdbg> thread local i 99
i: value set to: 99
qdbg> thread local i
integer i = 99

There are special debug variables. The thread debug event command prints out current event info:

qdbg> thread debug event
  type : event
  cmd : interrupt
  pgmid : 6
  tid : 2
  stamp : 2017-12-05 21:50:33.377561 Tue +01:00 (CET)
  func : funcEnter
  runstate : step
  frame : 0
  statementid : 29
  file : "test/qore/debug/test-debug-script.q"
  line : 14

The flow variable can change the flow for loops and functions in block/step events; this variable can be examined or set; see Flow control for more information:

qdbg> thread debug flow
  none
qdbg> thread debug flow break
qdbg> thread debug flow
  break
qdbg> go run

The dismiss variable is available in exception events. If true, then the flow will go on as if the event did not happened:

2017-12-05 22:27:17.300077: interrupt/exception, pgm: 6, tid: 2, (test/qore/debug/test-debug-script.q:53, stmtid: 16)
qdbg> go step
2017-12-05 22:28:19.637586: run/step, pgm: 6, tid: 2
2017-12-05 22:28:19.640177: run/step, pgm: 6, tid: 2, (test/qore/debug/test-debug-script.q:55, stmtid: 17)
qdbg> thread local ex
hash ex = {type: {type: "string", value: "System"}, file: {type: "string", value: "test/qore/debug/test-debug-script.q"}, line: {type: "integer", value: 53}, endline: {type: "integer", value: 53}, source: {type: "string", value: ""}, offset: {type: "integer", value: 0}, callstack: {type: "list", value: []}, err: {type: "string", value: "DIVISION-BY-ZERO"}, desc: {type: "string", value: "division by zero found in integer expression"}

Setting dismiss to true skips the exception block.

NOTE: There is a limitation that at this time the exception info is not available

2017-12-05 22:31:14.733864: interrupt/exception, pgm: 6, tid: 2, (test/qore/debug/test-debug-script.q:53, stmtid: 16)
qdbg> thread debug dismiss true
qdbg> thread debug dismiss     
  true
qdbg> go step
2017-12-05 22:31:40.521728: run/step, pgm: 6, tid: 2
2017-12-05 22:31:40.522376: interrupt/step, pgm: 6, tid: 2, (test/qore/debug/test-debug-script.q:57, stmtid: 19)

The thread debug result command provides get/set access for the value to be returned from the user function or method. It is available in funcEvent only.

2017-12-05 22:24:18.921484: interrupt/funcExit, pgm: 6, tid: 2, (test/qore/debug/test-debug-script.q:28, stmtid: 20)
qdbg> thread debug result
string result = "F-0"
qdbg> thread debug result F-1
  ok
qdbg> thread debug result
string result = "F-1"

Breakpoints

Breakpoints can be used to run a Program and until the Program flow reaches a breakpoint. In this case the Program is interrupted; an event is triggered, and control passes to the debugger.

The breakpoint itself consists of a list of statements. The statement is the location in the Qore source code, i.e. the reference is the file name and the source line number. The file name can be omitted when there is only one file.

Breakpoints also have properties to control their behavior as follows:

  • id: unique generated identifier
  • enable: enable or disable the breakpoint
  • policy: for multithreaded Program objects: the policy is applied with the thread list to select which threads the breakpoint belongs to:
    • none: does not provide any thread selection, i.e. the breakpoint is assigned to all threads
    • accept: the threads enumerated in the list are assigned to the breakpoint
    • reject: the threads not enumerated in the list are assigned to the breakpoint
  • thread: list of threads

To create a breakpoint for line 35 and run the Program:

qdbg> break create 35
ok
qdbg> break list
  id: 1     pgmId: 6    enabled: true  policy: none   threads: null
    @ 28    test/qore/debug/test-debug-script.q:35-35

qdbg> go run
2017-12-05 22:58:11.538881: run/run, pgm: 6, tid: 2

2017-12-05 22:58:11.542634: interrupt/step, pgm: 6, tid: 2, (test/qore/debug/test-debug-script.q:35, stmtid: 28)

To assign breakpoint for specific function or method add parenthesis with argument types to specify potentially overload function or method:

qdbg> statement assign 1 main()
ok
qdbg> break list
  id: 1     pgmId: 6    enabled: true  policy: none   threads: null
    @ 28    test/qore/debug/test-debug-script.q:35-35
    @ 3     test/qore/debug/test-debug-script.q:39-57

qdbg> statement assign 1 main(string,int)
cmd "program/6/breakpoint/1/statement/assign" server-side error: RUNTIME-OVERLOAD-ERROR: no variant matching 'main(string, int)' can be found; the following variants are defined:
main()

qdbg> statement assign 1 TestClass::log()
cmd "program/6/breakpoint/1/statement/assign" server-side error: RUNTIME-OVERLOAD-ERROR: no variant matching 'TestClass::log())' can be found; the following variants are defined:
TestClass::log(string fmt)
qdbg> statement assign 1 TestClass::log(string)
ok
qdbg> break list
  id: 1     pgmId: 6    enabled: true  policy: none   threads: null
    @ 28    test/qore/debug/test-debug-script.q:35-35
    @ 3     test/qore/debug/test-debug-script.q:39-57
    @ 29    test/qore/debug/test-debug-script.q:14-14

To activate the breakpoint only for thread 2:

qdbg> break policy 1 accept
ok
qdbg> break thread 1 2
ok
qdbg> break list
  id: 1     pgmId: 6    enabled: true  policy: accept threads: [2]
    @ 28    test/qore/debug/test-debug-script.q:35-35
    @ 3     test/qore/debug/test-debug-script.q:39-57
    @ 29    test/qore/debug/test-debug-script.q:14-14

To disable the breakpoint:

qdbg> break disable 1
ok
qdbg> break list
  id: 1     pgmId: 6    enabled: false policy: accept threads: [2]
    @ 28    test/qore/debug/test-debug-script.q:35-35
    @ 3     test/qore/debug/test-debug-script.q:39-57
    @ 29    test/qore/debug/test-debug-script.q:14-14

Create breakpoint for another module, i.e. program. The program must be allowed for debugging.

qdbg> pgm 4
remote program context set to "4"
qdbg> break create Util::compare_version(string,string)
ok
qdbg> break list
id: 2     pgmId: 4    enabled: true  policy: none   threads: null
@ 209   /usr/local/share/qore-modules/0.8.13/Util.qm:165-197

To list all breakpoints:

qdbg> break list all
id: 1     pgmId: 6    enabled: false policy: accept threads: [2]
@ 28    test/qore/debug/test-debug-script.q:35-35
@ 3     test/qore/debug/test-debug-script.q:39-57
@ 29    test/qore/debug/test-debug-script.q:14-14

id: 2     pgmId: 4    enabled: true  policy: none   threads: null
@ 209   /usr/local/share/qore-modules/0.8.13/Util.qm:165-197

To remove a statement from the breakpoint (switching program context is not necessary as the breakpoint is assigned to program):

qdbg> break list all
id: 1     pgmId: 6    enabled: false policy: accept threads: [2]
@ 3     test/qore/debug/test-debug-script.q:39-57
@ 29    test/qore/debug/test-debug-script.q:14-14

id: 2     pgmId: 4    enabled: true  policy: none   threads: null
@ 209   /usr/local/share/qore-modules/0.8.13/Util.qm:165-197

Extras

The help command also supports code completion and can provide additional information such as command aliases.

qdbg> help
Command root list: break, bt (backtrace, stack), cmd, frame, go, help, history, pgm (program), quit, session, stmt (statement), thread, version
qdbg> help break
break: breakpoint related commands

Subcommand list: create, delete, disable, enable, list, policy, statement, thread

To test a connection with a server use the version command. If the connection has been broken, then there is no host in the output.

qdbg> version
  client :
    class : DebugCommandLineRemote
  host :
    class : WebSocketDebugProgramControlHost

The session command provides an overview about the current session and also offers the possibility of saving the session to an external file and restoring it later.

NOTE: The session data saved includes the pgmId, breakpointId, filename, line number etc, which are only valid for a single process; if the session is restored in a new process, then these values will no longer be valid.

qdbg> session 
+ "test-debug-script.q" (program 6, 1 blocked 1 thread, 1 breakpoint (enabled: 0))
  - blocked 1 TID: 2

The **cmd** command sends a raw debug API command to the server. The command argument has the form of a path with parameters included.

qdbg> cmd version
Result: {class: "DebugProgramControlLocal"}
qdbg> cmd program//list
Result: {1: {scriptName: "qdbg", scriptPath: "bin/qdbg", threads: [1, 2], debugging: False}, 2: {scriptName: "DebugCmdLine.qm", scriptPath: "/home/tma/work/qore/qore/qlib/DebugCmdLine.qm", threads: [1], debugging: False}, 3: {scriptName: "DebugUtil.qm", scriptPath: "/home/tma/work/qore/qore/qlib/DebugUtil.qm", threads: [], debugging: False}, 4: {scriptName: "Util.qm", scriptPath: "/usr/local/share/qore-modules/0.8.13/Util.qm", threads: [], debugging: True}, 5: {scriptName: "DebugProgramControl.qm", scriptPath: "/home/tma/work/qore/qore/qlib/DebugProgramControl.qm", threads: [1, 2], debugging: False}, 7: {scriptName: "test-debug-script.q", scriptPath: "test/qore/debug/test-debug-script.q", threads: [2], debugging: True, debugged: True, interrupted: [2]}}

Thread-related commands require parameters not passed via command path.

qdbg> cmd thread/stack     
Result: {err: "DEBUG-HANDLER-ERROR", desc: "cannot process command \"thread/stack\""}
qdbg> cmd thread/stack "(tid=2)"
Result: [{function: "interrupt: attach", line: 128, endline: 128, file: "bin/qdbg", source: null, offset: 0, typecode: 0, type: "user", current: True}, {function: "Program::run", line: 128, endline: 128, file: "bin/qdbg", source: null, offset: 0, typecode: 1, type: "builtin"}, {function: "DebugWrapper::background operator", line: 128, endline: 128, file: "bin/qdbg", source: null, offset: 0, typecode: 2, type: "new-thread"}]

The situation is similar when setting a variable value using raw API commands:

qdbg> pgm
  7 :
    scriptName : "test-debug-script.q"
    scriptPath : "test/qore/debug/test-debug-script.q"
    threads : [2]
    debugging : True
    debugged : True
    interrupted : [2]
qdbg> cmd program/7/global/globalAny/set "(value=1)"
Result: "ok"
qdbg> cmd program/7/global/globalAny/get            
Result: {type: "integer", value: 1}
qdbg> pgm global globalAny
integer globalAny = 1

Internal Implementation

The server intercepts events at the Qore statement level and passes these events to the debug client. This will happen if a Program object has been explicitly added to list of Program objects being debugged.

When Program objects have been added as debugging targets and have been set to change control to the debugger, then, when they are executed, the client is notified before the next Qore statement is executed in any thread. The Program is interrupted; the client is notified with an attach event, and the server waits for a response that determines the Runstate which controls how the server will continue further processing.

Events

  • onAttach: executed once for every thread on the first Qore statement after the Program is registered for debugging
  • onBlock: executed when a Qore block is entered. An onStep event is executed for each block statement
  • onStep: executed before a statement is executed
  • onFunctionEnter: executed when a user function, method, or closure is entered
  • onFunctionExit: executed when a user function, method, or closure is exited. If the function returns a value, then the client may inspect/change the result value
  • onException: executed when an exception is raised. ExceptionInfo is available and the client may dismiss the exception, i.e. the Program will continue as if the exception had not been raised.
  • onDetach: executed when the server is detached from the Program

Runstate

  • detach: debugger detaches from the Program, i.e. the Program is left running and is no longer debugged
  • run: the Program is left running but can be interrupted manually, at a breakpoint or when an exception is raised
  • step: execute the next Qore statement and break again
  • stepover: execute the next Qore statement and break again but skip any procedure calls
  • untilreturn: execute until the Program flow leaves the Qore procedure

Flow control

When debugging in a loop, changing the control flow from the debugger affects how the loop is executed (ex: can be used to break a never-ending loop).

  • none: the Program will continue with the next regular statement
  • return: the Program will exit the current procedure
  • break: the Program will break the current loop
  • continue: the Program will continue with the next iteration
Add a custom footer
Add a custom sidebar
Clone this wiki locally