-
Notifications
You must be signed in to change notification settings - Fork 6
/
Copy pathUsing_Exceptions_for_Debugging.txt
289 lines (234 loc) · 13.5 KB
/
Using_Exceptions_for_Debugging.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
Using Exception information for Debugging Python parser
Summary for the Impatient:
There is NO final resolution here.
The defects found led me to believe that some
non trivial refactoring of parsing Comments and string literals is needed.
I wanted to relase the rest of the Thread change for UCC so
refactoring Python parser will wait.
===============================================================================
Below is an example of a Thread catching an Exception parsing a Python file
UCC.exe -threads 2 -nowarnings -dir "C:\C++\boost_1_48_0\libs"
-outdir C:\Utilities\TEST\UCC\Files_OUT -ascii
2 Threads started.
Optimized processing in use ...
Building list of source files...
Read, Analyze & Count keywords in files..DONE
Thread 1 Error: there was a Out of Range error exception.
invalid string position
Error: Unable to count file (C:\C++\boost_1_48_0\libs\python\pyste\src\Pyste\ClassExporter.py)
Looking for duplicate files..............DONE
Generating results to files..............DONE
21285 files for 52.0 files processed per second
Processing Step : seconds
------------------------- : --------
Build file list : 13
Read, Analyze & Count : 126
Find duplicates : 268
Generate results files : 1
Total time : 6 minutes 49 seconds
Some initial thoughts...
Was this the only severe error with Analyzing a file or did others happen that were not shown?
Only 1, no others.
Did the complete Error message get put into the errors file?
Yes.
So after looking at the error_log_06032015_090738.txt
it looks like all the other files were processed OK (there were some Warnings about Empty or Truncated)
so the above statistics are valid.
First thing to do for Debugging is to simplify the conditions to reproduce a defect.
VERY IMPORTANT:
Unless you strongly suspect a runtime error with the Thread interaction code itself:
Change to NOT use threads or to ONLY use
-threads 1
This will take a whole level of excitement of multiple threads out of the Debug process.
Also since this seems to be just 1 file without any complex interactions to get the defect:
(for instance not like running low on Memory first)
just copy the file elsewhere for the Debug session.
Below is a screen capture of a DEBUG build running on a Windows console window.
===============================================================================
Here is where you can attach a Debugger.
Hit Enter key to continue running in UCC main()
Optimized processing in use ...
Building list of source files...
Read, Analyze & Count keywords in files......Thread 81 Error: there was a Out of
Range error exception.
invalid string position
Error: Unable to count file (C:\Utilities\Test\UCC\Files_IN\ClassExporter.py)
DONE
Thread 81 Error: there was a Out of Range error exception.
invalid string position
Error: Unable to count file (C:\Utilities\Test\UCC\Files_IN\ClassExporter.py)
Looking for duplicate files..............DONE
Generating results to files..............DONE
4 files for 2.7 files processed per second
Processing Step : seconds
------------------------- : --------
Build file list : 0
Read, Analyze & Count : 1
Find duplicates : 0
Generate results files : 0
Total time : 1 seconds
Finished. Hit Enter key and this will exit.
===============================================================================
OK. I would rather be told 2 times about a severe error than not at all.
The beginning and end allow the SW Developer to attach a Debugger if wanted
and to capture the text output if wanted before exit.
Notice that Thread 81 is shown.
Because of a define in UCCThread.h where the thread maximum count is 80
running from the main thread the logical thread index is set to 81
Changed CPythonCounter.cpp CPythonCounter::CountDirectiveSLOC
to put a line number out for each line went up to 918 lines
as you can see about 1/2 way down.
This matches the expected line count.
Next breakpoint was CPythonCounter::LanguageSpecificProcess
added line number output as well with blank space after for easier reading
Optimized processing in use ...
Building list of source files...
Read, Analyze & Count keywords in files......12345678910111213141516171819202122
23242526272829303132333435363738394041424344454647484950515253545556575859606162
63646566676869707172737475767778798081828384858687888990919293949596979899100101
10210310410510610710810911011111211311411511611711811912012112212312412512612712
81291301311321331341351361371381391401411421431441451461471481491501511521531541
55156157158159160161162163164165166167168169170171172173174175176177178179180181
18218318418518618718818919019119219319419519619719819920020120220320420520620720
82092102112122132142152162172182192202212222232242252262272282292302312322332342
35236237238239240241242243244245246247248249250251252253254255256257258259260261
26226326426526626726826927027127227327427527627727827928028128228328428528628728
82892902912922932942952962972982993003013023033043053063073083093103113123133143
15316317318319320321322323324325326327328329330331332333334335336337338339340341
34234334434534634734834935035135235335435535635735835936036136236336436536636736
83693703713723733743753763773783793803813823833843853863873883893903913923933943
95396397398399400401402403404405406407408409410411412413414415416417418419420421
42242342442542642742842943043143243343443543643743843944044144244344444544644744
84494504514524534544554564574584594604614624634644654664674684694704714724734744
75476477478479480481482483484485486487488489490491492493494495496497498499500501
50250350450550650750850951051151251351451551651751851952052152252352452552652752
85295305315325335345355365375385395405415425435445455465475485495505515525535545
55556557558559560561562563564565566567568569570571572573574575576577578579580581
58258358458558658758858959059159259359459559659759859960060160260360460560660760
86096106116126136146156166176186196206216226236246256266276286296306316326336346
35636637638639640641642643644645646647648649650651652653654655656657658659660661
66266366466566666766866967067167267367467567667767867968068168268368468568668768
86896906916926936946956966976986997007017027037047057067077087097107117127137147
15716717718719720721722723724725726727728729730731732733734735736737738739740741
74274374474574674774874975075175275375475575675775875976076176276376476576676776
87697707717727737747757767777787797807817827837847857867877887897907917927937947
95796797798799800801802803804805806807808809810811812813814815816817818819820821
82282382482582682782882983083183283383483583683783883984084184284384484584684784
88498508518528538548558568578588598608618628638648658668678688698708718728738748
75876877878879880881882883884885886887888889890891892893894895896897898899900901
9029039049059069079089099109119129139149159169179181 2 3 4 5 6 7 8 9 10 11 12 13 <<< End of CountDirectiveSLOC,
14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 4 LanguageSpecificProcess showing Python file line numbers
0 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66
67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93
94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 1
15 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 1
35 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 1
55 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 1
75 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 1
95 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 2
15 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 2
35 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 2
55 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 2
75 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 2
95 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 3
15 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 3
35 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 3
55 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 3
75 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 3
95 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 4
15 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 4
35 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 4
55 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 4
75 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 4
95 496 497 Thread 81 Error: there was a Out of Range error exception. <<< and we got to line 497 and then the Exception
invalid string position
Error: Unable to count file (C:\Randy\Programming\Utilities\Test\UCC\Files_IN\Cl
assExporter.py)
DONE
Thread 81 Error: there was a Out of Range error exception.
invalid string position
Error: Unable to count file (C:\Randy\Programming\Utilities\Test\UCC\Files_IN\Cl
assExporter.py)
Optimized processing in use ...
Building list of source files...
Read, Analyze & Count keywords in files......Starting Detailed DEBUG <<< message I added to get a Breakpoint where I wanted
line "$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$'self_ns::'" std::basic_string
lineBak " namespace = namespaces.python + 'self_ns::'" std::basic_string
Comment from start of LanguageSpecificProcess
* Processes physical and logical lines according to language specific rules.
* NOTE: all the blank lines +
* whole line comments +
* lines with compiler directives
* should have been blanked from filemap by previous processing
* before reaching this function
This explains the difference between line and lineBak
So I traced for a bit and got inside LSLOC
// 2. if no parentheses enclosing, and if the char is a statement separator
if (paren_cnt == 0 && (line[i] == ';' || line[i] == ':')) <<< This is looking for Python statement code NOT a string literal !
{
tmp = CUtil::ClearRedundantSpaces(line);
// if line[..i] is else: then exit the outer if
if (tmp.rfind("else:") != tmp.length() - 5)
{
// 3.
strSize = CUtil::TruncateLine(i + 1 - start, strLSLOC.length(), this->lsloc_truncate, trunc_flag);
if (strSize > 0)
{
strLSLOC += line.substr(start, i);
strLSLOCBak += lineBak.substr(start, i);
}
if (result->addSLOC(strLSLOCBak, trunc_flag)) <<< This is where I traced until the light bulb went on...
Looking at the backup (unchanged) line we see an assignment statement with concatenation of a string literal at the end.
The string literal is
'self_ns::'
and the value of line[i] did indeed refer to the first : character in the literal.
So the problem happens before parsing gets here.
Now the problem appears that a Python string literal enclosed by single quote characters was not recognized earlier.
Because the line data is from before any detailed processing in LanguageSpecificProcess happens
it is reasonable to see how the line 497 is processed in CountDirectiveSLOC
line "$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$'self_ns::'"
This is the same as passed to LanguageSpecificProcess
Backing up to CPythonCounter::CountCommentsSLOC
{lineNumber=497 line=" namespace = namespaces.python + 'self_ns::'" }
for (filemap::iterator iter = fmap->begin(); iter != fmap->end(); iter++, itfmBak++)
{
#ifdef _DEBUG
line_number++;
// cout << line_number << " ";
if ( 497 == line_number )
{
cout << "Starting CountCommentsSLOC Detailed DEBUG" << endl;
}
#endif
contd_nextline = false;
quote_idx_start = 0;
idx_start = 0;
if (CUtil::CheckBlank(iter->line))
continue;
if (quote_contd) <<< Before call this is true; meaning a quote continued from a prior line
{
// Replace quote until next character
ReplaceQuote(iter->line, quote_idx_start, quote_contd, CurrentQuoteEnd); <<< AFTER call line is changed, CurrentQuoteEnd = ' WRONG !
line "$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$'self_ns::'"
NOW we know the line really should be
line " namespace = namespaces.python + '$$$$$$$$$'"
So on some previous line the code parsed a single quote and kept going (wiping out perfectly valid Python code) until line 497
Looking around some more:
QuoteStart = "\"'"; // used to be QuoteStart = "\"\'";
QuoteEnd = "\"'"; // used to be QuoteEnd = "\"\'";
Were declared wrong for single quote chars (did not need \ to escape them)
Recompiling and running again did NOT fix the problem.
I saw that the contd flag was true when it should have been false.
I then decided to use the Debugger and show the lines from the fmap and started scrolling from the start.
I was looking for when questionable replacement of literals with $$$$$
line 440 and 441
BOOST_SUPPORTED_OPERATORS = '+ - * / % ^ & ! ~ | < > == != <= >= << >> && || += -= '\
'*= /= %= ^= &= |= <<= >>='.split()
became
BOOST_SUPPORTED_OPERATORS = '$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$'\
$$$$$$$$'*= /= %= ^= &= |= <<= >>='$$$$$$$$
440 is OK but 441 is the start of being out of step. Time for some more Debugging...
More debugging led me into thinking that to really fix this is not a small low risk change.
So this is a change I do not want to hurry with just to put into Thread release.
Later...
Randy Maxwell