Error in running the code run_ogindia_simple_fast.py #18
Comments
|
@Abhinav1109 The issue you are having can be seen in the line that says I had hoped to have addressed this in PR #12. Can you confirm that you are running the
If you are doing 1 and 2, let me know because I thought I had addressed this issue. If not, try to update your local files and then build the HOWEVER, if you want this to run, be sure to just do your git sync stuff to get the master changes and then build the new dictionary. This is all a little bit confusing as we move between two microsimulation models with the same name! |
|
@jdebacker @rickecon @kumar-ab @sktpru @dc4tpru Let me enumerate the steps we have taken in trying to run the package:
|
|
@Abhinav1109 It looks like you are doing all the right steps. I am not sure how your Re your demographics question, I'm not sure of the answer. @rickecon was working on this and I may have merged an incomplete set of changes into the repo. We will work to try to get this straightened out. |
|
@Abhinav1109 and @jdebacker. Some background for Abhinav is the following. We came to TPRU on August 19 with a working copy of OG-India. However, it called on the U.S. version of the
|
|
@rickecon FYI - you can choose what data file to use for the taxcalc package with the |
|
@jdebacker @rickecon @kumar-ab @sktpru @dc4tpru As per the recommendation made by @rickecon , we are trying to run the run_ogindia_*.py files that call on the Indian tax calculator. Further, we also ran a policy which is consistent with the Indian taxcalc package as defined in https://github.com/TPRU-India/taxcalc/blob/master/Budget2019_reform.json. While running this code, we encountered the error as below:
From this, it is apparent that "GSTRecords.FIELD_VARS" is not being populated properly while executing the file "gstrecords.py". We tried to debug it unsuccessfully. Our microsimulation model seems to be running fine with the same set of files/data. Could you help regarding the same. |
|
I am getting a different error trying to run the For this reason, I want to make the new files the default in the Indian |
|
I just removed all my conda environments and packages and uninstalled Anaconda and Conda. I reinstalled Anaconda and Conda. Then I just did the following steps from my local OG-India directory. Then I went to the I am still getting the same error from the Any ideas why these two new data files are not uploading with the |
|
In looking through the Note that the |
|
OK. I see that the last two lines of environment.yml have the following instruction to download Indian Why is this not downloading all the files in the Indian |
|
@rickecon I'm just not sure if this is somehow missing those data files (it sounds like it is from what you describe - the package is build, it just can't find those files). But it looks like they are in the folder the package is created from. I will say that I've only used this method to build a package once before, so I may be missing something. For the India The |
|
@jdebacker. OK, so I think those last two lines in the |
|
I think the problem is the version of Indian By the way, Indian |
|
I just submitted a PR to the Indian |
|
@rickecon @jdebacker |
|
One fix to this problem is to just build the India
I'll look more into why the |
|
FYI, this issue is waiting for PR #159 to be merged into the TPRU-India/taxcalc repo. |
|
After @sebastiansajie has merged PR #159 in the taxcalc repo, I've tested running the example script in ...cutting off a bunch of output with warnings due to poor starting values... Note that I did not continue to run through TPI, but unless the budget parameters are way off and result in an explosion of government debt, I would expect the model to solve. |
|
While trying to run the updated codes and ran the program for almost one whole day, no errors are shown but the program is entering in an infinite process as per the following.
Any suggestions as to what could the possible reasons for these warnings/messages be (computational power or some problems in initial parameters)? |
|
@Abhinav1109 I think the updates to the example script I’m proposing to add in the currently open PR should fix this. Can you try to run that script? |
|
@Abhinav1109. I just merged that PR so you can get the file by just syncing to the master. Please report back on how it works. I ran the full baseline and reform through TPI in about 4-5 hours. Everything solved. |
|
Jason, I had to make some changes to taxcalc and the program ran, though in the end I got this message:- |
|
Are there any graphs/visuals to show the results? Thanks. |
|
@sebastiansajie I'm glad you were able to run the script through to completion. Everything looks to have solved. All output is saved in directories To plot these, you can use the plotting utilities in OG-India. I noticed one typo in the dictionary used for variable labels by the plots. I just merged a PR to fix this. So to follow my example below, first:
After you do this, you can follow this example to produce a bunch of standard output plots (you can generate other by using the other functions in This will save about 20 or so plots to the Here's one of them: |
|
@jdebacker , as ipython was'nt working for me, I put those commands into a two line file and ran it and generated the plots. I also needed to add to 'def plot_all()' in output_plots.py. I shall write the detailed changes made to records.py to make the main program work. Thanks. |
|
@sebastiansajie Glad you got the plots. If you create the directory to save the plots to first, e.g., Then no need to modify Yes - open a PR to the |
|
Unfortunately, the code is still not running. Recounting the steps, they are as follows: Now, while running the updated codes, again got the error in reading GST records as below:
Since it is evident that GSTRecords could not be read, tried to open the file gst.csv in the taxcalc. However, while opening, got errors as "The file format and extension of 'file.csv' don't match. The file could be corrupted or unsafe. Unless you trust its source, don't open it. Do you want to open it anyway?" and "Excel has detected that 'file.csv' is a SYLK file, but cannot load it. Either the file has errors or it is not a SYLK file format. Click OK to try to open the file in a different format." Since this appears to be a common error in excel in files where the first column header starts with 'ID', introduced quotes around the same as below: Though this should have corrected the error, we are still getting the same error of non-availability of GSTRecords. Any workaround which could be suggested for this issue. |
|
@Abhinav1109 I heard from @sebastiansajie that he had the same error and, as he notes above, he's got some changes to the I have some guesses about the root of the problem, but they are just guesses. I look forward to @sebastiansajie opening a PR in the |
|
@Abhinav1109 and @sebastiansajie this StackOverflow thread seems to address the issue. A couple solutions: 1) rename |
|
@sebastiansajie Sir, I tried changing the order of the columns in the csv file but that did not work. Looking forward to knowing the changes you had made in the taxcalc to resolve the error. |
|
@Abhinav1109, I was surprised by the error too and had to dig into the code. These are the changes I made to both gstrecords.py and records.py respectively. PLEASE NOTE: These changes I made directly in the package (to check if it works) which is embedded deep in the directory structure in my case this was as follows:- I will ask Martin why the FIELD_VARS are not accessible outside the read_var_info function. I will then update the tax calc. gstrecords.py records.py You can make the changes in taxcalc repository too but then you would need to rebuild the package. |
|
@Abhinav1109 To be sure, after you made the changes to the To do this you want to be in the
You can then go back to If you do not have iPython, you can install by |
I tried to check why FIELD_VARS are not accessible outside the read_var_info function and had an interesting observation (I put some check print commands in the _blow_up method) (and got this output) This shows that the FIELD_VARS are accessible outside the read_var_info function also. But something goes amiss from the year 2020 onward. Will try to look into this. |
|
The code modification suggested by @sebastiansajie worked at getting past the GSTRecords error. Its now not encountering any issues on this count. However, now trying to understand the following errors i got in different runs of the same code. a. Max age (s=0) insuff. data in year 2023. Fill in final ages with insuff. data with most recent successful estimate.
|
|
@Abhinav1109 could you check the dataset that is being read. At some point I had to manually copy pitSmalldata.csv directly into the site-package. |
|
yesterday I deleted all files on my computer(MacBook) and tried to run the program after fresh cloning. I have again started it and since last 2 hours it is still running. I will update once it is over. few of the output files are made in OG-India/run_examples/OUTPUT_BASELINE and OG-India/run_examples/OUTPUT_REFORM also. I will update once it is over. |
@rickecon @jdebacker We have been trying to run the OG Model using the file run_ogindia_simple_fast.py which we created during the sessions om a different computer. However, we are encountering the error as per the attached screenshot. Could you please see what we are missing in this.
Thanks, Abhinav
run_ogindia_simple_fast.txt
The text was updated successfully, but these errors were encountered: