-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathatom.xml
194 lines (112 loc) · 118 KB
/
atom.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<title>Jinge Li's Blog</title>
<subtitle>Hacking and Stargazing</subtitle>
<link href="https://chinapandaman.github.io/atom.xml" rel="self"/>
<link href="https://chinapandaman.github.io/"/>
<updated>2024-01-15T21:47:35.263Z</updated>
<id>https://chinapandaman.github.io/</id>
<author>
<name>Jinge Li</name>
</author>
<generator uri="https://hexo.io/">Hexo</generator>
<entry>
<title>CI/CD in PyPDFForm</title>
<link href="https://chinapandaman.github.io/2024/01/15/pypdfform-blog-1/"/>
<id>https://chinapandaman.github.io/2024/01/15/pypdfform-blog-1/</id>
<published>2024-01-15T21:47:39.000Z</published>
<updated>2024-01-15T21:47:35.263Z</updated>
<content type="html"><![CDATA[<h3 id="Preface"><a href="#Preface" class="headerlink" title="Preface"></a>Preface</h3><p>Happy new year fellow developers! The <a href="https://youtu.be/8t1RdAKwr9w?si=AcsOtfFHjuuoVOXC">speak</a> I did at the Chicago Python User Group was a huge success and I hope everyone enjoyed it. I’d like to start a series of blogs which will cover a number of subjects related to <a href="https://github.com/chinapandaman/PyPDFForm">PyPDFForm</a>. These subjects will hopefully cover some of the more in depth technical details that I didn’t get to cover during my speak.</p><p>In this first blog, I want to talk about all the CI/CD pipelines I have set up for the project that ensured its stability and automation. These pipelines are also in my opinion quite general purpose and can be used by any Python project in the future.</p><span id="more"></span><h3 id="Test-Automation"><a href="#Test-Automation" class="headerlink" title="Test Automation"></a>Test Automation</h3><p>PyPDFForm uses GitHub actions for all its CI/CD pipelines. When it comes to continuous integration, the first one that pretty much any project should have is build and test automation. PyPDFForm implements <a href="https://github.com/chinapandaman/PyPDFForm/tree/master/tests">all its tests</a> using <a href="https://pypi.org/project/pytest/">pytest</a> and since Python is an interpreted language it doesn’t need a lot of build steps other than downloading dependencies. The <a href="https://github.com/chinapandaman/PyPDFForm/blob/master/.github/workflows/python-package.yml">following action</a> defines the pipeline for PyPDFForm’s test automation:</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">name:</span> <span class="string">Tests</span></span><br><span class="line"></span><br><span class="line"><span class="attr">on:</span></span><br><span class="line"> <span class="attr">push:</span></span><br><span class="line"> <span class="attr">branches:</span> [ <span class="string">master</span> ]</span><br><span class="line"> <span class="attr">pull_request:</span></span><br><span class="line"> <span class="attr">branches:</span> [ <span class="string">master</span> ]</span><br><span class="line"></span><br><span class="line"><span class="attr">jobs:</span></span><br><span class="line"> <span class="attr">build:</span></span><br><span class="line"></span><br><span class="line"> <span class="attr">runs-on:</span> <span class="string">${{</span> <span class="string">matrix.os</span> <span class="string">}}</span></span><br><span class="line"> <span class="attr">strategy:</span></span><br><span class="line"> <span class="attr">matrix:</span></span><br><span class="line"> <span class="attr">os:</span> [<span class="string">ubuntu-latest</span>, <span class="string">windows-latest</span>, <span class="string">macos-latest</span>]</span><br><span class="line"> <span class="attr">python-version:</span> [<span class="number">3.7</span>, <span class="number">3.8</span>, <span class="number">3.9</span>, <span class="string">"3.10"</span>, <span class="number">3.11</span>, <span class="number">3.12</span>]</span><br><span class="line"></span><br><span class="line"> <span class="attr">steps:</span></span><br><span class="line"> <span class="bullet">-</span> <span class="attr">uses:</span> <span class="string">actions/checkout@v3</span></span><br><span class="line"> <span class="bullet">-</span> <span class="attr">name:</span> <span class="string">Set</span> <span class="string">up</span> <span class="string">Python</span> <span class="string">${{</span> <span class="string">matrix.python-version</span> <span class="string">}}</span></span><br><span class="line"> <span class="attr">uses:</span> <span class="string">actions/setup-python@v4</span></span><br><span class="line"> <span class="attr">with:</span></span><br><span class="line"> <span class="attr">python-version:</span> <span class="string">${{</span> <span class="string">matrix.python-version</span> <span class="string">}}</span></span><br><span class="line"> <span class="bullet">-</span> <span class="attr">name:</span> <span class="string">Install</span> <span class="string">dependencies</span></span><br><span class="line"> <span class="attr">run:</span> <span class="string">|</span></span><br><span class="line"><span class="string"> python -m pip install --upgrade pip</span></span><br><span class="line"><span class="string"> pip install -r requirements.txt</span></span><br><span class="line"><span class="string"></span> <span class="bullet">-</span> <span class="attr">name:</span> <span class="string">Test</span> <span class="string">with</span> <span class="string">pytest</span></span><br><span class="line"> <span class="attr">run:</span> <span class="string">|</span></span><br><span class="line"> <span class="string">pytest</span></span><br></pre></td></tr></table></figure><p>Let’s look at this block by block to get a better understanding of it. First:</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">on:</span></span><br><span class="line"> <span class="attr">push:</span></span><br><span class="line"> <span class="attr">branches:</span> [ <span class="string">master</span> ]</span><br><span class="line"> <span class="attr">pull_request:</span></span><br><span class="line"> <span class="attr">branches:</span> [ <span class="string">master</span> ]</span><br></pre></td></tr></table></figure><p>This chunk defines the condition that triggers this action. In this case we want to run our tests when there’s a PR created or if a commit is made directly towards the <code>master</code> branch.</p><p>This next section defines all the environments that the tests should be run on:</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">runs-on:</span> <span class="string">${{</span> <span class="string">matrix.os</span> <span class="string">}}</span></span><br><span class="line"><span class="attr">strategy:</span></span><br><span class="line"> <span class="attr">matrix:</span></span><br><span class="line"> <span class="attr">os:</span> [<span class="string">ubuntu-latest</span>, <span class="string">windows-latest</span>, <span class="string">macos-latest</span>]</span><br><span class="line"> <span class="attr">python-version:</span> [<span class="number">3.7</span>, <span class="number">3.8</span>, <span class="number">3.9</span>, <span class="string">"3.10"</span>, <span class="number">3.11</span>, <span class="number">3.12</span>]</span><br></pre></td></tr></table></figure><p>GitHub action has a very nice way of defining environments with the matrix strategy. In this case I would like to test PyPDFForm on all three operating systems. For each os I want to test all the currently supported Python versions. Overall these environment definitions will kick off 3x6=18 different builds when this action is invoked.</p><p><img src="/2024/01/15/pypdfform-blog-1/1.png"></p><p>Now the actual pipeline itself is divided into four steps. The first two steps simply checkout the source code and setup the Python environment, both of which use other existed actions:</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="bullet">-</span> <span class="attr">uses:</span> <span class="string">actions/checkout@v3</span></span><br><span class="line"><span class="bullet">-</span> <span class="attr">name:</span> <span class="string">Set</span> <span class="string">up</span> <span class="string">Python</span> <span class="string">${{</span> <span class="string">matrix.python-version</span> <span class="string">}}</span></span><br><span class="line"> <span class="attr">uses:</span> <span class="string">actions/setup-python@v4</span></span><br><span class="line"> <span class="attr">with:</span></span><br><span class="line"> <span class="attr">python-version:</span> <span class="string">${{</span> <span class="string">matrix.python-version</span> <span class="string">}}</span></span><br></pre></td></tr></table></figure><p>Note the <code>python-version</code> references the <code>python-verion</code> environment definition from the matrix.</p><p>The next step is to install the dependencies. Python unlike some other compiled languages doesn’t need a build process most of the time. So a simple <code>pip install</code> is enough for this step:</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="bullet">-</span> <span class="attr">name:</span> <span class="string">Install</span> <span class="string">dependencies</span></span><br><span class="line"> <span class="attr">run:</span> <span class="string">|</span></span><br><span class="line"><span class="string"> python -m pip install --upgrade pip</span></span><br><span class="line"><span class="string"> pip install -r requirements.txt</span></span><br></pre></td></tr></table></figure><p>And finally we get to actually run the tests, which is just a simple <code>pytest</code> command:</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="bullet">-</span> <span class="attr">name:</span> <span class="string">Test</span> <span class="string">with</span> <span class="string">pytest</span></span><br><span class="line"> <span class="attr">run:</span> <span class="string">|</span></span><br><span class="line"> <span class="string">pytest</span></span><br></pre></td></tr></table></figure><p><img src="/2024/01/15/pypdfform-blog-1/2.png"></p><p>Now we have a pretty standard action for running automated tests.</p><h3 id="Code-Coverage"><a href="#Code-Coverage" class="headerlink" title="Code Coverage"></a>Code Coverage</h3><p>The action for code coverage generation used to be part of the test automation action discussed in the last section. It was changed and became its own action in <a href="https://github.com/chinapandaman/PyPDFForm/pull/397">this PR</a>.</p><p>The reason behind this is exactly what the title of the PR says. It is meant to reduce the number of times code coverage is uploaded to Codecov. There seems to be an issue either on GitHub or Codecov where too many coverage uploads will start causing API rate limits, described in <a href="https://github.com/codecov/feedback/issues/126">this issue</a>.</p><p>With the old way, every time the test action is run there will be eighteen submissions of coverage to Codecov and it has caused me the above issue. So now code coverage is its own separate <a href="https://github.com/chinapandaman/PyPDFForm/blob/master/.github/workflows/python-coverage.yml">action</a>:</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">name:</span> <span class="string">Coverage</span></span><br><span class="line"></span><br><span class="line"><span class="attr">on:</span></span><br><span class="line"> <span class="attr">push:</span></span><br><span class="line"> <span class="attr">branches:</span> [ <span class="string">master</span> ]</span><br><span class="line"> <span class="attr">pull_request:</span></span><br><span class="line"> <span class="attr">branches:</span> [ <span class="string">master</span> ]</span><br><span class="line"></span><br><span class="line"><span class="attr">jobs:</span></span><br><span class="line"> <span class="attr">build:</span></span><br><span class="line"></span><br><span class="line"> <span class="attr">runs-on:</span> <span class="string">ubuntu-latest</span></span><br><span class="line"></span><br><span class="line"> <span class="attr">steps:</span></span><br><span class="line"> <span class="bullet">-</span> <span class="attr">uses:</span> <span class="string">actions/checkout@v3</span></span><br><span class="line"> <span class="bullet">-</span> <span class="attr">name:</span> <span class="string">Set</span> <span class="string">up</span> <span class="string">Python</span></span><br><span class="line"> <span class="attr">uses:</span> <span class="string">actions/setup-python@v4</span></span><br><span class="line"> <span class="attr">with:</span></span><br><span class="line"> <span class="attr">python-version:</span> <span class="string">'3.x'</span></span><br><span class="line"> <span class="bullet">-</span> <span class="attr">name:</span> <span class="string">Install</span> <span class="string">dependencies</span></span><br><span class="line"> <span class="attr">run:</span> <span class="string">|</span></span><br><span class="line"><span class="string"> python -m pip install --upgrade pip</span></span><br><span class="line"><span class="string"> pip install -r requirements.txt</span></span><br><span class="line"><span class="string"></span> <span class="bullet">-</span> <span class="attr">name:</span> <span class="string">Test</span> <span class="string">with</span> <span class="string">pytest</span></span><br><span class="line"> <span class="attr">run:</span> <span class="string">|</span></span><br><span class="line"><span class="string"> coverage run -m pytest</span></span><br><span class="line"><span class="string"></span> <span class="bullet">-</span> <span class="attr">name:</span> <span class="string">Upload</span> <span class="string">coverage</span> <span class="string">to</span> <span class="string">Codecov</span></span><br><span class="line"> <span class="attr">uses:</span> <span class="string">codecov/codecov-action@v3</span></span><br></pre></td></tr></table></figure><p>This action is very similar to the test automation action, with two major differences. First:</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">runs-on:</span> <span class="string">ubuntu-latest</span></span><br></pre></td></tr></table></figure><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="bullet">-</span> <span class="attr">name:</span> <span class="string">Set</span> <span class="string">up</span> <span class="string">Python</span></span><br><span class="line"> <span class="attr">uses:</span> <span class="string">actions/setup-python@v4</span></span><br><span class="line"> <span class="attr">with:</span></span><br><span class="line"> <span class="attr">python-version:</span> <span class="string">'3.x'</span></span><br></pre></td></tr></table></figure><p>With the above environment definition, this action will only run once on the latest version of Ubuntu, using the latest version of Python. This means it will only upload the code coverage once, having a much less chance of triggering the rate limits.</p><p>And finally there’s an extra step after the tests are run that uploads the coverage to Codecov:</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="bullet">-</span> <span class="attr">name:</span> <span class="string">Upload</span> <span class="string">coverage</span> <span class="string">to</span> <span class="string">Codecov</span></span><br><span class="line"> <span class="attr">uses:</span> <span class="string">codecov/codecov-action@v3</span></span><br></pre></td></tr></table></figure><p><img src="/2024/01/15/pypdfform-blog-1/3.png"></p><p>It is also worth noting that PyPDFForm does integrate the Codecov GitHub app to better work with this action.</p><p><img src="/2024/01/15/pypdfform-blog-1/4.png"></p><p>In the future if the rate limiting issue does get resolved, this action will likely be re-merged back to the test automation action.</p><h3 id="Code-Formatting"><a href="#Code-Formatting" class="headerlink" title="Code Formatting"></a>Code Formatting</h3><p>PyPDFForm uses two methods to ensure code quality. The first one is the automated code formatting action. Python has two great code formatting tools: <a href="https://pypi.org/project/black/">black</a> which formats all Python files in your source code, and <a href="https://pycqa.github.io/isort/">isort</a> which fixes all your import orders.</p><p>PyPDFForm makes use of <code>black</code> and <code>isort</code> with the following <a href="https://github.com/chinapandaman/PyPDFForm/blob/master/.github/workflows/python-black-isort.yml">action</a>:</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">name:</span> <span class="string">Code</span> <span class="string">Formatting</span></span><br><span class="line"></span><br><span class="line"><span class="attr">on:</span></span><br><span class="line"> <span class="attr">push:</span></span><br><span class="line"> <span class="attr">branches:</span> [ <span class="string">master</span> ]</span><br><span class="line"></span><br><span class="line"><span class="attr">jobs:</span></span><br><span class="line"> <span class="attr">build:</span></span><br><span class="line"> <span class="attr">runs-on:</span> <span class="string">ubuntu-latest</span></span><br><span class="line"></span><br><span class="line"> <span class="attr">steps:</span></span><br><span class="line"> <span class="bullet">-</span> <span class="attr">uses:</span> <span class="string">actions/checkout@v3</span></span><br><span class="line"> <span class="attr">with:</span></span><br><span class="line"> <span class="attr">token:</span> <span class="string">${{</span> <span class="string">secrets.BLACK_ISORT_TOKEN</span> <span class="string">}}</span></span><br><span class="line"> <span class="bullet">-</span> <span class="attr">name:</span> <span class="string">Set</span> <span class="string">up</span> <span class="string">Python</span></span><br><span class="line"> <span class="attr">uses:</span> <span class="string">actions/setup-python@v4</span></span><br><span class="line"> <span class="attr">with:</span></span><br><span class="line"> <span class="attr">python-version:</span> <span class="string">'3.x'</span></span><br><span class="line"> <span class="bullet">-</span> <span class="attr">name:</span> <span class="string">Install</span> <span class="string">black/isort</span></span><br><span class="line"> <span class="attr">run:</span> <span class="string">pip</span> <span class="string">install</span> <span class="string">black</span> <span class="string">isort</span></span><br><span class="line"> <span class="bullet">-</span> <span class="attr">name:</span> <span class="string">If</span> <span class="string">needed,</span> <span class="string">commit</span> <span class="string">black/isort</span> <span class="string">changes</span> <span class="string">to</span> <span class="string">the</span> <span class="string">pull</span> <span class="string">request</span></span><br><span class="line"> <span class="attr">run:</span> <span class="string">|</span></span><br><span class="line"><span class="string"> black .</span></span><br><span class="line"><span class="string"> isort .</span></span><br><span class="line"><span class="string"> git config --global user.name 'black-isort-bot'</span></span><br><span class="line"><span class="string"> git config --global user.email '[email protected]'</span></span><br><span class="line"><span class="string"> git diff --quiet && git diff --staged --quiet || git commit -am '[skip ci]: black/isort'</span></span><br><span class="line"><span class="string"> git push</span></span><br></pre></td></tr></table></figure><p>Unlike the test automation action, this action only runs when a commit is made to the <code>master</code> branch:</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">on:</span></span><br><span class="line"> <span class="attr">push:</span></span><br><span class="line"> <span class="attr">branches:</span> [ <span class="string">master</span> ]</span><br></pre></td></tr></table></figure><p>This is to ensure that none of the automated formatting will mess up diffs between branches and <code>master</code> in PRs.</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="bullet">-</span> <span class="attr">uses:</span> <span class="string">actions/checkout@v3</span></span><br><span class="line"> <span class="attr">with:</span></span><br><span class="line"> <span class="attr">token:</span> <span class="string">${{</span> <span class="string">secrets.BLACK_ISORT_TOKEN</span> <span class="string">}}</span></span><br></pre></td></tr></table></figure><p>The checkout source code step of the pipeline now makes use of a secret token called <code>BLACK_ISORT_TOKEN</code>. This is actually a GitHub developer token of mine stored under the repository’s secrets. We will talk about why it’s needed later on.</p><p>To actually use <code>black</code> and <code>isort</code>, obviously they will need to be installed:</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="bullet">-</span> <span class="attr">name:</span> <span class="string">Install</span> <span class="string">black/isort</span></span><br><span class="line"> <span class="attr">run:</span> <span class="string">pip</span> <span class="string">install</span> <span class="string">black</span> <span class="string">isort</span></span><br></pre></td></tr></table></figure><p>And finally, the following chunk gets executed:</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="bullet">-</span> <span class="attr">name:</span> <span class="string">If</span> <span class="string">needed,</span> <span class="string">commit</span> <span class="string">black/isort</span> <span class="string">changes</span> <span class="string">to</span> <span class="string">the</span> <span class="string">pull</span> <span class="string">request</span></span><br><span class="line"> <span class="attr">run:</span> <span class="string">|</span></span><br><span class="line"><span class="string"> black .</span></span><br><span class="line"><span class="string"> isort .</span></span><br><span class="line"><span class="string"> git config --global user.name 'black-isort-bot'</span></span><br><span class="line"><span class="string"> git config --global user.email '[email protected]'</span></span><br><span class="line"><span class="string"> git diff --quiet && git diff --staged --quiet || git commit -am '[skip ci]: black/isort'</span></span><br><span class="line"><span class="string"> git push</span></span><br></pre></td></tr></table></figure><p>In this step, the following happens in order:</p><ol><li><code>black</code> is run on all Python files.</li><li><code>isort</code> is run on all Python files.</li><li>Some <code>git</code> configurations are set so that a commit can be made.</li><li>Two <code>git diff</code> commands are run to determine if any file is modified as a result of <code>black</code> or <code>isort</code>. If so a new commit is made with modified files.</li><li>The commit is pushed to <code>master</code>.</li></ol><p>Note the commit message starts with <code>[skip ci]</code>. This way it will not trigger any new action after being pushed to <code>master</code>. This is intended as this commit only has code formatting changes.</p><p>Now you can see why the GitHub developer token needs to be setup when we initially checkout the source code. The credential is needed when we push the code formatting changes back to <code>master</code>.</p><p><img src="/2024/01/15/pypdfform-blog-1/5.png"><br><img src="/2024/01/15/pypdfform-blog-1/6.png"></p><p>Above you can see an example that shows the outcome of this action.</p><h3 id="Static-Analysis"><a href="#Static-Analysis" class="headerlink" title="Static Analysis"></a>Static Analysis</h3><p>In conjunction with the code formatting action, the following <a href="https://github.com/chinapandaman/PyPDFForm/blob/master/.github/workflows/python-linting.yml">action</a> utilizes <a href="https://pypi.org/project/pylint/#:~:text=Pylint%20is%20a%20static%20code,code%20without%20actually%20running%20it.">pylint</a> to statically analyze the project to cover some loopholes <code>black</code> might miss:</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">name:</span> <span class="string">Linting</span></span><br><span class="line"></span><br><span class="line"><span class="attr">on:</span></span><br><span class="line"> <span class="attr">push:</span></span><br><span class="line"> <span class="attr">branches:</span> [ <span class="string">master</span> ]</span><br><span class="line"> <span class="attr">pull_request:</span></span><br><span class="line"> <span class="attr">branches:</span> [ <span class="string">master</span> ]</span><br><span class="line"></span><br><span class="line"><span class="attr">jobs:</span></span><br><span class="line"> <span class="attr">build:</span></span><br><span class="line"></span><br><span class="line"> <span class="attr">runs-on:</span> <span class="string">ubuntu-latest</span></span><br><span class="line"></span><br><span class="line"> <span class="attr">steps:</span></span><br><span class="line"> <span class="bullet">-</span> <span class="attr">uses:</span> <span class="string">actions/checkout@v3</span></span><br><span class="line"> <span class="bullet">-</span> <span class="attr">name:</span> <span class="string">Set</span> <span class="string">up</span> <span class="string">Python</span></span><br><span class="line"> <span class="attr">uses:</span> <span class="string">actions/setup-python@v4</span></span><br><span class="line"> <span class="attr">with:</span></span><br><span class="line"> <span class="attr">python-version:</span> <span class="string">'3.x'</span></span><br><span class="line"> <span class="bullet">-</span> <span class="attr">name:</span> <span class="string">Install</span> <span class="string">dependencies</span></span><br><span class="line"> <span class="attr">run:</span> <span class="string">|</span></span><br><span class="line"><span class="string"> python -m pip install --upgrade pip</span></span><br><span class="line"><span class="string"> pip install -r requirements.txt</span></span><br><span class="line"><span class="string"></span> <span class="bullet">-</span> <span class="attr">name:</span> <span class="string">Analyze</span> <span class="string">code</span> <span class="string">with</span> <span class="string">pylint</span></span><br><span class="line"> <span class="attr">run:</span> <span class="string">|</span></span><br><span class="line"> <span class="string">pylint</span> <span class="string">PyPDFForm</span></span><br></pre></td></tr></table></figure><p>This action is 90% the same as the test automation action except the following:</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="bullet">-</span> <span class="attr">name:</span> <span class="string">Analyze</span> <span class="string">code</span> <span class="string">with</span> <span class="string">pylint</span></span><br><span class="line"> <span class="attr">run:</span> <span class="string">|</span></span><br><span class="line"> <span class="string">pylint</span> <span class="string">PyPDFForm</span></span><br></pre></td></tr></table></figure><p>Instead of running tests with <code>pytest</code>, it runs <code>pylint</code> on the source code for a static analysis.</p><p><img src="/2024/01/15/pypdfform-blog-1/7.png"></p><p><code>pylint</code>, paired up with <code>black</code> and <code>isort</code>, will ensure that PyPDFForm always has a clean, but more importantly a unified code style, regardless how each developer’s own coding style varies.</p><h3 id="Deploy"><a href="#Deploy" class="headerlink" title="Deploy"></a>Deploy</h3><p>So far we have only covered continuous integrations of PyPDFForm. It’s time to talk about the <a href="https://github.com/chinapandaman/PyPDFForm/blob/master/.github/workflows/python-publish.yml">action</a> that allows it to be continuously deployed:</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">name:</span> <span class="string">Deploy</span></span><br><span class="line"></span><br><span class="line"><span class="attr">on:</span></span><br><span class="line"> <span class="attr">release:</span></span><br><span class="line"> <span class="attr">types:</span> [<span class="string">created</span>]</span><br><span class="line"></span><br><span class="line"><span class="attr">jobs:</span></span><br><span class="line"> <span class="attr">deploy:</span></span><br><span class="line"></span><br><span class="line"> <span class="attr">runs-on:</span> <span class="string">ubuntu-latest</span></span><br><span class="line"></span><br><span class="line"> <span class="attr">steps:</span></span><br><span class="line"> <span class="bullet">-</span> <span class="attr">uses:</span> <span class="string">actions/checkout@v3</span></span><br><span class="line"> <span class="bullet">-</span> <span class="attr">name:</span> <span class="string">Set</span> <span class="string">up</span> <span class="string">Python</span></span><br><span class="line"> <span class="attr">uses:</span> <span class="string">actions/setup-python@v4</span></span><br><span class="line"> <span class="attr">with:</span></span><br><span class="line"> <span class="attr">python-version:</span> <span class="string">'3.x'</span></span><br><span class="line"> <span class="bullet">-</span> <span class="attr">name:</span> <span class="string">Install</span> <span class="string">dependencies</span></span><br><span class="line"> <span class="attr">run:</span> <span class="string">|</span></span><br><span class="line"><span class="string"> python -m pip install --upgrade pip</span></span><br><span class="line"><span class="string"> pip install setuptools wheel twine</span></span><br><span class="line"><span class="string"></span> <span class="bullet">-</span> <span class="attr">name:</span> <span class="string">Build</span> <span class="string">and</span> <span class="string">deploy</span> <span class="string">to</span> <span class="string">PyPI</span></span><br><span class="line"> <span class="attr">env:</span></span><br><span class="line"> <span class="attr">TWINE_USERNAME:</span> <span class="string">__token__</span></span><br><span class="line"> <span class="attr">TWINE_PASSWORD:</span> <span class="string">${{</span> <span class="string">secrets.PYPI_PASSWORD</span> <span class="string">}}</span></span><br><span class="line"> <span class="attr">run:</span> <span class="string">|</span></span><br><span class="line"><span class="string"> python setup.py sdist bdist_wheel</span></span><br><span class="line"><span class="string"> twine upload dist/*</span></span><br></pre></td></tr></table></figure><p>This action, unlike any other action we have discussed so far, is triggered when a <strong>release</strong> is made:</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">on:</span></span><br><span class="line"> <span class="attr">release:</span></span><br><span class="line"> <span class="attr">types:</span> [<span class="string">created</span>]</span><br></pre></td></tr></table></figure><p>The first few steps of the pipeline is quite similar to the other ones, checkout the source, setup Python, etc. The part we care about starts here:</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="bullet">-</span> <span class="attr">name:</span> <span class="string">Install</span> <span class="string">dependencies</span></span><br><span class="line"> <span class="attr">run:</span> <span class="string">|</span></span><br><span class="line"><span class="string"> python -m pip install --upgrade pip</span></span><br><span class="line"><span class="string"> pip install setuptools wheel twine</span></span><br></pre></td></tr></table></figure><p>The above will install <code>setuptools</code>, <code>wheel</code>, and <code>twine</code>, all of which are dependencies we need to package and deploy our projects.</p><p>Finally, this is where the actual deployment happens:</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="bullet">-</span> <span class="attr">name:</span> <span class="string">Build</span> <span class="string">and</span> <span class="string">deploy</span> <span class="string">to</span> <span class="string">PyPI</span></span><br><span class="line"> <span class="attr">env:</span></span><br><span class="line"> <span class="attr">TWINE_USERNAME:</span> <span class="string">__token__</span></span><br><span class="line"> <span class="attr">TWINE_PASSWORD:</span> <span class="string">${{</span> <span class="string">secrets.PYPI_PASSWORD</span> <span class="string">}}</span></span><br><span class="line"> <span class="attr">run:</span> <span class="string">|</span></span><br><span class="line"><span class="string"> python setup.py sdist bdist_wheel</span></span><br><span class="line"><span class="string"> twine upload dist/*</span></span><br></pre></td></tr></table></figure><p>It first packages the project based on the <a href="https://github.com/chinapandaman/PyPDFForm/blob/master/setup.py">setup.py</a> following <a href="https://packaging.python.org/en/latest/guides/distributing-packages-using-setuptools/#packaging-your-project">this guide</a>, and then uploads the packaged files to PyPI.</p><p>Note that this step uses two environment variables: <code>TWINE_USERNAME</code> which is set to <code>__token__</code>, and <code>TWINE_PASSWORD</code> which references a secret called <code>PYPI_PASSWORD</code>. These two variables configure the credential needed for PyPI and are what allow the final upload to happen.</p><p><img src="/2024/01/15/pypdfform-blog-1/8.png"></p><p>It is also worth noting that this step will need to be changed soon, as packaging by calling <code>setup.py</code> directly is <a href="https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html">deprecated</a>.</p><h3 id="Conclusion"><a href="#Conclusion" class="headerlink" title="Conclusion"></a>Conclusion</h3><p>Some of these actions were created from a while ago and it is quite possible that GitHub actions and Python suggest new best practices. But I think this article highlights a skeleton of how some CI/CDs can be done for Python projects and I hope anyone who is reading find this helpful.</p>]]></content>
<summary type="html"><h3 id="Preface"><a href="#Preface" class="headerlink" title="Preface"></a>Preface</h3><p>Happy new year fellow developers! The <a href="https://youtu.be/8t1RdAKwr9w?si=AcsOtfFHjuuoVOXC">speak</a> I did at the Chicago Python User Group was a huge success and I hope everyone enjoyed it. I’d like to start a series of blogs which will cover a number of subjects related to <a href="https://github.com/chinapandaman/PyPDFForm">PyPDFForm</a>. These subjects will hopefully cover some of the more in depth technical details that I didn’t get to cover during my speak.</p>
<p>In this first blog, I want to talk about all the CI&#x2F;CD pipelines I have set up for the project that ensured its stability and automation. These pipelines are also in my opinion quite general purpose and can be used by any Python project in the future.</p></summary>
<category term="Coding" scheme="https://chinapandaman.github.io/tags/Coding/"/>
<category term="PyPDFForm" scheme="https://chinapandaman.github.io/tags/PyPDFForm/"/>
<category term="CI/CD" scheme="https://chinapandaman.github.io/tags/CI-CD/"/>
</entry>
<entry>
<title>ChiPy Speak Script</title>
<link href="https://chinapandaman.github.io/2023/11/06/chipy-blog-1/"/>
<id>https://chinapandaman.github.io/2023/11/06/chipy-blog-1/</id>
<published>2023-11-07T04:16:41.000Z</published>
<updated>2023-11-21T18:16:46.469Z</updated>
<content type="html"><![CDATA[<h3 id="Preface"><a href="#Preface" class="headerlink" title="Preface"></a>Preface</h3><p>It’s been a while since the last time I wrote a blog. I have been doing a lot of travels in the last couple months.</p><p>On December 14th I will have the pleasure of presenting my first public tech speak at the Chicago Python User Group. <a href="https://www.chipy.org/meetings/238/">Here</a> is a link to the event with some description of my speak. So I’d like to use this blog as a place to write down the script of my speak so that I could keep practicing.</p><span id="more"></span><h3 id="Intro"><a href="#Intro" class="headerlink" title="Intro"></a>Intro</h3><p>Good evening. First of all I’d like to thank the Chicago Python User Group for obviously giving me this great opportunity to present in front of you all. My name is Jinge Li and today I’d like to talk about an open source project that I have been working on for the last three years. It is a Python library named PyPDFForm.</p><p>Here is a quick agenda for this speak. I’d like to spend the first 20 to 25 minutes talking about some backgrounds regarding this project. What are some of my personal tech journeys? What sparked this idea? And just general motivations and stuff. We will spend the next 10-15 minutes going through a small coding session where I will showcase some simple but essential functionalities of the library. And finally we will wrap up with some restrictions and the future of the library and Q and A. OK?</p><h3 id="Background"><a href="#Background" class="headerlink" title="Background"></a>Background</h3><p>So obviously you all are here to attend the Chicago <strong>Python</strong> User Group. And we can’t talk about background without talking about some background of Python. And I’m sure we all agree Python is in a great state right now.</p><p>It has stayed at the top of the TIOBE Index for many years. As a matter of fact I don’t recall when was the last time it dropped out of the top five.</p><p>On top of that it was three-time language of the year in the last five years. In the year of 2018, 2020, and 2021 Python is the language with the most popularity.</p><p>And as a general purpose language it has a large variety of usages across different areas. Here on the right we have two most outstanding ones, web development and data science. Devops also uses it to quickly script stuff. It’s also used a lot for serverless stuff such as AWS lambda.</p><p>It is loved by startups. Many startups need to spin up their service prototypes rapidly. This is where Python shines, it can develop things very quickly.</p><p>And finally it’s even used by non-developers to automate daily tasks. My girlfriend, for example, is an accountant. And the other day she came to me and was like could you teach me a bit of Python. Because she needed to bulk process some Excel sheets. So overall I’d say we Python developers are currently living in a good world.</p><p>Now that we know what a great tool Python is, it’s meant to be used to solve problems. And in the world of software engineering, it doesn’t matter what languages or tech stacks, one of the outstanding problems is PDF generation. And here on the right you can see why.</p><p>But jokes aside, PDF is something that suits almost any business model’s needs. Ecommerce business generates invoices or receipts. Law firm generates legal documents. And there are many more examples you can think about in other industries.</p><p>It is compressed, consistent, cross platform, and user friendly. A PDF file can be viewed on pretty much any machine nowadays. From your browser chrome to something more professional like Adobe Acrobat they all support PDFs. It has become a standard.</p><p>So now we know that PDF generation is a problem, we need a solution to it. And in the world of Python there is a great solution.</p><p>It is a Python library called reportlab. I’m not sure how many of you have heard of it. In essence reportlab describes PDF layouts using a markup styled syntax named RML. Now obviously the Python library abstracts these markup tags into different Python objects.</p><p>But with this library you can create complex layouts like grids or tables.</p><p>And it ultimately achieves PDF generation in a programmatic way using Python code.</p><p>Now however, and yes there is always the however.</p><p>It is one thing if you are trying to generate something like this. You know. You have a title at the top. The main section is trisected vertically. Each section has some simple texts. Nice and easy.</p><p>But it’s another thing if you are to, say, generate something that looks like this. Here you have tens of rows where each row’s cells don’t align with one another. Images that are squeezed into some cells. I don’t even want to talk about texts that are vertically aligned. Yuck.</p><p>So what does this tell us? Well, PDFs can be complex. I think we all agree this one on the right is complex and trust me this is not uncommon.</p><p>And for a PDF as complex as this, the reportlab code that’s used to generate this is messy. It is lengthy and can very quickly become unmaintainable.</p><p>And it’s just not very modern to implement such complex layouts using markup syntax. This is why in the world of frontend we invented React and Vue. Ask yourself when was the last time you wrote vanilla HTML when building web apps.</p><p>So now that we have learned some challenges when it comes to generating PDFs using reportlab, let’s shift the gear a little bit and I’d like to spend some time talking about myself for a bit.</p><p>When I first graduated from college, my first job was at this local startup here in Chicago. What they are is they are a third party software solution provider to government’s day to day tasks, more specifically the law enforcement department. I will give you an example of what they do. If let’s say you are driving in the northern side of Illinois, like in Lake County, and you are a little too heavy on your gas pedal and got pulled over by an officer and received a speeding ticket, there’s a chance that the speeding ticket is from where I worked.</p><p>So now with that being said, any of you who have ever worked with the government will know one thing that’s true. They have a lot of paperwork. And that complex PDF I just showed you earlier is actually very close to some of the documents we have to generate for our clients. And the struggles I was talking about when using reportlab were the exact same struggles I had at work.</p><p>So I started thinking about how I can improve this process, and this is when I started doing my research. And the thing that came to my mind is PDF forms. I’m sure many of you here have dealt with it at some point of your life. Like it used to be that when you go file your taxes those forms you downloaded and filled were PDF forms. When you apply for your U.S. passport the application you filled in was also a PDF form. So the moment I thought about that I was like, great I just need to prepare the PDF I want to generate into a form and fill it programmatically with the data that needs to be plugged in.</p><p>And in the world of software development, one thing that’s almost always true is that we shouldn’t be reinventing wheels. And that’s why I ask myself, is there an existing solution already? Right? There’s no way I’m the only person who ran into this problem and there’s gotta be something implemented already.</p><p>But it turned out the answer was no, and I was shocked because that’s a very rare answer to that question in our world. Ok it’s not that there’s no existing solutions. It turned out that existing Python libraries that do similar things require some non-Python dependencies. Some of them are at the system level. Other ones may require some other binaries in another language bundled together. And that’s not what I wanted.</p><p>And just like how many other open source projects were started, I didn’t like any of the existing solutions and that’s when I decided to write my own. Now the first question I ask myself is what data structure should I use to represent a PDF form. So with PDF forms the widgets on there can be tagged with a name right? Now I just need to pair the name of the widgets with their values. It is better to be something built in. This is quite obvious right, anyone?</p><p>A dictionary that’s right. And this is the start of PyPDFForm, or I should say at the time it was only a script I wrote. What it does in its essence is exactly what I just said, it fills a PDF form from a Python dictionary.</p><p>Now I did not implement it from scratch. It does have some dependencies but PyPDFForm and all its dependencies are pure Python.</p><p>It also has some other miscellaneous functionalities such as drawing images and merging multiple PDFs together, you know, just some nice to have stuff.</p><p>Now let me spend a bit time talking about this last bullet point, what do I mean by that? Well the previous process of generating PDF forms would be that your clients give you a PDF that they want you to generate, and it is on you, the developer, to write the script that would precisely describe the layout of it. This process is just very tedious and time consuming. Well now instead of you scripting out the layout, the layout is already there because when your clients hand you the PDF, your project manager can just go ahead and turn it into a PDF form using any main steam form preparation tool like Adobe Acrobat. All you have to do is figuring out what data needs to go on there, preparing the dictionary and plugging it in. This is a much less painful process for developers and draws clear scope for the process of PDF generation.</p><p>Before we move on, any question?</p><h3 id="Coding-Session"><a href="#Coding-Session" class="headerlink" title="Coding Session"></a>Coding Session</h3><p>So obviously talk is cheap. Let’s spend the next ten fifteen minutes on a coding session to show you guys some of the stuffs the library can do.</p><p>So right here I have an empty directory on my computer. Let’s start by first creating a virtual environment.</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">python3 -m venv venv</span><br><span class="line">source venv/bin/activate</span><br></pre></td></tr></table></figure><p>The installation of PyPDFForm is similar to any other libraries. You just do the normal pip install.</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">pip install PyPDFForm</span><br></pre></td></tr></table></figure><p>So I could show you how to prepare a PDF form from scratch but we only have that much time and there are many other tutorials online for many tools. So I will skip it and <a href="https://eforms.state.gov/Forms/ds11.pdf">here</a> I have a random PDF form I found online. It’s an application for the U.S. passport. Let’s use this for our demo. So let’s download it and put it in our directory.</p><p>And we can instantiate a PyPDFForm object using the form we just downloaded.</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">from</span> PyPDFForm <span class="keyword">import</span> PyPDFForm</span><br><span class="line"></span><br><span class="line">pdf = PyPDFForm(<span class="string">'ds11.pdf'</span>)</span><br></pre></td></tr></table></figure><p>Now the first thing you want to ask yourself when you get a PDF form is what are the names for all these widgets or elements, aka what are the keys of the dictionary you will fill it with. So here I wanna show you a somewhat new and experimental functionality I developed recently called preview.</p><p>So let’s create an output file and then write the preview of the form into it and see what happens.</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">with</span> <span class="built_in">open</span>(<span class="string">'output.pdf'</span>, <span class="string">'wb+'</span>) <span class="keyword">as</span> f:</span><br><span class="line"> f.write(pdf.preview)</span><br></pre></td></tr></table></figure><p>And if we go ahead and open up the output PDF, we will see that each element will have its name displayed on top of it in red.</p><p>So this does look better on a simpler PDF form. But as you can see with this one it gets quite messy when it’s complex, which is why it’s an experimental feature and I’m still thinking about how I can improve it.</p><p>A more mature method to inspect a form is the generate schema method. So you just have to do this:</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">from</span> pprint <span class="keyword">import</span> pprint</span><br><span class="line"></span><br><span class="line">pprint(pdf.generate_schema())</span><br></pre></td></tr></table></figure><p>And what you are seeing here is a JSON schema that describes the JSON object, or in the context of Python the dictionary that the object expects. So for example <code>Applicant First Name</code> takes a string and has a maximum length of 17. <code>Applicant Changing Gender</code> is a boolean value which is saying it is a checkbox. <code>Card Status</code> takes an integer type and has a maximum value of 3 and why is that? Because it’s a radio button and has 4 different choices.</p><p>Combining the schema and the preview will give you a pretty good idea on what data should go into the form. And another nice thing is when you do have your dictionary ready, you can use the schema to validate the dictionary just like how you would use your JSON schema to validate a JSON object.</p><p>So now that we know what data needs to go into the object, let’s fill it out. As you can see from the schema, none of these elements are required so let’s just fill the three elements we just discussed. And all you have to do is calling the fill method and pass the dictionary to it.</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">filled = pdf.fill(</span><br><span class="line"> {</span><br><span class="line"> <span class="string">"Applicant First Name"</span>: <span class="string">"Jinge"</span>,</span><br><span class="line"> <span class="string">"Applicant Changing Gender"</span>: <span class="literal">True</span>,</span><br><span class="line"> <span class="string">"Card Status"</span>: <span class="number">1</span>,</span><br><span class="line"> }</span><br><span class="line">)</span><br></pre></td></tr></table></figure><p>And now you have your filled form, the object implements itself similar to a file object. So you can simply write it out like this:</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">with</span> <span class="built_in">open</span>(<span class="string">'output.pdf'</span>, <span class="string">'wb+'</span>) <span class="keyword">as</span> f:</span><br><span class="line"> f.write(filled.read())</span><br></pre></td></tr></table></figure><p>And if we open up the output file, we will see our first filled PDF form.</p><p>So one thing you will notice is that the library does respect the styles of these elements. So if we look at this first name field you can see how each character is evenly spaced out. This is actually a property for these text fields called comb that you can set when you create the form and what’s nice about PyPDForm is that it’s smart enough to know the styling and adjust accordingly. And I will show you another example in just a minute.</p><p>So let’s move away from the code and try to tweak our PDF form for a bit. And there are many tools out there that you can use to prepare or modify PDF forms. The most professional one is probably Adobe Acrobat. The one I’m going to use here is a web based one called DocFly. So let’s upload our PDF form and make some changes to it.</p><p>As you can see here, this first name text field did have the combed property set and that’s why its characters are evenly spaced out. If I deselect the property you can see the text no longer do that.</p><p>Let’s look at another one. On this next page you have this other name field called <code>Name of Applicant 2</code>. What I’m going to do is to change the style of texts that are filled into it. So let’s first change the font from <code>Helvetica</code> to <code>Times New Roman</code>. And then I think the texts are just too big even though they are not so I’m changing the font size from 18 to 12. I’m also changing the font color to, let’s say magenta. I’m making it both bold and italic. And finally I’m making it align to the middle instead of to the left.</p><p>That seems to me enough changes. Let’s save the file and download it. And I’m making no other code changes besides changing which template to use and adding that new text field we just made a lot of changes to to the dictionary.</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">pdf = PyPDFForm(<span class="string">'ds11-copy.pdf'</span>)</span><br><span class="line"></span><br><span class="line">filled = pdf.fill(</span><br><span class="line"> {</span><br><span class="line"> <span class="string">"Applicant First Name"</span>: <span class="string">"Jinge"</span>,</span><br><span class="line"> <span class="string">"Applicant Changing Gender"</span>: <span class="literal">True</span>,</span><br><span class="line"> <span class="string">"Card Status"</span>: <span class="number">1</span>,</span><br><span class="line"> <span class="string">"Name of Applicant 2"</span>: <span class="string">"Jinge"</span>,</span><br><span class="line"> }</span><br><span class="line">)</span><br></pre></td></tr></table></figure><p>And if we run it, you will see now that the name field on the first page is no longer evenly spaced out, and the name field on the second page now has a different font with a smaller font size and color. It’s bold and italic and align to the middle. This just shows how smart the library is. If you use is and need to make styling changes, have your project manager do it because they should be experts at Adobe Acrobat and all you need to focus on is the data that needs to go in there.</p><p>Well then you may be wondering, sometimes the style of the template is not what it should be, but at the same time I don’t have one of these form preparation tools to change it. Well the PyPDFForm object does let you to tweak some of the styles during run time. Let’s look at how it works.</p><p>Let’s say now that we have made this first name field no longer spaced out, I feel like its font size should be larger. And the way you access all of these elements of your object is through the elements attribute. And just like how you would access a value of dictionary through its key, you do the same thing right here. And here you can set its font size by just doing this:</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">pdf.elements[<span class="string">"Applicant First Name"</span>].font_size = <span class="number">30</span></span><br></pre></td></tr></table></figure><p>And if we re-run our scripts, you will see that you can change the font size during run time if you don’t like it.</p><p>Same thing, so let’s say you regret making the second name field’s font so small and you no longer like magenta as its font color. You can make the same changes for it too.</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">pdf.elements[<span class="string">"Name of Applicant 2"</span>].font_size = <span class="number">18</span></span><br><span class="line">pdf.elements[<span class="string">"Name of Applicant 2"</span>].font_color = (<span class="number">1</span>, <span class="number">0</span>, <span class="number">0</span>)</span><br></pre></td></tr></table></figure><p>Here for the font color it takes an RGB tuple, let’s make it red. Re-run the script and let’s see the effect.</p><p>And finally you say forget everything I just want to make every single text I filled into the PDF form super large and blue. Well it turns out you can do that too when you instantiate the object.</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">pdf = PyPDFForm(<span class="string">'ds11-copy.pdf'</span>, global_font_size=<span class="number">50</span>, global_font_color=(<span class="number">0</span>, <span class="number">0</span>, <span class="number">1</span>))</span><br></pre></td></tr></table></figure><p>Obviously these are just the peak of iceberg. There are some other functionalities like draw text, draw images, and merge multiple PDFs together. But I really want to show the essential part of it, which is the actually filling piece.</p><p>Any question before we move on?</p><h3 id="Conclusion"><a href="#Conclusion" class="headerlink" title="Conclusion"></a>Conclusion</h3><p>I’d like to wrap up this speak with some discussions around restrictions and future improvements that are planned. Because many of you may be wondering, now that you have shown what the library can do, what can it not do?</p><p>Well the library requires you to have a static PDF form template as a baseline, so the largest restriction with it is that it cannot generate dynamic sections. So on the right you see a pretty common example of an invoice PDF. Notice that middle section where all the items are listed? That’s not something you can handle with the library. I mean sure maybe you can have like ten rows of text fields and it will probably work for the most part. But the point I’m trying to make is that it’s not flexible. So all and all for PDFs with dynamic sections you still need something like reportlab.</p><p>The next thing is that the library only supports some of the most common elements like you just saw in the demo. Now the elements that you guys saw probably cover 90% of the usage of PDF forms. But still there are more. I can give you two common ones. Image fields, you know, the one where you can click on it and attach an image on it. Another one is signature fields, kinda similar to image fields but instead where you sign.</p><p>And finally like what I said earlier in the demo, there are many PDF form preparation tools out there. Some of them generate the form the same way but some other ones are different under the hood. I believe I have made the library to support most of the mainstream ones out there but still there is no way I have explored every single one of them out there.</p><p>With that being said, let’s talk about some future improvements that are currently on my radar for the library.</p><p>First, obviously I’d love to support more elements. Using image fields for example, I tried to support it about two years ago but was not able to. But at the time I was also not as experienced in terms of developing this library as I am nowadays, so I have been having the thoughts of picking it back up again.</p><p>Next I would love to support more of those unexplored form preparation tools out there. This in fact is something I wrote on the README. If you ever run into a problem where whatever tools you are using to prepare the PDF form that doesn’t work with the library, open an issue and I will try to get it supported.</p><p>And you tell me! Ya that’s correct. And this brings me back to why I’m here to present in front of you all. Recently I have run into kinda a stale state when working on PyPDFForm where I’m starting to run out of ideas on what features to bring next. This is why I came here, to present in front of a group of public, and hopefully by speaking with you guys some new ideas will spark again. And this doesn’t have to happen immediately. If any of you find this library useful, or simply find it interesting and try play with it for a bit, I’m always a believer that users are the best source of ideas.</p><p>Any other question?</p><p>Well thank you very much. It is my honor to speak in front of you all and again thanks to Chicago Python User Group for having me.</p>]]></content>
<summary type="html"><h3 id="Preface"><a href="#Preface" class="headerlink" title="Preface"></a>Preface</h3><p>It’s been a while since the last time I wrote a blog. I have been doing a lot of travels in the last couple months.</p>
<p>On December 14th I will have the pleasure of presenting my first public tech speak at the Chicago Python User Group. <a href="https://www.chipy.org/meetings/238/">Here</a> is a link to the event with some description of my speak. So I’d like to use this blog as a place to write down the script of my speak so that I could keep practicing.</p></summary>
<category term="Speak" scheme="https://chinapandaman.github.io/tags/Speak/"/>
</entry>
<entry>
<title>Vim as Primary IDE - Buffer Related</title>
<link href="https://chinapandaman.github.io/2023/08/06/vim-blog-4/"/>
<id>https://chinapandaman.github.io/2023/08/06/vim-blog-4/</id>
<published>2023-08-06T18:40:47.000Z</published>
<updated>2023-08-06T18:40:14.467Z</updated>
<content type="html"><![CDATA[<h3 id="Thank-You-Bram"><a href="#Thank-You-Bram" class="headerlink" title="Thank You Bram"></a>Thank You Bram</h3><p>Before I start this blog today, please allow me to express my deepest grieving to Mr. Bram Moolenaar, the author of Vim who just left us forever. Without your work many of us developers will never have the opportunity of using such a great tool that boosts our efficiency. You, like many other contributors, are the true martyr to the spirit of open source.</p><h3 id="Buffer-Intro"><a href="#Buffer-Intro" class="headerlink" title="Buffer Intro"></a>Buffer Intro</h3><p>A buffer is an area of Vim’s memory used to hold text read from a file. Buffers are what allow Vim to edit multiple files simultaneously and switch between each of them. In this blog, I will be talking about some plugins and configs that make editing buffers of Vim an experience similar to a modern IDE.</p><span id="more"></span><h3 id="Tabs"><a href="#Tabs" class="headerlink" title="Tabs"></a>Tabs</h3><p>Pretty much all modern IDEs have a tabline at the top of their UI. Whenever we open a new file it adds a new tab to it. We can navigate through files we opened by navigating through different tabs. Vim’s buffer plays a similar role, except with its default behavior we will need to use commands like <code>:buffers</code> or <code>:ls</code> to know what each buffer holds and then navigate from there. Is there an easier way?</p><p>Luckily, one of Vim’s most popular plugins, <a href="https://github.com/vim-airline/vim-airline">vim-airline</a>, provides a very neat tabline. We can install the plugin like this:</p><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">call</span> plug#begin()</span><br><span class="line"> <span class="comment">" other plugins</span></span><br><span class="line"> Plug <span class="string">'vim-airline/vim-airline'</span></span><br><span class="line"><span class="keyword">call</span> plug#end()</span><br><span class="line"></span><br><span class="line"><span class="keyword">let</span> <span class="variable">g:airline_powerline_fonts</span> = <span class="number">1</span></span><br></pre></td></tr></table></figure><p>Note that the <code>airline_powerline_fonts</code> config requires <a href="https://github.com/powerline/fonts">Powerline Fonts</a>. After running <code>:PlugInstall</code> Vim will be geared with a lean and meaningful status bar at the bottom.</p><p><img src="/2023/08/06/vim-blog-4/1.png"></p><p>Notice how the status bar shares the same theme as my Vim without further configuring which is very nice.</p><p>Now we just need to add this config to enable the tabline:</p><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">let</span> <span class="variable">g:airline</span>#extensions#tabline#enabled = <span class="number">1</span></span><br></pre></td></tr></table></figure><p><img src="/2023/08/06/vim-blog-4/2.png"></p><p>So now we have a much cleaner visual view of buffers that are opened, but navigating between them still requires commands like <code>:b <buffer></code> or <code>:bnext</code>. So the next step I did was to map the Tab key for buffer navigation:</p><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">nnoremap</span> <span class="symbol"><Tab></span> :<span class="keyword">bn</span><span class="symbol"><CR></span></span><br><span class="line"><span class="keyword">nnoremap</span> <span class="symbol"><S-Tab></span> :<span class="keyword">bp</span><span class="symbol"><CR></span></span><br></pre></td></tr></table></figure><p>By adding the above configs, I’m able to go forwards with Tab and backwards with Shift + Tab keys through the tabline.</p><h3 id="File-Explorer"><a href="#File-Explorer" class="headerlink" title="File Explorer"></a>File Explorer</h3><p>A file tree explorer is also equipped for pretty much all modern IDEs. Nowadays as I’m more and more experienced with Vim and working with terminals I find myself using visual file explorer less and less. But I still think for some complex projects with many levels of nested directories a file explorer can come quite handy.</p><p>Vim’s best known file explorer is the good old <a href="https://github.com/preservim/nerdtree">NERDTREE</a>:</p><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">call</span> plug#begin()</span><br><span class="line"> <span class="comment">" other plugins</span></span><br><span class="line"> Plug <span class="string">'scrooloose/nerdtree'</span></span><br><span class="line"><span class="keyword">call</span> plug#end()</span><br></pre></td></tr></table></figure><p>Once installed, there are some small configs I added for it:</p><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">nnoremap</span> <span class="symbol"><Leader></span><span class="keyword">m</span> :NERDTreeToggle<span class="symbol"><CR></span></span><br><span class="line"><span class="keyword">let</span> NERDTreeShowHidden=<span class="number">1</span></span><br></pre></td></tr></table></figure><p>The first line is simply a keybinding I mapped to more easily open and close NERDTREE. The second line is to enable displaying of hidden files, as in most cases I will need to edit files like <code>.gitignore</code> or even <code>.vimrc</code>.</p><p><img src="/2023/08/06/vim-blog-4/3.png"></p><h3 id="Split-Navigations"><a href="#Split-Navigations" class="headerlink" title="Split Navigations"></a>Split Navigations</h3><p>Vim allows creating different splits in a view and has a big room for customization. This enables infinite possibilities as each split’s buffer can display more than just files opened. In fact many plugins, like NERDTREE discussed in the previous section, utilizes splits to display its UI.</p><p>In general though, developers will almost always use split’s basic functionalities like vertical splits which allows them to view different files side by side. Vim allows navigation of splits with <code>:wincmd <ijkl></code> which is in my opinion not the most convenient way. What I personally find out is that since Vim’s navigation within code blocks can already be done through the <code>ijkl</code> keys, it frees up the four arrow keys which I decided to map them to navigations between splits:</p><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">nnoremap</span> <span class="symbol"><Up></span> :<span class="keyword">wincmd</span> <span class="keyword">k</span><span class="symbol"><CR></span></span><br><span class="line"><span class="keyword">nnoremap</span> <span class="symbol"><Down></span> :<span class="keyword">wincmd</span> <span class="keyword">j</span><span class="symbol"><CR></span></span><br><span class="line"><span class="keyword">nnoremap</span> <span class="symbol"><Left></span> :<span class="keyword">wincmd</span> h<span class="symbol"><CR></span></span><br><span class="line"><span class="keyword">nnoremap</span> <span class="symbol"><Right></span> :<span class="keyword">wincmd</span> <span class="keyword">l</span><span class="symbol"><CR></span></span><br></pre></td></tr></table></figure><h3 id="Closing-Buffers"><a href="#Closing-Buffers" class="headerlink" title="Closing Buffers"></a>Closing Buffers</h3><p>One of the most annoying things when working with Vim is that you have all your splits set up and organized perfectly. Then you run <code>:bdelete</code> to delete a buffer and then Vim closes all windows that have the buffer opened and your layout is completely messed up. The good news is there is a small plugin that solves this exact problem named <a href="https://github.com/moll/vim-bbye">vim-bbye</a>:</p><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">call</span> plug#begin()</span><br><span class="line"> <span class="comment">" other plugins</span></span><br><span class="line"> Plug <span class="string">'moll/vim-bbye'</span></span><br><span class="line"><span class="keyword">call</span> plug#end()</span><br><span class="line"></span><br><span class="line"><span class="keyword">nnoremap</span> <span class="symbol"><Leader></span>q :Bdelete<span class="symbol"><CR></span></span><br></pre></td></tr></table></figure><p>By adding the above configs I’m able to close the current buffer with a \ + q keybind without messing up my current split layout.</p><h3 id="Conclusion"><a href="#Conclusion" class="headerlink" title="Conclusion"></a>Conclusion</h3><p>In this blog, I went over how I have been interacting with Vim’s buffers. Buffers are one of Vim’s most powerful core functionalities and what makes Vim so customizable UI wise. Even though this is my current most comfortable way of using it I have only scratched the surface of the iceberg.</p><p>In the next article, I will be going over how I configured my Vim to do searches and Git related workflows.</p>]]></content>
<summary type="html"><h3 id="Thank-You-Bram"><a href="#Thank-You-Bram" class="headerlink" title="Thank You Bram"></a>Thank You Bram</h3><p>Before I start this blog today, please allow me to express my deepest grieving to Mr. Bram Moolenaar, the author of Vim who just left us forever. Without your work many of us developers will never have the opportunity of using such a great tool that boosts our efficiency. You, like many other contributors, are the true martyr to the spirit of open source.</p>
<h3 id="Buffer-Intro"><a href="#Buffer-Intro" class="headerlink" title="Buffer Intro"></a>Buffer Intro</h3><p>A buffer is an area of Vim’s memory used to hold text read from a file. Buffers are what allow Vim to edit multiple files simultaneously and switch between each of them. In this blog, I will be talking about some plugins and configs that make editing buffers of Vim an experience similar to a modern IDE.</p></summary>
<category term="Coding" scheme="https://chinapandaman.github.io/tags/Coding/"/>
<category term="IDE" scheme="https://chinapandaman.github.io/tags/IDE/"/>
<category term="Vim" scheme="https://chinapandaman.github.io/tags/Vim/"/>
</entry>
<entry>
<title>Sonar Analysis Against PyPDFForm</title>
<link href="https://chinapandaman.github.io/2023/05/15/sonar-blog-1/"/>
<id>https://chinapandaman.github.io/2023/05/15/sonar-blog-1/</id>
<published>2023-05-16T01:33:03.000Z</published>
<updated>2023-05-16T01:39:37.395Z</updated>
<content type="html"><![CDATA[<h3 id="Motivation"><a href="#Motivation" class="headerlink" title="Motivation"></a>Motivation</h3><p>While working on <a href="https://github.com/chinapandaman/PyPDFForm">PyPDFForm</a>, I implemented many different mechanisms to ensure the code quality of the project. Some of these include:</p><ul><li>Testing using <a href="https://pytest.org/">pytest</a>.</li><li>Uploading coverage data to <a href="https://app.codecov.io/gh/chinapandaman/PyPDFForm">codecov</a>.</li><li>Formatting the code using <a href="https://github.com/chinapandaman/PyPDFForm/blob/master/.github/workflows/python-black-isort.yml">black/isort</a>.</li><li>Linting using <a href="https://pypi.org/project/pylint/">pylint</a>.</li></ul><p>Recently I have learned about the community loved static analysis tool <a href="https://www.sonarsource.com/products/sonarqube/">SonarQube</a>. So I ask myself what kind of static analysis result I would get from it, especially with all these mechanisms I already have to ensure code quality. In this blog, I will find out answers to this exact question.</p><span id="more"></span><h3 id="Setup-Sonar"><a href="#Setup-Sonar" class="headerlink" title="Setup Sonar"></a>Setup Sonar</h3><p>For now all I want is to see some analysis results from Sonar for PyPDFForm. There is no intention of making Sonar part of the CI pipeline yet. So there is no need of hosting Sonar on a remote server, instead all I want is to simply host Sonar locally and run a couple scans. Luckily Sonar does provide a way of doing that in <a href="https://docs.sonarqube.org/latest/try-out-sonarqube/">this</a> documentation.</p><p>The first step is to setup a local host for Sonar. As part of the prerequisites hosting Sonar requires Java 17 for version 10.0, which is something that’s already installed on my machine.</p><p><img src="/2023/05/15/sonar-blog-1/1.png"></p><p>The next thing is to download the zip file for Sonar host from <a href="https://www.sonarsource.com/products/sonarqube/downloads/">here</a>. Here I downloaded the community edition of it.</p><p>After unzipping the zip file it is pretty simple to start the server by running the following:</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><path_to_unzipped_folder>/bin/<OS>/sonar.sh console</span><br></pre></td></tr></table></figure><p>In my case I’m on a Surface Pro running Ubuntu. So the command would look like:</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><path_to_unzipped_folder>/bin/linux-x86-64/sonar.sh console</span><br></pre></td></tr></table></figure><p><img src="/2023/05/15/sonar-blog-1/2.png"></p><p>Execute the command and wait for it to finish spinning up the server. It is done when these prompts show up:</p><p><img src="/2023/05/15/sonar-blog-1/3.png"></p><p>Now if I go to <a href="http://localhost:9000/">http://localhost:9000/</a>, I will see this page which indicates Sonar is up and running locally:</p><p><img src="/2023/05/15/sonar-blog-1/4.png"></p><p>Enter admin/admin for login/password, it will then redirect me to a page where I need to change the admin password. Once done Sonar host will be ready to use.</p><p><img src="/2023/05/15/sonar-blog-1/5.png"></p><p>Now that I have the host setup, the next thing I need is the scanner that actually scans the code. For my purpose I’m downloading the generic Sonar scanner from <a href="https://docs.sonarqube.org/latest/analyzing-source-code/scanners/sonarscanner/">here</a>. Unzip the zip file and it will be ready to use.</p><p><img src="/2023/05/15/sonar-blog-1/6.png"></p><h3 id="Analyze-PyPDFForm"><a href="#Analyze-PyPDFForm" class="headerlink" title="Analyze PyPDFForm"></a>Analyze PyPDFForm</h3><p>I have both the Sonar host and Sonar scanner setup. It’s time to run my first scan against PyPDFForm.</p><p>Before I start scanning, I need to first generate a token from the Sonar host given that’s how Sonar authenticates. The token can be generated from <a href="http://localhost:9000/account/security">this page</a>. Here I named my token <code>local</code> and selected the type <code>User Token</code>:</p><p><img src="/2023/05/15/sonar-blog-1/7.png"></p><p>Once I have my token it can be used as part of the Sonar scanner command. Here is what it typically looks like:</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><path_to_unzipped_scanner_folder>/bin/sonar-scanner -Dsonar.host.url=http://localhost:9000/ -Dsonar.token=<generated_token> -Dsonar.projectKey=<project_key> -Dsonar.projectName=<project_display_name></span><br></pre></td></tr></table></figure><p>In my case, the command looks like this:</p><p><img src="/2023/05/15/sonar-blog-1/8.png"></p><p>Let’s look at this command piece by piece:</p><ul><li><code>-Dsonar.host.url</code> specifies the host URL of the Sonar server, which in my case is the local host.</li><li><code>-Dsonar.token</code> is the token I just generated.</li><li><code>-Dsonar.projectKey</code> is the unique identifier of the project.</li><li><code>-Dsonar.projectName</code> is the display name of the project.</li></ul><p>Execute the command and the scanner will kick off the scan. Wait until it finishes:</p><p><img src="/2023/05/15/sonar-blog-1/9.png"></p><p>Now if I refresh the Sonar host’s home page, I can see my new project show up there:</p><p><img src="/2023/05/15/sonar-blog-1/10.png"></p><p>If I click on the project and open up its dashboard I can see the analysis result of my first scan:</p><p><img src="/2023/05/15/sonar-blog-1/11.png"></p><p>So the good news is that my project seems to have no bugs, vulnerabilities, security hotspots, and duplications. All good stuffs and I can see how all those code quality mechanisms have helped me to achieve these.</p><p>The not so good news are:</p><ul><li>The coverage shows as 0%.</li><li>There are some code smells.</li></ul><p>For coverage I know PyPDFForm as a project has 100% coverage since I have introduced Codecov as part of my CI pipeline. For Sonar all I need to do is to upload my coverage data. This means I need to run another scan while specifying where Sonar should upload my coverage file.</p><p>So first of all let me re-run tests and generate a new coverage file:</p><p><img src="/2023/05/15/sonar-blog-1/12.png"></p><p>After that my new Sonar scanner command looks like this:</p><p><img src="/2023/05/15/sonar-blog-1/13.png"></p><p>Here I introduced another option <code>-Dsonar.python.coverage.reportPaths</code> which tells Sonar the location of my coverage file. For more information about test coverage on Sonar please read <a href="https://docs.sonarqube.org/latest/analyzing-source-code/test-coverage/overview/">here</a>.</p><p>Execute the command, wait until the scan finishes, and now let’s look at the dashboard:</p><p><img src="/2023/05/15/sonar-blog-1/14.png"></p><p>Now the coverage is no longer 0%, but instead of the expected 100% it shows 57%.</p><p>Let’s look into why this is the case by clicking on the <code>Code</code> tab of the dashboard.</p><p><img src="/2023/05/15/sonar-blog-1/15.png"></p><p>And here I found out why. It looks like the main package of <code>PyPDFForm</code> does have 100% coverage. It’s just that Sonar scanned some other files which are neither covered nor intended to be covered.</p><p>This means I will have to exclude certain files from my Sonar scans. Let’s modify my Sonar scanner command one more time:</p><p><img src="/2023/05/15/sonar-blog-1/16.png"></p><p>Here I added the option <code>-Dsonar.exclusions</code> which tells Sonar some files or directories I would like to exclude from the scan.</p><p>Execute the command and let’s look at the dashboard again:</p><p><img src="/2023/05/15/sonar-blog-1/17.png"></p><p>Finally the coverage is 100%. But there are still 4 code smells shown on the dashboard. Let’s take a look at them by clicking on the <code>Issues</code> tab.</p><p><img src="/2023/05/15/sonar-blog-1/18.png"></p><p>So all 4 code smells seem to point to the same thing, which is some of my functions could use some refactoring to reduce cognitive complexity. This is mostly due to complex logic such as too many if else blocks within a single function. For example:</p><p><img src="/2023/05/15/sonar-blog-1/19.png"></p><p>However, Sonar gave it a maintainability score of <code>A</code>, which seems to indicate it’s nothing major. On top of that reducing complexity of functions is something I have been continuously doing anyway while working on PyPDFForm.</p><p><img src="/2023/05/15/sonar-blog-1/20.png"></p><h3 id="Conclusion"><a href="#Conclusion" class="headerlink" title="Conclusion"></a>Conclusion</h3><p>Overall I’d say the outcome of Sonar analysis against PyPDFForm is excellent. It is a pleasure to know that all the previous effort I put into code quality assurance is recognized by the community loved static analysis tool. I would also recommend anyone who reads this blog give SonarQube a try as it is a great tool and will make your code much cleaner.</p>]]></content>
<summary type="html"><h3 id="Motivation"><a href="#Motivation" class="headerlink" title="Motivation"></a>Motivation</h3><p>While working on <a href="https://github.com/chinapandaman/PyPDFForm">PyPDFForm</a>, I implemented many different mechanisms to ensure the code quality of the project. Some of these include:</p>
<ul>
<li>Testing using <a href="https://pytest.org/">pytest</a>.</li>
<li>Uploading coverage data to <a href="https://app.codecov.io/gh/chinapandaman/PyPDFForm">codecov</a>.</li>
<li>Formatting the code using <a href="https://github.com/chinapandaman/PyPDFForm/blob/master/.github/workflows/python-black-isort.yml">black&#x2F;isort</a>.</li>
<li>Linting using <a href="https://pypi.org/project/pylint/">pylint</a>.</li>
</ul>
<p>Recently I have learned about the community loved static analysis tool <a href="https://www.sonarsource.com/products/sonarqube/">SonarQube</a>. So I ask myself what kind of static analysis result I would get from it, especially with all these mechanisms I already have to ensure code quality. In this blog, I will find out answers to this exact question.</p></summary>
<category term="Coding" scheme="https://chinapandaman.github.io/tags/Coding/"/>
<category term="Static Analysis" scheme="https://chinapandaman.github.io/tags/Static-Analysis/"/>
<category term="Sonar" scheme="https://chinapandaman.github.io/tags/Sonar/"/>
</entry>
<entry>
<title>Vim as Primary IDE - Basic Editing</title>
<link href="https://chinapandaman.github.io/2023/05/04/vim-blog-3/"/>
<id>https://chinapandaman.github.io/2023/05/04/vim-blog-3/</id>
<published>2023-05-04T13:14:29.000Z</published>
<updated>2023-05-04T15:18:45.278Z</updated>
<content type="html"><![CDATA[<h3 id="Finding-Plugins"><a href="#Finding-Plugins" class="headerlink" title="Finding Plugins"></a>Finding Plugins</h3><p>In the previous blog, I discussed how to use vim-plug to manage plugins. So the question now is how do I find all those fancy plugins out there?</p><p>On top of just simply google and browse through GitHub, there is a website called <a href="https://vimawesome.com/">VimAwesome</a> which allows me to browse and search different plugins by categories.</p><p>In this blog, I will be talking about some basic plugins and configs that would boost my basic editing experience.</p><span id="more"></span><h3 id="Theme"><a href="#Theme" class="headerlink" title="Theme"></a>Theme</h3><p>Every coder wants their IDE to look fancy, I think everyone of us can admit that and I’m no exception. After carefully evaluating a lot of popular themes out there the theme I chose in the end is the dark mode of <a href="https://github.com/morhetz/gruvbox">gruvbox</a> for two reasons:</p><ul><li>It is the default theme for <a href="https://spacevim.org/">SpaceVim</a>, which is one of the more successful Vim config project in the community.</li><li>Out of all the themes I have tried, gruvbox has the best syntax highlighting, especially for some less popular languages like Groovy.</li></ul><p>Gruvbox can be installed just like a plugin, and to use the dark mode of it I have to set the background to dark. Finally enabling terminal GUI colors will give the theme a richer set of colors to work with:</p><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">call</span> plug#begin()</span><br><span class="line"> <span class="comment">" other plugins</span></span><br><span class="line"> Plug <span class="string">'morhetz/gruvbox'</span></span><br><span class="line"><span class="keyword">call</span> plug#end()</span><br><span class="line"></span><br><span class="line"><span class="keyword">colo</span> gruvbox</span><br><span class="line"><span class="keyword">set</span> background=dark</span><br><span class="line"><span class="keyword">set</span> termguicolors</span><br></pre></td></tr></table></figure><p>Here is a screenshot of the theme:</p><p><img src="/2023/05/04/vim-blog-3/3.png"></p><h3 id="Line-Numbers"><a href="#Line-Numbers" class="headerlink" title="Line Numbers"></a>Line Numbers</h3><p>No one codes without line numbers. Line numbers help us to better navigate through files, especially during collaborative coding. Furthermore line numbers have an added benefit for Vim which is making navigation using the j and k key bindings much more efficient.</p><p>Vim supports three types of line numbers. The first type is the usual line number that one would normally see in other IDEs. It can be turned on by adding this to <code>.vimrc</code> file:</p><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">set</span> <span class="keyword">number</span></span><br></pre></td></tr></table></figure><p>This type of line number is called absolute line number, as it shows the absolute position of each line of code.</p><p>The second type of line number is called relative line number, which can be turned on by adding this to <code>.vimrc</code> file:</p><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">set</span> relativenumber</span><br></pre></td></tr></table></figure><p>For relative line number, the line number on each line of code is the relative distance to the line where your cursor is on. This type of line number is extremely useful for Vim as I navigate through files using the “number + j” or “number + k” key bindings.</p><p>If I combine the absolute and relative line number by adding both of them to <code>.vimrc</code> file:</p><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">set</span> <span class="keyword">number</span></span><br><span class="line"><span class="keyword">set</span> relativenumber</span><br></pre></td></tr></table></figure><p>I create the third type of line number, which is called hybrid line number. For this type of line number, it will display relative line numbers for any other lines in the file, except the line where your cursor is on in which it will display absolute line number. Here is a screenshot:</p><p><img src="/2023/05/04/vim-blog-3/1.png"></p><p>Hybrid line number combines the best of absolute and relative line number, allows me to benefit from both the j and k key binding navigation Vim provides and knowing exactly where I’m at within the file. And because of that, hybrid line number is my default config for Vim when it comes to line numbers.</p><h3 id="Indentation"><a href="#Indentation" class="headerlink" title="Indentation"></a>Indentation</h3><p>Personally I’m not a big fan of re-configuring indentation for different types of files. So I found a very interesting plugin called <a href="https://github.com/tpope/vim-sleuth">vim-sleuth</a> which automatically adjusts indentation for me:</p><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">call</span> plug#begin()</span><br><span class="line"> <span class="comment">" other plugins</span></span><br><span class="line"> Plug <span class="string">'tpope/vim-sleuth'</span></span><br><span class="line"><span class="keyword">call</span> plug#end()</span><br></pre></td></tr></table></figure><p>On top of vim-sleuth, <a href="https://github.com/preservim/vim-indent-guides">vim-indent-guides</a> is another plugin I use to better visualize indentations. This plugin enables displaying of indent levels and is great for when I work with languages like Python:</p><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">call</span> plug#begin()</span><br><span class="line"> <span class="comment">" other plugins</span></span><br><span class="line"> Plug <span class="string">'nathanaelkane/vim-indent-guides'</span></span><br><span class="line"><span class="keyword">call</span> plug#end()</span><br><span class="line"></span><br><span class="line"><span class="keyword">let</span> <span class="variable">g:indent_guides_enable_on_vim_startup</span> = <span class="number">1</span></span><br></pre></td></tr></table></figure><p>Here is a screenshot of vim-indent-guides:</p><p><img src="/2023/05/04/vim-blog-3/2.png"></p><h3 id="Syntax-Highlighting"><a href="#Syntax-Highlighting" class="headerlink" title="Syntax Highlighting"></a>Syntax Highlighting</h3><p>Obviously when I’m coding, I will want syntax highlighting on for my code:</p><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">syntax</span> <span class="keyword">on</span></span><br></pre></td></tr></table></figure><h3 id="Backspace"><a href="#Backspace" class="headerlink" title="Backspace"></a>Backspace</h3><p>In Vim, backspace does not function as a key binding for deletion by default. This can be changed by adding the following to <code>.vimrc</code> file:</p><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">set</span> backspace=<span class="built_in">indent</span>,eol,start</span><br></pre></td></tr></table></figure><h3 id="Round-Square-Curly-Brackets"><a href="#Round-Square-Curly-Brackets" class="headerlink" title="Round/Square/Curly Brackets"></a>Round/Square/Curly Brackets</h3><p>When coding, it is almost inevitable to work with the three types of brackets: round, square, and curly. To make the editing experience for them better on Vim there are two plugins I use.</p><p>The first one is called <a href="https://github.com/jiangmiao/auto-pairs">auto-pairs</a>. This small plugin will insert and delete brackets in pair, which is a feature that most other modern IDEs have by default.</p><p>The second one is called <a href="https://github.com/luochen1990/rainbow">rainbow</a>. This plugin helps me to read code with many nested levels of brackets by showing different levels of them in different colors.</p><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">call</span> plug#begin()</span><br><span class="line"> <span class="comment">" other plugins</span></span><br><span class="line"> Plug <span class="string">'jiangmiao/auto-pairs'</span></span><br><span class="line"> Plug <span class="string">'luochen1990/rainbow'</span></span><br><span class="line"><span class="keyword">call</span> plug#end()</span><br><span class="line"></span><br><span class="line"><span class="keyword">let</span> <span class="variable">g:rainbow_active</span> = <span class="number">1</span></span><br></pre></td></tr></table></figure><p>Here is a screenshot of rainbow brackets:</p><p><img src="/2023/05/04/vim-blog-3/4.png"></p><p>These two plugins combined with Vim’s builtin key binding for % have helped me deal with any situation when it comes to brackets.</p><h3 id="Search-Highlight"><a href="#Search-Highlight" class="headerlink" title="Search Highlight"></a>Search Highlight</h3><p>When I use Vim’s builtin search functionality, I always want my searches to be highlighted. So I added this line to my <code>.vimrc</code> file:</p><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">set</span> hlsearch</span><br></pre></td></tr></table></figure><p>Here is a screenshot of the effect:</p><p><img src="/2023/05/04/vim-blog-3/5.png"></p><h3 id="Conclusion"><a href="#Conclusion" class="headerlink" title="Conclusion"></a>Conclusion</h3><p>In this blog, I have covered some plugins and configs that improve quality of life when it comes to the basic editing experience of Vim.</p><p>In the next article, I will talk about buffers, one of the core features for any IDE that enables working with multiple files and how I make my Vim experience better when working with them.</p>]]></content>
<summary type="html"><h3 id="Finding-Plugins"><a href="#Finding-Plugins" class="headerlink" title="Finding Plugins"></a>Finding Plugins</h3><p>In the previous blog, I discussed how to use vim-plug to manage plugins. So the question now is how do I find all those fancy plugins out there?</p>
<p>On top of just simply google and browse through GitHub, there is a website called <a href="https://vimawesome.com/">VimAwesome</a> which allows me to browse and search different plugins by categories.</p>
<p>In this blog, I will be talking about some basic plugins and configs that would boost my basic editing experience.</p></summary>
<category term="Coding" scheme="https://chinapandaman.github.io/tags/Coding/"/>
<category term="IDE" scheme="https://chinapandaman.github.io/tags/IDE/"/>
<category term="Vim" scheme="https://chinapandaman.github.io/tags/Vim/"/>
</entry>
<entry>
<title>Vim as Primary IDE - Boilerplate</title>
<link href="https://chinapandaman.github.io/2023/05/02/vim-blog-2/"/>
<id>https://chinapandaman.github.io/2023/05/02/vim-blog-2/</id>
<published>2023-05-02T22:52:55.000Z</published>
<updated>2023-05-02T21:32:24.300Z</updated>
<content type="html"><![CDATA[<h3 id="Reference"><a href="#Reference" class="headerlink" title="Reference"></a>Reference</h3><p>Before I start, make sure to checkout <a href="https://github.com/chinapandaman/vim-config">the GitHub repository</a> which contains all the configs and scripts I will be covering in this series of blogs.</p><h3 id="Plugin-Managing"><a href="#Plugin-Managing" class="headerlink" title="Plugin Managing"></a>Plugin Managing</h3><p>To make Vim as beefy as an IDE, it is not possible without plugins, at least for me. Vim has a large and active community and there are thousands of plugins out there that can boost my developing experience.</p><p>To install plugins I will first need a plugin manager. In my case the plugin manager I picked is <a href="https://github.com/junegunn/vim-plug">vim-plug</a>.</p><p>Installing vim-plug is quite simple:</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">curl -fLo ~/.vim/autoload/plug.vim --create-dirs https://raw.githubusercontent.com/junegunn/vim-plug/master/plug.vim</span><br></pre></td></tr></table></figure><p>The actual usage of vim-plug revolves around modifying my <code>.vimrc</code> file. The below snippet exists at the very top of it:</p><span id="more"></span><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">call</span> plug#begin()</span><br><span class="line"> <span class="comment">" a list of plugins to install</span></span><br><span class="line"> Plug <span class="string">'morhetz/gruvbox'</span></span><br><span class="line"><span class="keyword">call</span> plug#end()</span><br></pre></td></tr></table></figure><p>By adding plugins between the two <code>call</code> blocks, vim-plug allows me to easily add/change/remove plugins via my <code>.vimrc</code> file.</p><p>After defining plugins, I can simply install them by running <code>:PlugInstall</code> within Vim. And after the installation has completed, I can check the status of plugins by running <code>:PlugStatus</code>. Here is a screenshot of the UI:</p><p><img src="/2023/05/02/vim-blog-2/1.png"></p><p>So now I have setup vim-plug which allows me to utilize plugins to expand Vim.</p><h3 id="Making-My-Configuration-Portable"><a href="#Making-My-Configuration-Portable" class="headerlink" title="Making My Configuration Portable"></a>Making My Configuration Portable</h3><p>The other thing that came to my mind is that I want to make my Vim configuration as portable as I can. So whenever I’m on a new machine it will require minimum effort to restore my config.</p><p>To achieve this I will need to write some <code>bash</code> script. Here is what I have:</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># setup.sh</span></span><br><span class="line"><span class="built_in">rm</span> -rf ~/.vim/</span><br><span class="line">curl -fLo ~/.vim/autoload/plug.vim --create-dirs https://raw.githubusercontent.com/junegunn/vim-plug/master/plug.vim</span><br><span class="line"><span class="built_in">cp</span> .vimrc ~/.vimrc</span><br><span class="line">vim +PlugInstall +qa</span><br></pre></td></tr></table></figure><p>Let’s look at this line by line.</p><p>The first line removes the current existed <code>~/.vim/</code> directory, allowing me to have a fresh, plugin free Vim environment for my own config.</p><p>The second line is the same command that installs vim-plug which we have covered in the last section.</p><p>The third line copies my <code>.vimrc</code> to where the file should be at on the designated machine. In the future <code>.vimrc</code> will contain a lot of configs I will be making. But for now it simply has all the needed plugins managed by vim-plug.</p><p>The last line is a sequence of commands. It first launches Vim, runs the command <code>:PlugInstall</code> which installs all the plugins via vim-plug, and finally runs the command <code>:qa</code> which quits Vim. It may seem a bit complex but all it’s doing is installing plugins.</p><p>From here, I can make use of my own Vim configuration on any machine by just running:</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">bash setup.sh</span><br></pre></td></tr></table></figure><p>And the <code>bash</code> script will cleanup the Vim environment, install vim-plug, install my vim config, and install all plugins needed for the config.</p><h3 id="Conclusion"><a href="#Conclusion" class="headerlink" title="Conclusion"></a>Conclusion</h3><p>In this blog, I have gone over some boilerplate steps that would make configuring and making use of a Vim config easier.</p><p>In the next article I will talk about some simple configurations that would enhance the basic editing experience.</p>]]></content>
<summary type="html"><h3 id="Reference"><a href="#Reference" class="headerlink" title="Reference"></a>Reference</h3><p>Before I start, make sure to checkout <a href="https://github.com/chinapandaman/vim-config">the GitHub repository</a> which contains all the configs and scripts I will be covering in this series of blogs.</p>
<h3 id="Plugin-Managing"><a href="#Plugin-Managing" class="headerlink" title="Plugin Managing"></a>Plugin Managing</h3><p>To make Vim as beefy as an IDE, it is not possible without plugins, at least for me. Vim has a large and active community and there are thousands of plugins out there that can boost my developing experience.</p>
<p>To install plugins I will first need a plugin manager. In my case the plugin manager I picked is <a href="https://github.com/junegunn/vim-plug">vim-plug</a>.</p>
<p>Installing vim-plug is quite simple:</p>
<figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">curl -fLo ~/.vim/autoload/plug.vim --create-dirs https://raw.githubusercontent.com/junegunn/vim-plug/master/plug.vim</span><br></pre></td></tr></table></figure>
<p>The actual usage of vim-plug revolves around modifying my <code>.vimrc</code> file. The below snippet exists at the very top of it:</p></summary>
<category term="Coding" scheme="https://chinapandaman.github.io/tags/Coding/"/>
<category term="IDE" scheme="https://chinapandaman.github.io/tags/IDE/"/>
<category term="Vim" scheme="https://chinapandaman.github.io/tags/Vim/"/>
</entry>
<entry>
<title>Vim as Primary IDE - Intro</title>
<link href="https://chinapandaman.github.io/2023/04/28/vim-blog-1/"/>
<id>https://chinapandaman.github.io/2023/04/28/vim-blog-1/</id>
<published>2023-04-28T19:31:10.000Z</published>
<updated>2023-05-02T21:32:16.624Z</updated>
<content type="html"><![CDATA[<p>Throughout the years I have been coding, different IDEs/editors have attracted me and improved my workflow in different ways. Back in college, the very first editor I was taught to use a lot for CS classes was the good old Vim that comes with most Linux distributions. At the time I did not spend much time hacking it and simply learned a lot of the basic ways of using it. But it did serve its purposes for a college student with general syntax highlighting for most main stream languages and allowed me completing most of my projects.</p><span id="more"></span><p>When I first started my career, I came to realization that I finally need something more beefy than just an editor. And this is around the time where I was introduced to the JetBrains IDEs, in particular PyCharm. The rich features bundled in even just the community edition of this IDE such as GUI debugging, refactoring, searches and static analysis are amazing. Not to mention with the professional edition (which most companies would be more than happy to purchase for their software engineers), you can have supports on some of the most popular frameworks such as Django and Flask, and it has an awesome GUI database tool which makes day-to-day DB workload just that much simpler. To me, PyCharm is truly, like what JetBrains suggests, the essential tool for software developers. Throughout these years, I have tried different editors such as Atom, Sublime, vscode, etc. However none of these, even with a rich set of plugins, can compete with the kind of developing experience PyCharm brought to me.</p><p>After being spoiled for a long time by JetBrains IDEs, things finally started changing a little bit. I have recently acquired a new job and there are certain things with the new role that makes me shift away from an IDE with lots of out-of-the-box features. First of all the system admin restricts a lot of accesses on the laptops given to the employees, making installing any software quite a process. These restrictions however revolves mostly around the GUIs and does not stop you from doing a lot of things using the terminal. On top of that the kind of work I’m doing with this role is more and more based on terminal than GUIs. Finally unlike a lot of my previous roles where the tech stacks are usually monolithic, this time I have to use a wide variety of languages and frameworks. Ultimately this means I have to look for something that’s more general purpose than a monolithic IDE.</p><p>All and all I have decided that instead of something like PyCharm I need another tool that can boost my development experience with the new role. But I also said I didn’t really like a lot of the editors I tried like say vscode. And this is the moment I put my eyes back on my good old friend from college, Vim. I have used Vim heavily back in the days and know how strong it is when it comes to editing. And despite the fact that I have been a bandwagon on the IDE side, I’m also aware how configurable Vim can be and there have been projects like SpaceVim I know of which turns Vim into an IDE. And obviously Vim, even though it can be run as a GUI, is primarily terminal based.</p><p>So finally I made the decision of switching back to Vim. And my goal is ultimately making developing using Vim a similar if not the same experience as an IDE like PyCharm, but also more general purpose and terminal based. In this series of blogs I will do my best to present how I progressively made Vim better and in the end my primary tool of coding for both work and personal projects today.</p>]]></content>
<summary type="html"><p>Throughout the years I have been coding, different IDEs&#x2F;editors have attracted me and improved my workflow in different ways. Back in college, the very first editor I was taught to use a lot for CS classes was the good old Vim that comes with most Linux distributions. At the time I did not spend much time hacking it and simply learned a lot of the basic ways of using it. But it did serve its purposes for a college student with general syntax highlighting for most main stream languages and allowed me completing most of my projects.</p></summary>
<category term="Coding" scheme="https://chinapandaman.github.io/tags/Coding/"/>
<category term="IDE" scheme="https://chinapandaman.github.io/tags/IDE/"/>
<category term="Vim" scheme="https://chinapandaman.github.io/tags/Vim/"/>
</entry>
</feed>