Skip to content

Commit a45c3a8

Browse files
Rangeet PanRangeet Pan
Rangeet Pan
authored and
Rangeet Pan
committed
updating notebooks
1 parent 0efc6c2 commit a45c3a8

2 files changed

Lines changed: 240 additions & 7 deletions

File tree

‎docs/examples/java/generate_unit_tests.ipynb‎

Lines changed: 238 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,242 @@
1010
"metadata": {
1111
"collapsed": false
1212
},
13-
"id": "b8da254b236c8a4b"
13+
"id": "ee51e198aaebcd9b"
14+
},
15+
{
16+
"cell_type": "markdown",
17+
"source": [
18+
"# Using CLDK to generate JUnit tests\n",
19+
"\n",
20+
"In this tutorial, we will use CLDK to generate a JUnit test for all the methods in a Java Application.\n",
21+
"\n",
22+
"By the end of this tutorial, you will have a JUnit test for all the methods in a Java application. You'll be able to explore some of the benefits of using CLDK to perform fast and easy program analysis and build a LLM-based test generator.\n",
23+
"\n",
24+
"You will learn how to do the following:\n",
25+
"\n",
26+
"<ol>\n",
27+
"<li> Create a new instance of the CLDK class.\n",
28+
"<li> Create an analysis object over the Java application.\n",
29+
"<li> Iterate over all the files in the project.\n",
30+
"<li> Iterate over all the classes in the file.\n",
31+
"<li> Iterate over all the methods in the class.\n",
32+
"<li> Get the code body of the method.\n",
33+
"<li> Initialize the treesitter utils for the class file content.\n",
34+
"<li> Sanitize the class for analysis.\n",
35+
"</ol>\n",
36+
"Next, we will write a couple of helper methods to:\n",
37+
"\n",
38+
"<ol>\n",
39+
"<li> Format the instruction for the given focal method and class.\n",
40+
"<li> Prompts the local model on Ollama.\n",
41+
"<li> Prints the instruction and LLM output.\n",
42+
"</ol>"
43+
],
44+
"metadata": {
45+
"collapsed": false
46+
},
47+
"id": "428dbbfa206f5417"
48+
},
49+
{
50+
"cell_type": "markdown",
51+
"source": [
52+
"## Prequisites\n",
53+
"\n",
54+
"Before we get started, let's make sure you have the following installed:\n",
55+
"\n",
56+
"<ol>\n",
57+
"<li> Python 3.11 or later\n",
58+
"<li> Ollama 0.3.4 or later\n",
59+
"</ol>\n",
60+
"We will use ollama to spin up a local granite model that will act as our LLM for this turorial."
61+
],
62+
"metadata": {
63+
"collapsed": false
64+
},
65+
"id": "f619a9379b9dd006"
66+
},
67+
{
68+
"cell_type": "markdown",
69+
"source": [
70+
"### Prerequisite 1: Install ollama\n",
71+
"\n",
72+
"If you don't have ollama installed, please download and install it from here: [Ollama](https://ollama.com/download).\n",
73+
"Once you have ollama, start the server and make sure it is running.\n",
74+
"If you're on MacOS, Linux, or WSL, you can check to make sure the server is running by running the following command:"
75+
],
76+
"metadata": {
77+
"collapsed": false
78+
},
79+
"id": "3485879a7733bcba"
80+
},
81+
{
82+
"cell_type": "code",
83+
"execution_count": null,
84+
"outputs": [],
85+
"source": [
86+
"systemctl status ollama"
87+
],
88+
"metadata": {
89+
"collapsed": false
90+
},
91+
"id": "2f67be6c8c024e12"
92+
},
93+
{
94+
"cell_type": "markdown",
95+
"source": [
96+
"If not, you may have to start the server manually. You can do this by running the following command:"
97+
],
98+
"metadata": {
99+
"collapsed": false
100+
},
101+
"id": "273e60ca598e0a53"
102+
},
103+
{
104+
"cell_type": "code",
105+
"execution_count": null,
106+
"outputs": [],
107+
"source": [
108+
"systemctl start ollama"
109+
],
110+
"metadata": {
111+
"collapsed": false
112+
},
113+
"id": "cc6877ce338e9102"
114+
},
115+
{
116+
"cell_type": "markdown",
117+
"source": [
118+
"Once ollama is up and running, you can download the latest version of the Granite 8b Instruct model by running the following command:\n",
119+
"\n",
120+
"There are other granite versions available, but for this tutorial, we will use the Granite 8b Instruct model. You if prefer to use a different version, you can replace `8b-instruct` with any of the other [versions](https://ollama.com/library/granite-code/tags)."
121+
],
122+
"metadata": {
123+
"collapsed": false
124+
},
125+
"id": "c024dc7ec2869a72"
126+
},
127+
{
128+
"cell_type": "code",
129+
"execution_count": null,
130+
"outputs": [],
131+
"source": [
132+
"ollama pull granite-code:8b-instruct"
133+
],
134+
"metadata": {
135+
"collapsed": false
136+
},
137+
"id": "5ad0e8ac33c7108e"
138+
},
139+
{
140+
"cell_type": "markdown",
141+
"source": [
142+
"Let's make sure the model is downloaded by running the following command:"
143+
],
144+
"metadata": {
145+
"collapsed": false
146+
},
147+
"id": "14f9946fdc5e2025"
148+
},
149+
{
150+
"cell_type": "code",
151+
"execution_count": null,
152+
"outputs": [],
153+
"source": [
154+
"ollama run granite-code:8b-instruct \\\"Write a python function to print 'Hello, World!'"
155+
],
156+
"metadata": {
157+
"collapsed": false
158+
},
159+
"id": "e3410ce4d0afa788"
160+
},
161+
{
162+
"cell_type": "markdown",
163+
"source": [
164+
"### Prerequisite 3: Install ollama Python SDK"
165+
],
166+
"metadata": {
167+
"collapsed": false
168+
},
169+
"id": "d8c0224c3c4ecf4d"
170+
},
171+
{
172+
"cell_type": "code",
173+
"execution_count": null,
174+
"outputs": [],
175+
"source": [
176+
"pip install ollama"
177+
],
178+
"metadata": {
179+
"collapsed": false
180+
},
181+
"id": "5539b5251aee5642"
182+
},
183+
{
184+
"cell_type": "markdown",
185+
"source": [
186+
"### Prerequisite 4: Install CLDK\n",
187+
"CLDK is avaliable on github at github.com/IBM/codellm-devkit.git. You can install it by running the following command:"
188+
],
189+
"metadata": {
190+
"collapsed": false
191+
},
192+
"id": "cea573e625257581"
193+
},
194+
{
195+
"cell_type": "code",
196+
"execution_count": null,
197+
"outputs": [],
198+
"source": [
199+
"pip install git+https://github.com/IBM/codellm-devkit.git"
200+
],
201+
"metadata": {
202+
"collapsed": false
203+
},
204+
"id": "eeb38b312427329d"
205+
},
206+
{
207+
"cell_type": "markdown",
208+
"source": [
209+
"### Step 1: Get the sample Java application\n",
210+
"For this tutorial, we will use apache commons cli. You can download the source code to a temporary directory by running the following command:"
211+
],
212+
"metadata": {
213+
"collapsed": false
214+
},
215+
"id": "ca7682c71d844b68"
216+
},
217+
{
218+
"cell_type": "code",
219+
"execution_count": null,
220+
"outputs": [],
221+
"source": [
222+
"wget https://github.com/apache/commons-cli/archive/refs/tags/rel/commons-cli-1.7.0.zip -O /tmp/commons-cli-1.7.0.zip && unzip -o /tmp/commons-cli-1.7.0.zip -d /tmp"
223+
],
224+
"metadata": {
225+
"collapsed": false
226+
},
227+
"id": "a4d08ca64b9dbccb"
228+
},
229+
{
230+
"cell_type": "markdown",
231+
"source": [
232+
"The project will now be extracted to `/tmp/commons-cli-rel-commons-cli-1.7.0`. We'll remove these files later, so don't worry about the location."
233+
],
234+
"metadata": {
235+
"collapsed": false
236+
},
237+
"id": "51d30f3eb726afc0"
238+
},
239+
{
240+
"cell_type": "markdown",
241+
"source": [
242+
"### Building a JUnit test generator using CLDK and Granite Code Instruct Model\\n\n",
243+
"Now that we have all the prerequisites installed, let's start building a JUnit test generator using CLDK and the Granite Code Instruct Model."
244+
],
245+
"metadata": {
246+
"collapsed": false
247+
},
248+
"id": "98e69eb0bccedfc9"
14249
},
15250
{
16251
"cell_type": "markdown",
@@ -119,8 +354,8 @@
119354
"source": [
120355
"# Create a new instance of the CLDK class\n",
121356
"cldk = CLDK(language=\"java\")\n",
122-
"# Create an analysis object over the java application. Provide the application path using JAVA_APP_PATH\n",
123-
"analysis = cldk.analysis(project_path=\"JAVA_APP_PATH\", analysis_level=AnalysisLevel.symbol_table)\n",
357+
"# Create an analysis object over the java application. Provide the application path.\n",
358+
"analysis = cldk.analysis(project_path=\"/tmp/commons-cli-rel-commons-cli-1.7.0\", analysis_level=AnalysisLevel.symbol_table)\n",
124359
"# Go through all the classes in the application\n",
125360
"for class_name in analysis.get_classes():\n",
126361
" class_details = analysis.get_class(qualified_class_name=class_name)\n",

‎docs/examples/java/validating_code_translation.ipynb‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,6 @@
55
"execution_count": null,
66
"outputs": [],
77
"source": [
8-
"from cldk.utils.treesitter.tree_sitter_utils import TreeSitterUtils\n",
98
"!pip install ollama"
109
],
1110
"metadata": {
@@ -30,6 +29,8 @@
3029
"execution_count": null,
3130
"outputs": [],
3231
"source": [
32+
"from cldk.analysis.python.treesitter import PythonSitter\n",
33+
"from cldk.analysis.java.treesitter import JavaSitter\n",
3334
"import ollama\n",
3435
"from cldk import CLDK\n",
3536
"from cldk.analysis import AnalysisLevel"
@@ -112,9 +113,6 @@
112113
"execution_count": null,
113114
"outputs": [],
114115
"source": [
115-
"from cldk.analysis.python.treesitter import PythonSitter\n",
116-
"from cldk.analysis.java.treesitter import JavaSitter\n",
117-
"\n",
118116
"# Create a new instance of the CLDK class\n",
119117
"cldk = CLDK(language=\"java\")\n",
120118
"# Create an analysis object over the java application. Provide the application path using JAVA_APP_PATH\n",

0 commit comments

Comments
 (0)