-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathindex.html
287 lines (268 loc) · 24 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
<!DOCTYPE html>
<html lang="en_us">
<head>
<title>Pickles!</title>
<meta charset="utf-8" />
<meta name="generator" content="Pelican" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<link rel="stylesheet" href="/static/css/main.css" />
<link rel="stylesheet" href="/static/css/theme.css" />
<link rel="icon" href="/images/yazani/yazani_1_extracted_bg_big_eyes_cropped.png" type="image/png" />
<link rel="apple-touch-icon" href="/images/yazani/yazani_1_extracted_bg_big_eyes_cropped.png" type="image/png" />
<script src="/static/misc.js"></script>
<script src="/blog/banner_image.js"></script>
<meta name="tags" content="programming, c, language-design" />
<meta property="og:site_name" content="dragoncoder047’s blog" />
<meta property="og:title" content="Pickles!" />
<meta property="og:description" content="I’ve been playing around a little bit with LIL on my ESP32 arduino. It works, but there are a few things I don’t like. LIL isn’t object-oriented by default, so I can’t do a lot of what I am used to writing code in Javascript and …" />
<meta property="og:image" content="/images/yazani/yazani_1_extracted_bg.png" />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://dragoncoder047.github.io/blog/2023/pickles" />
<meta property="og:locale" content="['']" />
<meta name="twitter:card" content="summary_large_image" />
<meta name="twitter:title" content="dragoncoder047’s blog - Pickles!" />
<meta name="twitter:description" content="I’ve been playing around a little bit with LIL on my ESP32 arduino. It works, but there are a few things I don’t like. LIL isn’t object-oriented by default, so I can’t do a lot of what I am used to writing code in Javascript and …" />
<meta name="twitter:image" content="/images/yazani/yazani_1_extracted_bg.png" />
<!-- PrismJS -->
<script src="/static/prism.js" data-autoloader-path="https://cdn.jsdelivr.net/npm/prismjs@v1.x/components/"></script>
<script src="/static/prism-runbutton.js"></script>
<script src="/phoo/prism-phoo.js"></script> <!-- /PrismJS -->
<!-- Katex -->
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/katex@0.16.11/dist/katex.css" type="text/css" />
<script defer src="https://cdn.jsdelivr.net/npm/katex@0.16.11/dist/katex.js"></script>
<script defer src="https://cdn.jsdelivr.net/npm/katex@0.16.11/dist/contrib/mhchem.js"></script>
<script defer src="https://cdn.jsdelivr.net/npm/katex@0.16.11/dist/contrib/auto-render.js"></script>
<link href="https://cdn.jsdelivr.net/npm/katex@0.16.11/dist/contrib/copy-tex.css" rel="stylesheet" type="text/css" />
<script src="https://cdn.jsdelivr.net/npm/katex@0.16.11/dist/contrib/copy-tex.js"></script>
<script>
window.addEventListener("DOMContentLoaded", () => {
renderMathInElement(document.body,
{
delimiters: [
{ left: "$$", right: "$$", display: true },
{ left: "$", right: "$", display: false },
{ left: "\\begin{align}", right: "\\end{align}", display: true },
]
}
);
});
</script> <!-- /Katex -->
</head>
<body class="match-braces rainbow-braces">
<header>
<a href="https://dragoncoder047.github.io/blog" class="flex-row"><div class="flex-row"><img src="/images/yazani/yazani_1_extracted_bg.png" style="max-height:10em" id="banner-image" /><div id="sitename-text"><h1>dragoncoder047’s blog</h1><h2>random thoughts about nonrandom things</h2></div></div></a>
<nav>
<ul>
<li><a href="https://dragoncoder047.github.io/blog/">Home</a></li>
<li><a href="https://dragoncoder047.github.io/blog/archives">Archives</a>
</li>
<li><a href="https://dragoncoder047.github.io/blog/tags">By tag</a>
</li>
<li><a href="/">Site root</a>
</li>
<li><a href="#">Projects</a>
<ul>
<li><a href="https://dragoncoder047.github.io/thuepaste">Thuepaste</a>
</li>
<li><a href="https://dragoncoder047.github.io/armdroid">Armdroid</a>
</li>
<li><a href="https://dragoncoder047.github.io/langton-music">Langton's Ant Music</a>
</li>
<li><a href="https://dragoncoder047.github.io/schemascii">Schemascii</a>
</li>
<li><a href="https://dragoncoder047.github.io/parasite">Parasite</a>
</li>
</ul>
</li>
<li>
<form action="https://www.google.com/search" method="GET">
<input name="q" type="search" placeholder="Search"></input>
<input type="hidden" name="as_sitesearch" value="https://dragoncoder047.github.io/blog"></input>
<input type="submit" value="Search"></input>
</form>
</li>
</ul>
</nav>
</header>
<main>
<h1><a href="https://dragoncoder047.github.io/blog/2023/pickles" rel="bookmark" title="Permalink to this page">Pickles!</a></h1>
<div class="flex-row">
<span style="flex: 1">← Previous:
<a href="https://dragoncoder047.github.io/blog/2023/schemascii-0">
Schemascii ± 0
</a>
</span>
<span>Next:
<a href="https://dragoncoder047.github.io/blog/2023/well-i-got-something">
Well, I Got Something...
</a> →
</span>
</div>
<div class="post-info">
Posted <time class="published" datetime="2023-02-23T00:00:00-05:00">Thu 23 February 2023</time>
<address>By
<a href="https://dragoncoder047.github.io/blog/">dragoncoder047</a>
</address>
<div class="tags">
Tags:
<a href="https://dragoncoder047.github.io/blog/tag/c">c</a>
<a href="https://dragoncoder047.github.io/blog/tag/language-design">language-design</a>
<a href="https://dragoncoder047.github.io/blog/tag/programming">programming</a>
</div>
</div>
<p><em>This post is part 1 of the pickle series:</em></p>
<ol class="series">
<li class="active">
<a href='https://dragoncoder047.github.io/blog/2023/pickles'>Pickles!</a>
</li>
<li >
<a href='https://dragoncoder047.github.io/blog/2023/manual-memory-management-madness'>Manual Memory Management Madness</a>
</li>
<li >
<a href='https://dragoncoder047.github.io/blog/2023/pickle-tokenizer'>Pickle Tokenizer</a>
</li>
<li >
<a href='https://dragoncoder047.github.io/blog/2023/yet-another-garbage-collector'>Yet Another Garbage Collector</a>
</li>
<li >
<a href='https://dragoncoder047.github.io/blog/2023/powerful-pickle-pattern-matching'>Powerful PICKLE Pattern Matching</a>
</li>
<li >
<a href='https://dragoncoder047.github.io/blog/2023/pickle-has-regular-expressions-apparently'>PICKLE Has Regular Expressions, Apparently</a>
</li>
<li >
<a href='https://dragoncoder047.github.io/blog/2023/its-september'>It's September!!</a>
</li>
<li >
<a href='https://dragoncoder047.github.io/blog/2023/continuations-and-the-thunk-queue'>Continuations and the thunk queue</a>
</li>
<li >
<a href='https://dragoncoder047.github.io/blog/2024/the-lesser-of-two-evils'>The Lesser of Two Evils</a>
</li>
<li >
<a href='https://dragoncoder047.github.io/blog/2024/a-hash-mapped-mess'>A Hash-Mapped Mess</a>
</li>
</ol>
<p>I’ve been playing around a little bit with LIL on my ESP32 arduino. It works, but there are a few things I don’t like. LIL isn’t object-oriented by default, so I can’t do a lot of what I am used to writing code in Javascript and Python. LIL also forces the result of every expression (the <code class="language-tcl highlight">expr</code> command) to be a number (so I can’t do a Python-esque <code class="language-python highlight">"foo" * 3</code> to repeat a string), has no operator overloading because nothing is an object and supports overloading anyway, and (most importantly) doesn’t support lexical closures.</p>
<p>LIL also does not have what I think is a really useful operator: the <code>|></code> pipe operator, for function chaining. Javascript is working on one (it was <a href="https://github.com/tc39/proposal-pipeline-operator">proposed</a>) and it will be incredibly useful if and when it actually goes through.</p>
<p>There are several things about LIL that I do like, though: it has a nice recursive parser, a very simple hashmap implementation, and the use of native C arrays (i.e. double-pointer-indirection, two asterisks) to implement list types. Back when I was working on TEHSSL, I actually decided against the last one because I (erroneously) feared that if I had to <code class="language-c highlight">realloc()</code> the array, it would move objects and corrupt the pointers of every object that pointed to them. When I looked at LIL, it finally dawned on me that the array was an array of <em>pointers</em> to the objects, and <code class="language-c highlight">realloc()</code>‘ing the array would only move the <em>pointers</em>, not the objects, and so it’s okay to use as long as you don’t overrun your array before you <code class="language-c highlight">realloc()</code> it.</p>
<p>LIL also forces the use of curly brackets (<code class="language-tcl highlight">{ }</code>) for delimiting code blocks (which are really just strings), which as a Python programmer, I don’t like. I really like the cleanness of the colon-plus-indent of Python.</p>
<p>As-is, there is one feature that I want in my scripting language that I don’t see in any other language. It is the ability to <strong>define custom operators</strong>.</p>
<p>Suppose I am writing a little DSL for routing audio nodes (which seems a possibility in the future, I’m not sure why). Albeit, with a Tcl-like language, I can just have the user pass in a string of the code, and process it as the DSL. But then I’d be stuck writing the parser, interpreter, etc. for that all over again and I’d probably take away half of the features of the enclosing language in the process.</p>
<p>The code would be, I don’t know, using the <code><-</code> operator to link the node input to the node output. The <code><-</code> operator wouldn’t mean a darn thing to number objects (like 14 and 23.5), or anything else, for that matter, so a language that isn’t designed with this in mind would probably never implement the <code><-</code> operator.</p>
<p>However, if I can define a new operator, <code><-</code>, with a specific precedence, and then link it to a special “magic method” (taking inspiration from Python) that gets called on the objects when they are operated on by that operator (such as <code>__left_arrow__</code> for <code><-</code>), it would allow this DSL to be made in native code, and not sacrifice anything.</p>
<p>So, I think I’m at least going to try to implement my own language. I’ll be able to fall back on LIL if all goes horribly wrong, but if it works, I’ll have something that works better for me, and maybe for someone else.</p>
<p>Here are my goals:</p>
<ul>
<li>Be small enough to fit on an Arduino – specifically, an ESP32, with 16MB of program storage space and 124KB of RAM.</li>
<li>Be thread-safe and re-entrant (except for stuff like global variables, which are shared via mutexes) so I can use FreeRTOS tasks to implement co-operative threading</li>
<li>Have the aforementioned user-definable operators</li>
<li>Be object-oriented</li>
<li>Have closures, classes, lambdas, etc. that a good high-level scripting language like Python or Javascript has</li>
<li>Allow operator overloading on objects</li>
<li>Utilize dynamic typing</li>
<li>Utilize dynamic memory allocation and a garbage collector</li>
<li>Is homoiconic (code↔data interoperability)</li>
<li>Possibly implement syntactic macros</li>
</ul>
<h2 id="objects-and-inheritance">Objects and inheritance</h2>
<p>The inheritance system I’m probably going to go with is Javascript’s primitive prototype-based inheritance, even though the only way it supports multiple inheritance is through monkey-patching. The easy way I see to allow multiple inheritance is to allow an object to have, well, multiple prototypes, which are searched recursively. That would also circumvent an (intentional) bug in Python where this code would crash because it creates an ambiguous inheritance order (see if you can spot why):</p>
<pre class="highlight"><code class="language-python3">class A: pass
class B: pass
class X(A, B): pass
class Y(B, A): pass
class Crash(X, Y): pass</code></pre>
<p>Paste it into a Python console – you’ll get <code>TypeError: Cannot create a consistent MRO for bases A, B</code>. The reason why Python crashes is because <code>Crash</code> inherits from <code>X</code> and <code>Y</code>, and each of those inherit from <code>A</code> and <code>B</code>, but in a different order, so Python doesn’t know whether to look at <code>A</code> or <code>B</code> first. How my new language would go about it is by recursively searching the superclasses in the order they were specified - in the case of <code>Crash</code>, which listed <code>X</code> first, it would search in the order <code>Crash, X, A, B, Y, B, A</code> (and in theory, could automatically skip searching A and B the second time, but that’s an optimization and I’m not concerned with speed).</p>
<p>Another thing I am going to take from Javascript is that everything in the “global” scope is really properties of a global object (called <code class="language-js highlight">globalThis</code> in Javascript for some reason, per the spec) and so that simplifies scope management a whole lot.</p>
<p>One thing I am <em>not</em> going to take from Javascript is the indistinction of items and properties of objects. Think of a Python dictionary: it has methods such as <code>copy()</code>, but you cannot access them by writing <code class="language-python highlight">mydict["copy"]()</code> like would work if it was Javascript. Neither can you access dictionary items stored with <code class="language-python highlight">mydict["foobar"]</code> using the <code>mydict.foobar</code> syntax as you can in Javascript.</p>
<h2 id="the-garbage-collector">The garbage collector</h2>
<p>When I was working on TEHSSL, I had implemented a pure stop-the-world mark-and-sweep garbage collector. Mark-and-sweep collectors are <em>perfect</em>, that is, they always collect all the unreachable objects and never leak memory, but this got really hard to manage in TEHSSL. For example, the evaluator creates a lot of intermediate objects it uses once and then throws out (garbage!), and this quickly causes the number of objects to hit the high-water mark and activate the garbage collector, freeing all those objects that it’s done with, but also the ones it’s <em>not</em> done with, corrupting pointers. The common way to prevent that (used by uLisp) is to use a temporary garbage collector stack to store those objects on, but knowing when and whether to push an pop objects from that that made it extremely confusing.</p>
<p>The approach Python takes is a little different – a reference counting garbage collector. That is, each object maintains a count of how many C pointers and other objects point to it, and when it drops to zero the object is immediately freed. This does suffer from problems when objects point to each other and create a reference cycle, preventing those objects from ever being freed. Python does have a “cycle-busting” mechanism in place, but as far as I know, it isn’t described very thouroughly. Perhaps it’s just a mark-and sweep collector.</p>
<p>The garbage collector I am going to try to implement will use reference-counting for most things, and then only call a mark-and-sweep at strategic times to clean up the reference cycles, when I am sure there aren’t any intermediate objects that are still in use being pointed to by C variables. This scheme has the benefit of being able to quickly recycle memory – when the object loses all its references, it gets finalized (free system data, decrement the references of all other object it points to, etc) but doesn’t actually get freed (that is the job of the mark-and-sweep). Then when a new object is allocated, the allocator can recycle a finalized “corpse” object instead of allocating new memory. And because of the way new allocations are made known to the virtual machine, recently allocated objects are towards the top of this list and so objects with short lifetimes are recycled quickly.</p>
<h2 id="closures">Closures</h2>
<p>LIL does not have closures. That is, this code:</p>
<pre class="highlight"><code class="language-tcl">set foo "global text"
func make-closure {foo} {
set closure [func {} {print $foo}]
return $closure
}
set bar [make-closure "closed text"]
$baz</code></pre>
<p>prints “global text” – the passed-in value of “closed text” is lost.</p>
<p>But, <code>func</code> is a function itself, that creates functions when executed. Couldn’t the returned function have a pointer to the enclosing scope it was declared in, so that it could use closed-over variables? Seems like that would work, so that’s what I’m going to do in my language.</p>
<p>Bob Nystrom’s <a href="https://craftinginterpreters.com/closures.html"><em>Crafting Interpreters</em> book</a> utilizes the Lua-like method of storing “upvalues” in a special closure wrapper of a plain function, which has the advantage that closures only close over the values they actually will use, saving memory (maybe). But because of Tcl’s <code class="language-tcl highlight">upeval</code> (and LIL’s extension <code>downeval</code>) those “possibly-untouched” closure values may actually be used after all. The semantics of <code class="language-tcl highlight">upeval</code> and closures I’ll figure out when I get there. Storing the entire scope seems like an easy way to allow <code class="language-tcl highlight">upeval</code> to work okay.</p>
<h2 id="colon-blocks">Colon blocks</h2>
<p>Python is probably the first scripting language to use indentation to delimit blocks of code. Python is also bytecode-compiled, so every single function, if statement, loop, etc. is compiled into bytecode and as such all the “colon plus indent” blocks must be valid code. Also, only certain constructs can take a block after them, and those cannot be added to.</p>
<p>However, I realized it would be more than simple to add colon-delimited blocks to a Tcl-like language. The way it would work is that when the compiler sees a colon followed by a newline, it treats it like a <code>{</code>, ends the previous string if any, and begins collecting a new string. Except instead of counting the nesting depth of the <code>{</code>-<code>}</code> pairs, it would simply look at each line’s indent level and stop when it detects a dedent. The final string would have the leading indentation stripped from each line. So these two pieces of code would be equivalent:</p>
<pre class="highlight"><code class="language-tcl">while {foo} {
bar
}
baz</code></pre>
<p>And:</p>
<pre class="highlight"><code class="language-tcl">while {foo}:
bar
baz</code></pre>
<p>This saves a line, and it looks a lot cleaner (and a lot more like Python). An interesting side effect is that you can append a “block” to the end of any function, and it will take it as a string. So you could write this:</p>
<pre class="highlight"><code class="language-tcl">print:
foo
bar</code></pre>
<p>Which would print <code>foo</code> and <code>bar</code> each on their own line, and with no indentation. Neat!</p>
<h2 id="naming-it">Naming it</h2>
<p>I’m not exactly sure what I’m going to call it. It’s inspired by all of Python, Tcl, and Javascript, simultaneously. I had originally though of calling it PICL – for “Python Inspired Command Language” – but that name is already taken. Seeing that PICL is pronounced like “pickle”, I might just end up calling it PICKLE. I could also call it something competely different. I’ll figure it out when I get there.</p>
<hr />
<p><strong>Related Posts</strong></p>
<ul>
<li><a href="https://dragoncoder047.github.io/blog/2024/pointer-soup">Pointer Soup</a></li>
<li><a href="https://dragoncoder047.github.io/blog/2024/a-hash-mapped-mess">A Hash-Mapped Mess</a></li>
<li><a href="https://dragoncoder047.github.io/blog/2024/the-lesser-of-two-evils">The Lesser of Two Evils</a></li>
<li><a href="https://dragoncoder047.github.io/blog/2023/continuations-and-the-thunk-queue">Continuations and the thunk queue</a></li>
<li><a href="https://dragoncoder047.github.io/blog/2023/powerful-pickle-pattern-matching">Powerful PICKLE Pattern Matching</a></li>
</ul>
<script src="https://giscus.app/client.js"
data-repo="dragoncoder047/blog"
data-repo-id="R_kgDOHCL60w"
data-category="Post Comments"
data-category-id="DIC_kwDOHCL6084CRxCW"
data-mapping="og:title"
data-reactions-enabled="1"
data-input-position="top"
data-theme="dark"
data-lang="en"
crossorigin="anonymous"
async
></script>
<section id="extras">
<div class="blogroll">
<ul>
<li><a href="https://www.conwaylife.com/">Conwaylife.com Forums</a></li>
<li><a href="https://www.python.org/">Python</a></li>
<li><a href="http://www.ulisp.com/">uLisp</a></li>
</ul>
</div>
<div class="social">
<ul>
<li><a href="https://github.com/dragoncoder047">dragoncoder047 on GitHub</a></li>
<li><a href="https://youtube.com/@dragoncoder047">dragoncoder047 on YouTube</a></li>
<li><a href="https://instagram.com/dragoncoder047/">dragoncoder047 on Instagram</a></li>
</ul>
</div>
</section>
</main>
<footer>
<address>
Site built by <a href="https://getpelican.com/">Pelican</a>
</address>
<a href="#" onclick="window.scrollTo({top: 0, left: 0});">Back to top</a>
<!-- Google tag (gtag.js) -->
<script async src="https://www.googletagmanager.com/gtag/js?id=G-XR0F89CCGK"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag() {
dataLayer.push(arguments);
}
gtag("js", new Date());
gtag("config", "G-XR0F89CCGK");
</script>
</footer>
</body>
</html>