miloignis/kraken
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update website

Browse Source
This commit is contained in:
Nathan Braswell
2020-09-06 22:10:59 -04:00
parent 0d07d22995
commit c0eca02e43
2 changed files with 287 additions and 84 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
compile_for_web.sh
View File

@@ -1,4 +1,4 @@
#!/usr/bin/env bash
emcc ./k_prime.krak.c -o k_prime.html -s EXPORTED_FUNCTIONS='["_main"]' -s EXTRA_EXPORTED_RUNTIME_METHODS='["ccall", "cwrap"]' -s ERROR_ON_UNDEFINED_SYMBOLS=0
emcc ./k_prime.krak.c -o k_prime.html --embed-file k_prime_stdlib -s EXPORTED_FUNCTIONS='["_main"]' -s EXTRA_EXPORTED_RUNTIME_METHODS='["ccall", "cwrap"]' -s ERROR_ON_UNDEFINED_SYMBOLS=0
#emcc ./k_prime.krak.c -o k_prime.js -s EXPORTED_FUNCTIONS='["_fun_execute_code_starcharactercolonobkcbk_"]' -s EXTRA_EXPORTED_RUNTIME_METHODS='["ccall", "cwrap"]' -s ERROR_ON_UNDEFINED_SYMBOLS=0

353
index.html
View File

@@ -3,122 +3,325 @@
<meta charset="UTF-8">
<head>
<style>
h1, h2 ,h3 { line-height:1.2; }
body {
max-width: 45em;
margin: 1em auto;
padding: 0 .62em;
font: 1.2em/1.62 sans-serif;
}
h1,h2,h3 {
line-height:1.2;
}
@media print{
body{
max-width:none
th { text-align: center; }
th, td { padding: 0.5em; }
table, td {
border: 1px solid #333;
text-align: right;
}
thead, tfoot {
background-color: #000;
color: #fff;
}
#hello_editor { height: 7em; width: 70em; }
#hello_output { height: 7em; width: 70em; }
#prelude_editor { height: 54em; width: 70em; }
#prelude_output { height: 7em; width: 70em; }
#method_editor { height: 58em; width: 70em; }
#method_output { height: 7em; width: 70em; }
#bf_editor { height: 67em; width: 70em; }
#bf_output { height: 7em; width: 70em; }
#fib_editor { height: 8em; width: 70em; }
#fib_output { height: 7em; width: 70em; }
</style>
</head>
<body>
<header> <h3>Nathan Braswell's Current Programming Language / Compiler Research</h3> </header>
<h4> Current idea:</h4>
<header><h2>Nathan Braswell's Current Programming Language / Compiler Research</h2></header>
Repository: <a title="Kraken on GitHub" href="https://github.com/limvot/kraken">https://github.com/limvot/kraken</a>
<br> <br>
<b>Table of Contents:</b> <i>If you're impatient, jump to the code examples!</i>
<ul>
<li> Minimal, close to the metal Scheme (operate on words & bytes) as AST / core language
<li> Full Context-free (and eventually, context sensitive) reader macros using FUN-GLL (<a title="fun-gll paper" href="https://www.sciencedirect.com/science/article/pii/S2590118420300058">paper</a>)
<li> Implement Type Systems as Macros (<a title="type systems as macros paper 1" href="http://www.ccs.neu.edu/home/stchang/pubs/ckg-popl2017.pdf">paper, up to System Fomega</a>) (<a title="type systems as macros paper 2" href="https://www.ccs.neu.edu/home/stchang/pubs/cbtb-popl2020.pdf">second paper, up to dependent types</a>)
<li> Use above macros to create richer language and embed entire other programming languages for flawless interop/FFI (C, Go, Lua, JS, etc)
<li> Regionalized Value State Dependence Graph as backend-IR, enabling simpler implementations of powerful optimizations (<a title="RSVDG paper" href="https://arxiv.org/pdf/1912.05036.pdf">RSVDG paper</a>)
<li><a href="#concept">Concept</a>
<li><a href="#about">About</a>
<li><a href="#hello_example">Example: Hello World</a>
<li><a href="#vau_core">Vau as a core</a>
<li><a href="#method_example">Example: Implementing Methods</a>
<li><a href="#bf_example">Example: Embedding BF</a>
<li><a href="#next_steps">Next Steps</a>
</ul>
<h4> Status:</h4>
<p> Currently, I am bootstrapping this new core Lisp out of my prior compiler for my programming language, Kraken. I have implemented the first version of the FUN-GLL algorithm and have working context-free reader macros. I'll have enough to self-host this core soon, and will then use the more efficent core Lisp implementation to implement the Type Systems as Macros paper and add a type system to the new language.
<p> Below is an example of using the live grammer modification / context-free reader macros to embed the BF language into the core Lisp. The core Lisp implementation has been compiled to WebAssembly and should be able to run in your browser. Feel free to make edits and play around below.
<a name="concept"/>
<h3>Concept:</h3>
<ul>
<li> Minimal, close to the metal Kernel/Scheme (operate on words, bytes, vectors) as AST / core language, with Kernel/Vau calculus inspiration oblivating the need for non-reader macros (<a title="Kernel/Vau calculus thesis" href="https://web.wpi.edu/Pubs/ETD/Available/etd-090110-124904/unrestricted/jshutt.pdf">Kernel/Vau calculus thesis</a>)
<li> Full Context-free (and eventually, context sensitive) reader macros using FUN-GLL (<a title="fun-gll paper" href="https://www.sciencedirect.com/science/article/pii/S2590118420300058">FUN-GLL paper</a>) to extend language's syntax dynamically
<li> Implement Type Systems as Macros (but using Vaus instead of macros) (<a title="type systems as macros paper 1" href="http://www.ccs.neu.edu/home/stchang/pubs/ckg-popl2017.pdf">paper, up to System Fω</a>) (<a title="type systems as macros paper 2" href="https://www.ccs.neu.edu/home/stchang/pubs/cbtb-popl2020.pdf">second paper, up to dependent types</a>)
<li> Use above "type systems as vaus" to create richer language and embed entire other programming languages (syntax, semantics, and type system) for flawless interop/FFI (C, Go, Lua, JS, etc)
<li> File is interpreted, and then if "main" exists it is compiled, spidering backwards to referenced functions and data (Allows interpreted code to do metaprogramming, dependency resolution, generate code, etc, which is then compiled)
<li> Regionalized Value State Dependence Graph as backend-IR, enabling simpler implementations of powerful optimizations (<a title="RSVDG paper" href="https://arxiv.org/pdf/1912.05036.pdf">RSVDG paper</a>) so that embedded languages have good performance when compiled with little code
</ul>
<a name="about"/>
<h3> About:</h3>
<p> Currently, I am bootstrapping this new core Lisp out of my prior compiler for my programming language, Kraken. I have implemented the first version of the FUN-GLL algorithm and have working vaus and context-free reader macros.
<p> The general flow is that the input files will be executed with the core Lisp interpreter, and if there is a "main" symbol defined the compiler emits C code for that function & all other functions & data that it references. In this way the language supports very powerful meta-programming at compile time, including adding syntax to the language, arbitrary computation, and importing other files, and then compiles into a static executable.
<p> Below are a few examples of using the vau / live grammar modification / context-free reader macros to implement basic methods as well as embed the BF language into the core Lisp. The core Lisp implementation has been compiled to WebAssembly and should be able to run in your browser. Feel free to make edits and play around below.
<br>
<h4>Code:</h4>
<textarea id="code" cols=130 rows=30>
; Of course
Note that the current implementation is inefficient, and sometimes has problems running in phone web browsers.
<a name="hello_example"/>
<h4>Runnable Example Code:</h4>
<button onclick="executeKraken(hello_editor.getValue(), 'hello_output')"><b>Run</b></button> <br>
<div id="hello_editor">; Of course
(println "Hello World")
; Just print 3
(println "Math works:" (+ 1 2))
</div>
<h4>Output:</h4>
<textarea id="hello_output">Output will appear here</textarea>
<a name="vau_core"/>
<h4>Vau/Kernel as simple core:</h4>
By constructing our core language on a very simple Vau/Kernel base, we can keep the base truely tiny, and build up normal Lisp functions and programming language features in the language itself. This should help implement other programming languages concisely, and will hopefully make optimization easier and more broadly applicable.
<br>
Below is the current prelude that adds quoting, quasiquoting, syntax for vectors and quoting/quasiquoting, do, if, let, and even lambda itself!
<br>
<button onclick="executeKraken(prelude_editor.getValue(), 'prelude_output')"><b>Run</b></button> <br>
<div id="prelude_editor">
(set! current-env (vau de () de))
(set! quote (vau _ (x) x))
; Use the power of GLL reader macros to implement
; BF support
(add_grammar_rule (quote form) (quote ( "'" optional_WS form )) (vau de (_ _ f) (vector quote (eval f de))))
(add_grammar_rule 'form '( "\\[" optional_WS space_forms optional_WS "\\]" ) (vau de (_ _ fs _ _) (concat (vector vector) (eval fs de))))
; Utility until we get stdlib & datastructures figured out
(def! with_update (fn* [arr idx val]
(if (= idx 0)
(cons val (rest arr))
(cons (first arr) (with_update (rest arr) (- idx 1) val)))))
(set! vapply (vau de (f p ede) (eval (concat [(eval f de)] (eval p de)) (eval ede de))))
(set! let1 (vau de (s v b) (eval [[vau '_ [s] b] (eval v de)] de)))
(set! let (vau de (vs b) (cond (= (len vs) 0) (eval b de) true (vapply let1 [(idx vs 0) (idx vs 1) [let (slice vs 2 -1) b]] de))))
;(set! lambda (vau se (p b) (let1 f (eval [vau '_ p b] se) (vau de (& op) (vapply f (map (vau dde (ip) (eval (eval ip dde) de)) op) se)))))
(set! lambda (vau se (p b) (wrap (eval [vau '_ p b] se))))
(set! fun (vau se (n p b) (eval [set! n [lambda p b]] se)))
(fun lapply (f p) (eval (concat [(unwrap f)] p) (current-env)))
(fun do (& params) (cond
(= 0 (len params)) nil
true (idx params (- (len params) 1))))
(set! if (vau de (con than & else) (cond
(eval con de) (eval than de)
(> (len else) 0) (eval (idx else 0) de)
true nil)))
(fun print_through (x) (let (_ (println x)) x))
(fun is_pair? (x) (and (vector? x) (> (len x) 0)))
(set! quasiquote (vau de (x)
(cond (is_pair? x)
(cond (and (symbol? (idx x 0)) (= (get-text (idx x 0)) "unquote"))
(eval (idx x 1) de)
true
(cond (and (is_pair? (idx x 0)) (symbol? (idx (idx x 0) 0)) (= (get-text (idx (idx x 0) 0)) "splice-unquote"))
(concat (eval (idx (idx x 0) 1) de) (vapply quasiquote [(slice x 1 -1)] de))
true
(concat [(vapply quasiquote [(idx x 0)] de)] (vapply quasiquote [(slice x 1 -1)] de))))
true x)))
(add_grammar_rule 'form '("`" optional_WS form) (lambda (_ _ f) ['quasiquote f]))
(add_grammar_rule 'form '("~" optional_WS form) (lambda (_ _ f) ['unquote f]))
(add_grammar_rule 'form '("," optional_WS form) (lambda (_ _ f) ['splice-unquote f]))
(println "now with both vector and quasiquote syntax, check out " `(1 2 3 ~(+ 7 8) ,[ 5 6 7]))
</div>
<h4>Output:</h4>
<textarea id="prelude_output">Output will appear here</textarea>
<a name="method_example"/>
<h4>Method Example:</h4>
Let's use our meta system (attaching objects to other objects) to implement basic objects/methods.
We will attach a vector of alternating symbols / functions (to make this example simple, since maps aren't built in) to our data as the meta, then look up methods on it when we perform a call. The add_grammar_rule function modifies the grammar/parser currently being used to parse the file and operates as a super-powerful reader macro. We use it in this code to add a rule that transforms <pre><code>a.b(c, d)</code></pre> into <pre><code>(method-call a 'b c d)</code></pre> where method-call is the function that looks up the symbol 'b on the meta object attached to a and calls it with the rest of the parameters.
<br>
<button onclick="executeKraken(method_editor.getValue(), 'method_output')"><b>Run</b></button>
<br>
<div id="method_editor">
; Load prelude so we get fun, lambda, if, quoting, etc
(load-file "./k_prime_stdlib/prelude.kp")
; First quick lookup function, since maps are not built in
(fun get-value-helper (dict key i) (if (>= i (len dict))
nil
(if (= key (idx dict i))
(idx dict (+ i 1))
(get-value-helper dict key (+ i 2)))))
(fun get-value (dict key) (get-value-helper dict key 0))
; Our actual method call function
(fun method-call (object method & arguments) (let (method_fn (get-value (meta object) method))
(if (= method_fn nil)
(println "no method " method)
(lapply method_fn (concat [object] arguments)))))
; Some nice syntactic sugar for method calls
(add_grammar_rule 'form ['form "\\." 'atom 'optional_WS "\\(" 'optional_WS 'space_forms 'optional_WS "\\)"]
(lambda (o _ m _ _ _ p _ _) `(method-call ~o '~m ,p)))
; Ok, let's create our object by hand for this example
(set! actual_obj (with-meta [0] [
'inc (lambda (o) (set-idx! o 0 (+ (idx o 0) 1)))
'dec (lambda (o) (set-idx! o 0 (- (idx o 0) 1)))
'set (lambda (o n) (set-idx! o 0 n))
'get (lambda (o) (idx o 0))
]))
(do
; Use our new sugar
actual_obj.set(1337)
actual_obj.inc()
(println "get: " actual_obj.get())
actual_obj.dec()
(println "get: " actual_obj.get())
; Use methods directly
(method-call actual_obj 'set 654)
(method-call actual_obj 'inc)
(println "get: " (method-call actual_obj 'get))
(method-call actual_obj 'dec)
(method-call actual_obj 'dec)
(println "get: " (method-call actual_obj 'get))
nil)
</div>
<h4>Output: </h4>
<textarea id="method_output">Output will appear here</textarea>
<a name="bf_example"/>
<h4>More Complicated Example: BF as an embedded language</h4>
<button onclick="executeKraken(bf_editor.getValue(), 'bf_output')"><b>Run</b></button> <br>
<div id="bf_editor">
(load-file "./k_prime_stdlib/prelude.kp")
; We don't have atoms built in, mutable vectors
; are our base building block. In order to make the
; following BF implementation nice, let's add atoms!
; They will be implmented as length 1 vectors with nice syntax for deref
(fun make-atom (x) [x])
(fun set-atom! (x y) (set-idx! x 0 y))
(fun get-atom (x) (idx x 0))
(add_grammar_rule 'form ["@" 'form] (lambda (_ x) `(get-atom ~x)))
; Now begin by defining our BF syntax & semantics
; Define our tokens as BF atoms
(add_grammer_rule 'bfs_atom ["<"] (fn* [xs] (list 'left)))
(add_grammer_rule 'bfs_atom [">"] (fn* [xs] (list 'right)))
(add_grammer_rule 'bfs_atom ["\\+"] (fn* [xs] (list 'plus)))
(add_grammer_rule 'bfs_atom ["-"] (fn* [xs] (list 'minus)))
(add_grammer_rule 'bfs_atom [","] (fn* [xs] (list 'in)))
(add_grammer_rule 'bfs_atom ["."] (fn* [xs] (list 'out)))
(add_grammar_rule 'bfs_atom ["<"] (lambda (_) '(set-atom! cursor (- @cursor 1))))
(add_grammar_rule 'bfs_atom [">"] (lambda (_) '(set-atom! cursor (+ @cursor 1))))
(add_grammar_rule 'bfs_atom ["\\+"] (lambda (_) '(set-idx! tape @cursor (+ (idx tape @cursor) 1))))
(add_grammar_rule 'bfs_atom ["-"] (lambda (_) '(set-idx! tape @cursor (- (idx tape @cursor) 1))))
(add_grammar_rule 'bfs_atom [","] (lambda (_) '(let (value (idx input @inptr))
(do (set-atom! inptr (+ 1 @inptr))
(set-idx! tape @cursor value)))))
(add_grammar_rule 'bfs_atom ["."] (lambda (_) '(set-atom! output (concat [(idx tape @cursor)] @output))))
; Define strings of BF atoms
(add_grammer_rule 'non_empty_bfs_list ['bfs_atom] (fn* [xs] (list (nth xs 0))))
(add_grammer_rule 'non_empty_bfs_list ['bfs_atom 'optional_WS 'non_empty_bfs_list] (fn* [xs] (cons (nth xs 0) (nth xs 2))))
(add_grammer_rule 'bfs_list [] (fn* [xs] xs))
(add_grammer_rule 'bfs_list ['non_empty_bfs_list] (fn* [xs] (nth xs 0)))
(add_grammar_rule 'bfs ['bfs_atom *] (lambda (x) x))
; Add loop as an atom
(add_grammer_rule 'bfs_atom ["\\[" 'bfs_list "]"] (fn* [xs]
`(let* (f (fn* []
(if (= 0 (nth (deref arr) (deref ptr)))
; (note that closure cannot yet close over itself by value, so we pass it in)
(add_grammar_rule 'bfs_atom ["\\[" 'bfs "]"] (lambda (_ x _)
`(let (f (lambda (f)
(if (= 0 (idx tape @cursor))
nil
(do ,(nth xs 1) (f)))))
(f))))
; Top level BFS rule
(add_grammer_rule 'bfs ['bfs_list] (fn* [xs] (nth xs 0)))
(do ,x (f f)))))
(f f))))
; For now, stick BFS rule inside an unambigious BFS block
; and add compilation/implementation
; Note that this compilation into the underlying Lisp
; happens at macro evaluation time. If this code were
; to be compiled to C, it would be compiled all the way
; to C code with no trace of the original BF code.
(add_grammer_rule 'form ["bf" 'optional_WS "{" 'optional_WS 'bfs 'optional_WS "}"]
(fn* [xs]
`(fn* [input]
(let* (
arr (atom (vector 0 0 0 0 0))
output (atom [])
ptr (atom 0)
inptr (atom 0)
left (fn* [] (swap! ptr (fn* [old] (- old 1))))
right (fn* [] (swap! ptr (fn* [old] (+ old 1))))
plus (fn* [] (swap! arr (fn* [old] (with_update old (deref ptr) (+ (nth (deref arr) (deref ptr)) 1)))))
minus (fn* [] (swap! arr (fn* [old] (with_update old (deref ptr) (- (nth (deref arr) (deref ptr)) 1)))))
in (fn* [] (let* ( h (nth input (deref inptr))
_ (swap! inptr (fn* [old] (+ old 1))))
(swap! arr (fn* [old] (with_update old (deref ptr) h)))))
out (fn* [] (swap! output (fn* [old] (cons (nth (deref arr) (deref ptr)) old)))))
(do ,(nth xs 4) (deref output))))))
; Also add setup code
(add_grammar_rule 'form ["bf" 'optional_WS "{" 'optional_WS 'bfs 'optional_WS "}"]
(lambda (_ _ _ _ x _ _)
`(lambda (input)
(let (
tape (vector 0 0 0 0 0)
cursor (make-atom 0)
inptr (make-atom 0)
output (make-atom (vector))
)
(do (println "beginning bfs") ,x (idx output 0))))))
; Let's try it out! This BF program prints the input 3 times
(println (bf { ,>+++[<.>-] } [1337]))
</textarea>
<button onclick="executeKraken()">Run</button> <br>
; we can also have it compile into our main program
(fun main () (do (println "BF: " (bf { ,>+++[<.>-] } [1337])) 0))
</div>
<h4>Output: </h4>
<textarea id="output" cols=130 rows=10>Output will appear here</textarea>
<textarea id="bf_output">Output will appear here</textarea>
<a name="benchmarks"/>
<!--<h3>Performance Benchmarks</h3>-->
<!--<p>Performance is quite poor (for the interpreter mainly, the C compiler seems to be smart enough to make even the very inefficient generated C code fast), as almost no work has gone into it as of yet.-->
<!--We are currently focusing on the FUN-GLL macros and creating a more fully-featured language on top of the core Lisp using them. We will focus more on performance with the implementation of the functional persistent data structures and the self-hosting rewrite, and performance will be the main focus of the RVSDG IR part of the project.-->
<!--<p> Even so, it is worth keeping a rough estimate of performance in mind. For this, we have compiled a very basic benchmark below, with more benchmark programs (sorting, etc) to be included as the language gets developed:-->
<!--<br>-->
<!--<table>-->
<!--<thead>-->
<!--<tr>-->
<!--<th></th>-->
<!--<th>Core Lisp Interpreter</th>-->
<!--<th>Core Lisp Compiled to C</th>-->
<!--<th>Hand-written C</th>-->
<!--</tr>-->
<!--</thead>-->
<!--<tbody>-->
<!--<tr>-->
<!--<td><b>Fibonacci(27)</b></td>-->
<!--<td>51.505s</td>-->
<!--<td>0.007s</td>-->
<!--<td>0.002s</td>-->
<!--</tr>-->
<!--</tbody>-->
<!--</table>-->
<!--<br>-->
<!--Here is the core Lisp code run / compiled by the above test, which you can run in your web browser. The hand-written C code is an exact translation of this into idiomatic C.-->
<!--<br><i>Note: N is lowered in the web demo so WebAssembly doesn't run out of memory.</i>-->
<!--<a name="fib_example"/>-->
<!--<h4>Fibonacci:</h4>-->
<!--<button onclick="executeKraken(fib_editor.getValue(), 'fib_output')"><b>Run</b></button> <br>-->
<!--<div id="fib_editor">(def! fib (fn* (n) (cond (= 0 n) 0-->
<!--(= 1 n) 1-->
<!--true (+ (fib (- n 1)) (fib (- n 2))))))-->
<!--(let* (n 16)-->
<!--(println "Fib(" n "): " (fib n)))-->
<!--</div>-->
<!--<h4>Output:</h4>-->
<!--<textarea id="fib_output">Output will appear here</textarea>-->
<a name="next_steps"/>
<h3>Next Steps</h3>
<ul>
<li> Implement persistent functional data structures
<ul>
<li> Hash Array-Mapped Trie (HAMT) / Relaxed Radix Balance Tree (RRB-Tree)
<li> Hash Map based on the above
<li> Hash Set based on the above
</ul>
<li> Prototype Type Systems as Macros, may require macro system rewrite/upgrade
<li> Sketch out Kraken language on top of core Lisp, includes basic Hindley-Milner type system implemented with Macros and above data structures
<li> Re-self-host using functional approach in above Kraken language
<li> Use Type System Macros to implement automatic transient creation on HAMT/RBB-Tree as an optimization
<li> Implement RVSDG IR and develop best bang-for-buck optimizations using it
</ul>
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/ace/1.4.11/ace.min.js"></script>
<script>
ace.config.set('basePath', 'https://cdnjs.cloudflare.com/ajax/libs/ace/1.4.11/')
var hello_editor = ace.edit("hello_editor")
var prelude_editor = ace.edit("prelude_editor")
var method_editor = ace.edit("method_editor")
var bf_editor = ace.edit("bf_editor")
//var fib_editor = ace.edit("fib_editor")
//for (let editor of [hello_editor, method_editor, bf_editor, fib_editor]) {
for (let editor of [hello_editor, prelude_editor, method_editor, bf_editor]) {
editor.session.setMode("ace/mode/clojure")
editor.setOption("displayIndentGuides", false)
editor.setShowPrintMargin(false)
}
var output_name = ""
var Module = {
noInitialRun: true,
onRuntimeInitialized: () => {
},
print: txt => {
document.getElementById("output").value += txt + "\n";
document.getElementById(output_name).value += txt + "\n";
},
printErr: txt => {
document.getElementById("output").value += "STDERR:[" + txt + "]\n";
document.getElementById(output_name).value += "STDERR:[" + txt + "]\n";
}
};
function executeKraken() {
document.getElementById("output").value = "";
let code = document.getElementById("code").value;
console.log("gonna execute", code);
function executeKraken(code, new_output_name) {
output_name = new_output_name
document.getElementById(new_output_name).value = "running...\n";
Module.callMain(["-C", code]);
}
</script>