```
![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
CL-USER> (defun (setf bob) (object object2 object3)
(format t "Setting Bob ~a , ~a , ~a~&" object object2 object3))
(SETF BOB)
CL-USER> (setf (bob 'bob1 'bob2) 'bob3)
Setting Bob BOB3 , BOB1 , BOB2
NIL
```
Simple worked example I couldn't find on the general internet.
Common Lisp is the dynamic language I keep returning to. Garbage collection; a reasonably large amount of libraries; not-absurd policies around exceptions (hi Perl), not-absurd design around power (hi Python), not-absurd design around formatting & naming (hi Go).
Scala is, plausibly, at this point, the only other major language I rather like hacking in, and it's JVM tied (sigh), slow to compile (sigh), and frankly rather verbose.
And, too, the advanced-type languages often turn into a game of "lets defeat the type system so it works right". That's not to knock Rust or its less popular peers - Nim, Pony - but it's just, after a while, a drag.
The basic problem with Common Lisp is the lack of effective static assertions - this isn't so bad when you're in the moment, but when the codebase gets larger and you wind up modifying things "far away" - or you come back to the project months later - things don't work quiiite as well.
I will have to keep grinding on this little problem, but I think I will need to cajole up a basic solution:
A dynamic-binding macro system which acts as a type asserter that acts at eval-time time to check that the current args seem to be valid.
More sophisticatedly, this would have to be a treewalker that finds, expands, and checks function invocations.
Essentially, if I write
(DEFUN-TYPED foo ((x int) (y funny-object))
(compute-funny y (+ x 1)))
then the -TYPED system should validate that yes, x is doing int-y things and yes, compute-funny takes a funny object in the second position - and this should happen at eval-time.
If, however, some dynamic binding *no-typed-assert is turned off, then at eval time this should not occur.
The previous effort I know about for Common Lisp gradual typing is Coalton, which is a sort of library-in-CL, which turned into its own CL-hosted language. Meh. Then there's https://github.com/mmontone/cl-gradual, which seems to be sort of off and on.
Anyway. I might use one of those, or cobble together some 'orrible 'ack myself.
(defmacro defstruct* (name &rest slots)
"Provides a DEFSTRUCT interface to a DEFCLASS"
(flet
((accessor-symbol
(sym)
(intern (concatenate 'string (string name) "-" (string sym)))))
(let* ((actual-slots
(etypecase
(car slots)
(string
(cdr slots))
(symbol
slots)
(cons
slots)))
(slot-list
(loop for s in actual-slots
collect
(etypecase s
(symbol
`(,s
:initarg ,s
:accessor ,(accessor-symbol s)))
(cons
`(,(car s)
:initarg ,(car s)
:accessor ,(accessor-symbol (car s))
:initform ,(cadr s))))
)))
`(defclass ,name ()
,slot-list
(:documentation ,(etypecase (car slots)
(string
(car slots))
(symbol
(format nil "The struct defining ~a, containing slots ~{~a~^, ~}" name slots))))
))))
A few years ago, I went on a "best language for home development" hunt. I sorted through pretty much all I knew and came out the other side with "Ocaml".
Some 1100 lines of Ocaml later, I terminated the project.
Couple reasons.
I'd like to believe it's better now. I invested a lot of time into that code. The concision was fantastic. That said, the friction was very high.
I went and did things with Scala after that...
nothing in consumer electronics is so frustrating right now to me as the state of music playing.
mp3 players are a defunct line of hardware now reserved for crappy crappy ipod knockoffs and some running devices.
None of the above needed to happen except, maybe, the hardware (due to profit/loss aspects).
Hosting my own music streaming service is doable, I guess, but then I lose access to the streaming libraries for which I (happily) pay for. And albums are expensive.
What's extra fantastic is that there doesn't seem to be a standard "wifi speaker" protocol, so I can't reliably send music from my android apps to Alexa or Google Home.
This is all bound in with what the choice was for the software to be written and the agency the individual programmers had at these different companies.
So it seems that the right way to do this is to buy a phone (because companies prefer to stream quality through apps, not browsers), hook it to wifi, and then have it wired into a speaker system. The phone would also have all my mp3s hosted on it.
Fantastically overcomplicated....
package main
import "fmt"
func MapMerge1[K comparable, V any](a map[K]V, b map[K]V) map[K]V {
m := map[K]V{}
for k, v := range a {
m[k] = v
}
for k, v := range b {
m[k] = v
}
return m
}
func main() {
disabledThing := map[string]interface{}{
"enabled": false,
}
other := map[string]interface{}{
"options": map[string]interface{}{
"thungus": false,
},
}
merged := MapMerge1(disabledThing, other)
fmt.Println(merged)
}
Need to write more professionally. One part, relearning how to write more, one part, working out ideas.
Topically, looking to remark on distributed computing, cloud design, algorithmic shifts, Linux, programming, programming languages, techie productivity, etc.
CLINK> (time (ref *foo* :row
(multi-column-query *foo*
(list
`(1 ,#'(lambda (s) (find #\1 s)))
`(0 ,(lambda (x) (> x 300)))
`(3 ,(lambda (x) (> x 900)))))))
Evaluation took:
4.015 seconds of real time
4.149000 seconds of total run time (4.149000 user, 0.000000 system)
103.34% CPU
8,676,250,063 processor cycles
1,252,768 bytes consed
CLINK> (time (ref *foo* :row
(multi-column-query *foo*
(list
`(1 ,#'(lambda (s) (find #\1 s)))
`(0 ,(lambda (x) (> x 300)))
`(3 ,(lambda (x) (> x 900)))))))
Evaluation took:
0.766 seconds of real time
0.879000 seconds of total run time (0.879000 user, 0.000000 system)
114.75% CPU
1,655,372,433 processor cycles
1,074,592 bytes consed
Common Lisp tooling typically isn't oriented around the continuous integration/build systems that we're accustomed to in 2014.
I don't terribly like that, particularly since it's one of the few tools that have been demonstrated to work, and work well, in software engineering.
Anyway, I updated my TOML parser to work with (Travis CI)[https://github.com/pnathan/pp-toml] (but the principles are the same regardless of box host). Here's the code, followed by a write-up.
As a YAML:
before_script:
- curl -O -L http://prdownloads.sourceforge.net/sbcl/sbcl-1.2.6-x86-64-linux-binary.tar.bz2
- tar xjf sbcl-1.2.6-x86-64-linux-binary.tar.bz2
- pushd sbcl-1.2.6-x86-64-linux/ && sudo bash install.sh && popd
- curl -O -L http://beta.quicklisp.org/quicklisp.lisp
- sbcl --load quicklisp.lisp --eval '(quicklisp-quickstart:install)' --eval '(quit)'
script:
- sbcl --script run-sbcl-tests.lisp
Where the run-sbcl-tests.lisp looks as follows:
#!/usr/local/bin/sbcl
(require "sb-posix")
#-quicklisp
(let ((quicklisp-init (merge-pathnames "quicklisp/setup.lisp"
(user-homedir-pathname))))
(when (probe-file quicklisp-init)
(load quicklisp-init)))
(defparameter *pwd* (concatenate 'string (sb-posix:getcwd) "/"))
(push *pwd* asdf:*central-registry*)
(ql:quickload :pp-toml-tests)
(let ((result-status (pp-toml-tests:run-tests)))
(sb-posix:exit (if result-status 0 1) ))
Under the hood and in Lisp, pp-toml-tests:run-tests drives some fiveAM code to run the extant tests. FiveAM is, as far as I can tell, designed for on-the-fly interactive testing, as most Lisp tooling is. It was entirely too surprising in an integration with continuous integration. I've written my own bad hack for a unit testing framework, the "checker" system, designed for running in CI, but it's kind of, well, a bad hack. I should look elsewhere.
A few key highlights of what's going on in this code:
Hope this helps your Common Lisp integration testing!
I'm on the record of denying the idea of software engineering. We don't work with the physical the way physical engineers do. We don't have the science<->practice chain the way the regular engineers do. Worse, I think the IEEE SWEBOK is horse poop.
But there's still a practice and rigor to software development. I've been in this world for a few years now, in a few different teams. I've drunk from the wisdom of others. I think I can say something not entirely worthless about the matter of writing good software.
The first thing is the goal.
These questions describe the scale of the effort, the kind of people asked to work on the effort, the tools used during the effort, and the process best implemented during the execution. This is a very simple set of questions designed to understand what the problem is and where you want to go. Put simply, these are the strategic points required to put substrategies and tactics into play.
The common ends in business are "makes us money" or "saves us money". The time it must be done by is usually impossible and best described as "yesterday". The actual product is often mutable and is the usual concern of software creators.
The second thing to consider is the famous cost-quality-speed triangle (pick two). Your company mandates the quality. While you the creator control it, your company may find you lacking if you mishandle it. This is partially true of speed, however.Very few products make or break a company by release date, and software projects are notorious for being late. Particularly when estimates from the line people are disregarded. Cost is, again, something you don't really control for software projects: it's labor + overhead for your workspace + support staff.
As the creator of software, you can materially affect speed and quality. Let us presume that you are going to work your usual 35-45 hour work week and have a reasonable competence at the particular technology that you're dealing with - same as everyone else. How do you manage - from your level as an individual contributor and perhaps mentor to others - keeping things working in alignment with your company? That is the next blog post.
The third thing to consider is politics, or, more euphemistically, "Social questions". An old rule of thumb is that software architecture reflects the organization it was developed in. Another rule of thumb is that people are promoted to their level of incompetence. Yet another is that most organizations stratify vertically and attempt to build internal empires. Let us not assume that our fine institution is not subject to these pressures. It probably has already succumbed in part.
Several implications result from this.
None of these things have to do with software engineering; they hold pretty true pretty well across any producing endeavor. But they lay the context and foundation for the next blog post.
I've taken an at-home coding vacation this week and last week. I've been doing reading on model railroads and sailing, as well as a smattering of other books.
I flipped through the list of the papers from POPL2014; it's kind of frustrating to me- they all appear to be focused on type systems. I'm not sure why this is such a thing in programming language design (perhaps it is enjoyable to work on the math!). But I don't think the big problem - software crisis if you will - is in data types. Data types in most of the developer's world sits in the grungy province of the C(C, C++, Java, C#, ObjC) family or in the more fluid province of the Perl family (Perl, Ruby, Python, Groovy, etc). Neither of these languages has the type system problem solved even at the level of ML languages (which are, AFAICT, sort of old hat today). So from a "cool insights useful in fifteen years" level, I'm depressed! These results probably won't ever get to an Ordinary Practitioner.
For me, the bridge into the Best Possible World is founded in Software Contracts, most commonly associated with Eiffel and occasionally supported in obscure extensions for other languages. Suppose that I claim that not only will some variable quux is an int, (solvable in C) but in fact it is between 0 and 10 (solvable in Ada, somewhat in SBCL, and perhaps in the Agda & family of dependently typed languages), and not only that, quux will always be passed into frobbing_quux and used to generate a result (similarily quantified). Let me call that "Code Type System". It may be that complete analysis of the code demands a complete type system of the data. Certainly some of this work has already been done in the model checking & static analysis community, as they can already detect certain concurrency bugs and malloc/free errors. Then, of course, we find ourselves building a model of the program in our "Code Type System" before we actually build the program, and if we find we need to alter the system, we have to rebuild the model framework. This is well understood to be the classic failure model of fully verified software development.
Let me instead take this thought experiment by way of an example, using Common Lisp.
(def-checked-fun mondo (quux baz)
"Some docs"
(check (and (calls-into frobbing_quux quux)
(type-of quux integer)
(range quux 0 10))
;; do stuff
(frobbing_quux quux)
;; do stuff
)
The theoretical def-checked-fun macro will examine the code, both as-is and macro-expanded, verifying that it does indeed appear that quux satisfies the required judgement. Of course, we can't answer that quux is of a particular type or that it falls into a certain range at this point: in order to make that judgement, either run-time checks need to be added (to constrain possibilities), or intraprocedural analysis needs to be performed. However, the calls-into check can be either demonstrated or an "unproven" result returned. This is simple - some CAR needs to be frobbing_quux with a bar in the argument list.
Some of this, in some measure is already done in SBCL. I have long thought (at least 8 months now), that a compile-time macro system (def-checked-fun is an example) could provide some very interesting insight, particularly if you begin to share information).
The worst possible result is that the checker returns something to the effect of "No possible errors detected; no data found to make judgement". In short, it's useless. The best possible result is that intraprocedural contracts can be built based on not simply types but expected semantics in a succinct system (only modelling what is useful), then when time comes for running, all this information is quietly removed.
I propose that this is a useful idea, and practical, too - for an Ordinary Practitioner. It's like lightweight unit tests that require almost no lines of code and almost no tweaking.
It's important to understand, of course, that I'm thinking about "correct" results, not "complete" results, quite two different things! Dawson Engler's talk/paper "A Few Billion Lines of Code Later" really strikes at the heart of what makes a system useful for this kind of work in practice and the exigencies of real world work. I won't summarize it here.
What I don't know is if CoQ or other systems (ACL2, etc) already implement this sort of fluid procedural static type checking. Possibly they do.
