Luce packages
LuceLuce BaseLuciaOS

dymokomi/luce-std

Luce's standard library: files, paths, processes and the environment, networking, time and dates, random numbers, math, sorting, UTF-8, Unicode and crash reports; everything a program imports beyond the language runtime.

A library written in luce-base. Latest release 0.7.5.

Install#

luc add dymokomi/luce-std

Documentation#

luce-std

Luce's standard library, as a package of its own. The compiler ships only the language runtime (memory, ownership, interop, threads, strings, I/O streams); everything else a program imports comes from here, versioned apart from the toolchain. It is written in Luce Base and serves Luce and Base programs alike.

Modules

importwhat it holds
from luce_std import filesFiles and directories: whole-file text and bytes, File objects, listings, walks, metadata, copying, renaming, temporary directories, read-only memory-mapped files (map)
from luce_std import pathsLexical filesystem paths: join, normalize, split
from luce_std import processRunning programs (run, background Command, which wakes a waker as it writes and ends), environment variables, the working directory
from luce_std import wakeWaker: how background work tells a waiting loop it has news, from any thread, without either knowing the other; luce-ui's Notifier.waker() makes one that wakes the UI loop, and wake.waker(call, context, finish) makes one from Base
from luce_std import netTCP connections and listeners by host name, name lookup, UDP; HTTP and WebSocket wire formats for Base; for event loops, connects that do not wait (Connection.start_connect, finish_connect) and name lookups on a thread of their own (Lookup)
from luce_std import clockThe monotonic clock, durations, sleeping, calendar dates and ISO 8601
from luce_std import rampHoudini's ramp parameter (VEX chramp): a value or a color along 0..1 from points, each with a position, a value or color and an interpolation (constant, linear, smooth Catmull-Rom, monotone cubic, bspline), kept as a flat f64 list; lookup(data, t), lookup_color and lookup_into read it without allocating, check(data) validates it. See Ramps
from luce_std import randomA seeded pseudo-random generator
from luce_std import mathThe mathematical functions
from luce_std import math32Single-precision mathematical functions
from luce_std import sortSorting Base arrays in place without allocating: sort(values), by(values, less), using(values, context, less) (introsort, O(n log n), not stable); stable and stable_using keep equal values in order; order and order_by write the indices that would sort an array (argsort). Luce lists have sort and sorted of their own
from luce_std import parallelA parallel for over independent items for Base: for_each(count, context, work) runs work(context, index, worker) on the caller and a process-wide pool of one thread per processor, started once and kept, so a call costs microseconds; worker is below workers(count) and unique to one thread for the call, for per-worker scratch; the first error is returned; calls nest and may come from many threads; for_each_range hands out chunks of indices; start runs items in the background and returns a Job to poll with completed()/progress() and collect with wait(); idle threads sleep after a few microseconds; warm starts the pool early
from luce_std import unicodeUnicode 17.0.0 casing and normalization
from luce_std import utf8Strict UTF-8 scalar encoding and decoding
from luce_std import collectionsGrowable storage for Base: List[T], a byte Buffer, and a TextPool of copies that never move
from luce_std import crashCrash reports in ~/.luce/crashes, named after the program's package; hooks that save work after a trap (on_crash, recovery_directory, note_recovery); starting the program again to show its report (relaunch_on_crash, report_to_show), which luce-ui's crash window uses. Nothing is sent anywhere

Using it

Add the dependency with luc add dymokomi/luce-std, or in package.prisma:

def dependency "luce-std" {
    str owner = "dymokomi"
    str version = "^0.4.0"
}

From Luce, the modules read like Python's:

from luce_std import files, clock

pub func main(arguments: list[str]) -> int!:
    let start = clock.now()
    with files.open("notes.txt") as notes:
        while let line = notes.read_line():
            print(line)
    print(f"read in {start.elapsed().seconds()} s at {clock.DateTime.now().text()}")
    return 0

The standard library chapter of the Luce guide describes what Luce programs call. A Luce program sees the functions, structs and objects whose signatures cross the Base boundary; the rest is for Base.

From Base

Functions that answer fresh storage, such as files.read, files.list, paths.join, process.run or unicode.to_upper, answer an interop.Owned[...] result. Luce copies the value and releases it. A Base caller reads .value and releases the result when done:

let names = try files.list("notes")
defer names.release()
for name in names.value:
    print(name)

The storage comes from the allocator current at the call; texts are NUL-terminated, so they may be passed on as c.str. Objects Luce holds (files.File, files.TemporaryDirectory, net.Connection, net.Listener, process.Command) remain ordinary values in Base.

Mapped files

files.map(path) answers a whole file's bytes as a read-only mapping, much as Python's mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ) does, but without a file to keep open: the result is an interop.Owned[const u8[]] whose release() unmaps it.

let font = try files.map("/System/Library/Fonts/Helvetica.ttc")
defer font.release()
let tag = font.value[..<4]          # pages are read from the file as they are touched

The mapping is shared, so its pages belong to the file: they cost nothing until read, the system may drop them under memory pressure and read them again, and every process mapping the same file shares them. That suits large files read in parts, such as fonts. An empty file answers an empty view. The view stays valid after the file is closed, renamed or opened elsewhere, but not after it is truncated (reading past the new end faults, as it does in Python); write a replacement through write_atomic instead of changing a mapped file in place. A Luce program calling files.map receives a copy as bytes, so from Luce it reads like files.read.

Growable storage

collections is for Base only; Luce has its own list and bytes. A zero value of each type is empty and ready. The first growth takes the current allocator and keeps it, as strings.Builder does; destroy gives the storage back, after which the value may be used again.

from luce_std import collections

var rows: collections.List[Row]
defer rows.destroy()
try rows.append(Row(id = 1))
try rows.insert(0, Row(id = 0))
rows.remove(1)                      # keeps the order of the rest; remove(index, count) for a run
for row in rows.view(): ...         # const Row[]; rows.items() is a Row[] to change or sort

var line: collections.Buffer        # an io.Writer
defer line.destroy()
try line.put_text("total ")
try line.put_number(42)
try strings.write_f64(&line, 2.5, ".2f")
print(line.text())                  # the bytes as far as they are valid UTF-8
let path = try line.terminated()    # a c.str, valid until the next change

var pool: collections.TextPool      # copies stay where they are until pool.destroy()
let name = try pool.keep_text(column_bytes)
  • List[T]: length, capacity, reserve, append, insert, remove(index, count = 1), truncate, at, set, view, items, clear, destroy. The capacity at least doubles when it grows, so view and items are valid until the next append, insert or reserve. An index out of range traps, as indexing a span does.
  • Buffer: put, put_text, put_byte, put_number, write (io.Writer), remove, truncate, view, text, terminated, length, reserve, clear, destroy.
  • TextPool: keep(bytes), keep_text(bytes) (cut where the bytes stop being valid UTF-8), destroy. Copies go into 64 KiB chunks; a larger one gets a chunk of its own.

Ramps

A ramp is a list of numbers a host keeps as it is (a node's values, a document's settings) and ramp.lookup evaluates; luce-ui's ParameterPanel edits one and draws its preview through the same function, so an editor and a cook never disagree. Version 1:

indexholds
01, the version
1channels: 1 a float ramp, 3 an RGB color ramp (2 and 4 are read as well; the fourth is alpha)
2the number of points, at least 1
then, per pointchannels + 2 numbers: the position (0..1), the channels, the interpolation code

A float ramp of n points is 3 + 3n numbers, an RGB ramp 3 + 5n; `ramp.size(channels, count)` says so. Points are sorted by position; two at one position make a step. The code of point i decides the segment from it to point i + 1 (the last point's is kept, unused):

codeInterpolationbetween points i and i + 1, u along the segment
0constantpoint i's value
1linearthe straight line
2smoothCatmull-Rom: a cubic Hermite whose slope at a point is the line from the point before to the point after, zero at the first and last point and beside a step (two points ease in and out)
3monotoneFritsch–Carlson's monotone cubic with PCHIP's slopes: zero where the values turn, a weighted harmonic mean of the two secants elsewhere, PCHIP's limited three-point rule at the ends; never goes past its points
4bsplinethe uniform cubic B-spline with the points as control points, passing near the inner points; a control point missing beyond an end or a step is the end reflected through its neighbour, so the curve still starts and ends on the end points

Before the first point and after the last the ramp holds their values; a NaN t gives the first. Each channel is interpolated alone, and every rule reads only the points around the segment, so a lookup is a bisection and a few multiplications.

from luce_std import ramp

try ramp.check(data)                     # once, when the data arrives
let falloff = ramp.lookup(data, distance)        # channel 0
let tint = ramp.lookup_color(data, t)            # ramp.Color: red, green, blue, alpha
var lanes: f64[4]
let channels = ramp.lookup_into(data, t, lanes[0..])

lookup never fails: data check would refuse answers 0 (or black), and an unknown code reads as linear. ramp.copy(data) answers a copy as an owned list, for a host handing the ramp it keeps to Luce, which sees it as a list[float].

Depends on

  • luce-base compiler (the runtime modules)

Platforms

macOS, Windows, Linux.

Native libraries it links, by platform (declared in package.prisma, linked only when the program reaches code that needs them):

  • windows: ws2_32

Tests

luc test runs every module's test blocks, with the module tests under tests/<module>/, and every program under tests/: the file, network, process, math, Unicode and crash checks, the HTTP server driven by curl, the Unicode tables against their generator, and the Luce programs tests/luce_*, each of which must print its expected output and leave no object alive.

The same run passes on macOS, Linux and Windows. The checks whose C reference is POSIX code, with faults injected through dlsym or FIFOs made by mkfifo, skip themselves on Windows and say why; tests/windows_files and tests/windows_network check Windows' side of the file and network contracts instead, and skip elsewhere. On Windows, run it from Git's bash, whose sh, echo and nm some checks call.

License

MIT or Apache-2.0, at your option.

Versions#

0.7.5

Browse source · pack · commit e183ce2ac377

SHA-256 c52cfc062bf2f83cd8f3261c3c651ef50df63ea34eebac9cc6522ecc6a708b97

luce-std 0.7.5: batch release

0.7.4

Browse source · pack · commit b8c9791e7167

SHA-256 d01583f10f76191b6640a8a9b87a228b7199c0582e6b8b217369c16ef8812139

luce-std 0.7.4: batch release

0.7.3

Browse source · pack · commit 172d1f20b3b7

SHA-256 26faad54c51040403d8c755132f47bbd45a1e47ea6e3c28908da305e8676d7ff

luce-std 0.7.3: batch release

0.7.2

Browse source · pack · commit ccabcf7c352b

SHA-256 b70189856b54b1fd5638eb43cfeac732d7368a5c5250db0fa91dcc6f4f4d0117

luce-std 0.7.2: batch release

0.7.1

Browse source · pack · commit f6dded9d8518

SHA-256 8a3c2d11b9226ca388edbd910daabc0480b2c06821e3fe106c18ffeaee3cf950

luce-std 0.7.1: batch release

0.7.0

Browse source · pack · commit 3921e14e30d0

SHA-256 e569f4808ee13853d6da45c1567740cb49208c0f709c2b263a6a203856b82a72

collections (List, Buffer, TextPool), process.call, crash hooks that return a handle, crash reports and relaunch