Pen programming language

GitHub Action License Twitter

Pen is the programming language for scalable software development, focused on software maintainability and portability.

import Os'Context { Context }
import Os'File
import Os'Process

sayHello = \(ctx Context) none | error {
  File'Write(ctx, File'StdOut(), "Hello, world!\n")?

  none
}

main = \(ctx context) none {
  e = sayHello(ctx.Os)

  if _ = e as none {
    none
  } else if error {
    Process'Exit(ctx.Os, 1)
  }
}

Install

See Install.

Documentation

Vision

Pen aims to make large-scale software development efficient where a number of people develop software together for a long time. To realize that, it focuses on software maintainability and portability.

  • Maintainability
    • Simplicity: The language is small and easy to learn but also full featured.
    • Testability: Unit tests are always fast and reliable.
    • Modifiability: Developers can change application logic independently from implementation details.
  • Portability
    • Programs written in the language can be ported to different platforms including WebAssembly.

Features

Minimal language

  • Its syntax and type system are small, simple, and easy to learn.
  • Yet, the language supports all the modern features.

Concurrent/parallel computation

  • The language and its runtime enables thread-safe concurrent/parallel computation.
  • For more information, see Concurrency and parallelism.

System packages

  • System packages encapsulate side effects as separate packages.
  • No other packages have side effects unless injected them into explicitly.

Reliable testing

  • Unit tests are always deterministic and fast.
  • No flaky or slow tests bother developers.

Even more...

  • Static typing
  • Immutable values
  • Pure functions by default
  • Errors as values
  • Asynchronous I/O
  • Cross compile
  • Rust/C Foreign Function Interface (FFI)

License

Pen is released under open source licenses. See its LICENSE file for more information.

Guides

This chapter contains how-to guides for software development in the language.

Getting started

Install

See Install.

Creating a package

To create your first package, run the following command.

pen create foo

Then, you should see a foo directory under the current directory. When you go to the foo directory, you should see a main.pen source file and a pen.json file for package configuration.

Building a package

To build the package, run the following command in the foo directory.

pen build

Then, you will see an executable file named app in the directory. Run it to see your first "Hello, world!"

./app

For more information...

Now, you can start editing *.pen files and build your own application!

  • For more code examples, see Examples.
  • To know more about the language's constructs, see Syntax and Types.
  • To know how to use the standard packages, see Standard packages.
  • To know how to add more modules in your package, see Modules.
  • To know how to import other packages, see Packages.

Install

Via Homebrew

On Linux, macOS and WSL, you can install Pen through Homebrew.

  1. Install Homebrew.

  2. Run the following command in your terminal.

    brew install pen-lang/pen/pen

Now, you should be able to run a pen command in your shell.

pen --version

Building from source

You can also build Pen from source on your local machine.

  1. Install the following software using a package manager of your choice (e.g. apt for Ubuntu and Homebrew for macOS.)

  2. Clone the Git repository.

    git clone https://github.com/pen-lang/pen

  3. Run a cargo command in the repository's directory.

    cargo install --path cmd/pen

  4. Set a PEN_ROOT environment variable to the directory.

    export PEN_ROOT=<directory>

Now, you are ready to use the pen command built manually.

Testing

This page describes how to write and run unit tests for programs written in the language.

Testing codes consists of the following steps:

  1. Write tests as test functions in test modules.
  2. Run the tests with a pen test command.

Writing tests

You can write tests as test functions in test modules. All modules with the .test.pen file extension are test modules. And, all public functions in test modules are test functions. The test functions need to have a type of \() none | error and should return error values when they fail.

For example, to test a Foo function in a Foo.pen module, write a Foo.test.pen test module with the following contents.

import Test'Assert
import 'Foo

CheckFoo = \() none | error {
  Assert'True(Foo'Foo() == 42)
}

The Test package

The Test standard package includes some utilities which helps you to write tests.

Running tests

To run tests, you can run a pen test command in your package's directory. Then, you should see test results of test functions in test modules. The pen test command exits with a non-zero status code if some tests fail.

Concurrency and parallelism

Concurrent programs use CPU time efficiently without blocking on I/O or synchronization. And parallel programs leverage multi-core CPUs to compute something in parallel for speed.

The language provides simple syntax for concurrent and parallel programming. Its go expression runs a given function concurrently, and possibly in parallel. The expression is composed of the go keyword followed by a function with no argument.

f = go \() number {
  computeExpensive(x, y, z)
}

The go expression returns a function of the type same as the given expression (a.k.a. futures or promises in other languages.) The function returns a resulting value of the function execution.

The go expression may or may not run a given function immediately depending on its implementation. For example, the standard Os system package runs the given function in parallel if multiple CPU cores are available.

Coding style

This page describes the common coding style used in programs written in the language.

Spacing

Use 2 space characters for indentation.

Foo = \(x number) number {
  if x == 0 {
    "Succeeded!"
  } else {
    "Failed..."
  }
}

Naming convention

Naming is important to keep codes consistent. The language currently has the following naming conventions.

KindCase styleExamples
VariablesCamel casefooBar, FooBar, i, x
FunctionsCamel casefooBar, FooBar, f, g
TypesCamel casefooBar, FooBar
ModulesCamel casefooBar, FooBar
Module directoriesCamel casefooBar, FooBar
PackagesUpper camel caseFooBar

Global and local names

You should use descriptive names for global functions and types. But, on the other hand, you are encouraged to use abbreviated names for local variables as long as that doesn't incur ambiguity. For example, you might use the following abbreviated names:

  • i for index
  • c for requestCount
  • sys for system
  • ctx for context

Acronyms

Acronyms are treated as single words.

  • Cpu
  • Ast

References

This chapter details language features, command line tools, and standard packages.

Language

This chapter describes the language's constructs and its type, module and package systems.

Syntax

This page describes syntactical constructs of the language. You can compose programs building up those constructs. See also Types about syntax for specific data types.

Module

Modules are sets of type and function definitions. Syntactically, a module consists of statements. See Modules about how modules themselves interact with each other.

Statements

Statements are constructs that declare functions and types in modules.

Import statement

It imports types and functions from another module from the same or another package.

See Modules for more details.

import Foo'Bar

Foreign import statement

It imports a function from a foreign language.

See Foreign Function Interface (FFI) for more details.

import foreign "c" foo \(number, number) number

Record type definition

It defines a record type.

See Records for more details.

type foo {
  bar number
  baz string
}

Type alias

It gives another name to a type.

type foo = number | none

Function definition

It defines a function with a given name. The right-hand side of = signs must be function expressions.

foo = \(x number, y number) number {
  x + y
}

Foreign function definition

It defines a function exported to foreign languages.

See Foreign Function Interface (FFI) for more details.

foreign "c" foo = \(x number, y number) number {
  x + y
}

Block

A block consists of 1 or more expressions wrapped in { and }. Values of the last expressions are treated as resulting values of the blocks.

{
  foo(ctx, z)

  x + y + z
}

If you want to keep values of intermediate expressions for later use, you can define variables putting their names and = operators in front of the expressions.

{
  x = 42

  ...
}

Expressions

Expressions represent some computation. Expressions can be nested; expressions often contain other expressions inside.

Function call

It calls a function to evaluate it with given arguments returning its result value.

f(x, y)

Operators

Arithmetic

Arithmetic operators add, subtract, multiply, or divide a number with another.

1 + 1
1 - 1
1 * 1
1 / 1

Comparison

Equality

Equal (==) and not-equal (!=) operators compare two values and return a boolean value indicating if they are equal or not.

1 == 1
1 != 1

The operators can compare any types except functions and types containing them.

"foo" == "bar"
foo{x: 0} == foo{x: 1}
42 != none
Ordering

Order operators compare two numbers and return a boolean value indicating if the condition is correct or not.

1 < 1
1 <= 1
1 > 1
1 >= 1

Boolean

A not operator flips a boolean value.

!true

An and operator returns true if both operands are true, or false otherwise.

true & false

An or operator returns true if either operand is true, or false otherwise.

true | false

Error handling

? suffix operators immediately exit the current functions with operands if they are of the error type. Both the operands and result values of functions where the operators are used must be a union type containing the error type.

x?

Function

It creates a function.

First, functions declare their argument names and types (x number and y number) and their result types (number). After that, function bodies of blocks describe how the functions compute result values.

\(x number, y number) number {
  x + y
}

Conditionals

If expression

It evaluates one of blocks depending on a condition of an expression of a boolean type.

  • It evaluates the first block if a given boolean value is true.
  • Otherwise, it evaluates the second block.
if x {
  ...
} else {
  ...
}

If-type expression

It evaluates one of blocks depending on the type of a given expression. The expression (foo()) needs to be bound to a variable (x) and, in each block, the variable is treated as its specified type.

if x = foo() as number {
  ...
} else if string | none {
  ...
} else {
  ...
}

If-list expression

It deconstructs a list and evaluates one of blocks depending on if the list is empty or not.

  • If a given list has 1 or more element, it evaluates the first block with a function that returns its first element (x) and rest of elements as a list (xs).
  • If it has no element, it evaluates the second block.
if [x, ...xs] = ... {
  ...
} else {
  ...
}

Loop

List comprehension

It iterates over elements in a given list and generates a new list with elements of a given expression.

[number f(x) for x in xs]

Concurrency

go expression

It executes a function in parallel. Its returned value is a future represented as a function that returns a result of the executed function.

f = go \() number {
  ...
}

Comment

Comments start with # and end with new-line characters.

# This is a comment.

Types

This page describes different data types in the language.

Number

It represents a real number. It is implemented as a 64-bit floating point number of IEEE 754.

number

Literals

3.14
-42

Boolean

It is a boolean value of true or false.

boolean

Literals

true
false

None

It represents a missing value. It has only a single value of none.

none

Literals

none

String

It is a byte array.

string

Literals

String literals are sequences of bytes. They are often used to represent texts encoded in UTF-8.

"foo"

Escape sequences

String literals can contain the following escape sequences.

Escape sequenceName
\nNew line
\rCarriage return
\tTab
\"Double quote
\\Backslash
\x9fByte

Functions

A function represents reusable computation with arguments and a result.

Functions represent not only pure computation but may also execute side effects, such as I/O.

\(number, number) number

Literals

\(x number, y number) number {
  x + y
}

Lists

It is a list of values. Its element type is put between [ and ].

[number]

Literals

A list literal contains its element type and elements as expressions.

[number]
[number 1]
[number 1, 2, 3]

You can create new lists from existing ones by spreading elements of the old ones prefixed by ... into the new ones.

[number x, ...xs]

Note that expressions within list literals are evaluated lazily; they are evaluated only if their values are required.

Records

It combines multiple types into a single type. Each field of a record type is composed of its name and type.

Fields are not accessible outside modules where they are defined by default.

type person {
  name string
  age number
}

To expose fields as well as the type itself to other modules, you need to capitalize their names.

type Person {
  Name string
  Age number
}

Literals

Record values are constructed using record literals containing their field names and values separated by commas.

person{name: "foo", age: 42}

You can also create new records from existing ones spreading fields of the old ones into the new ones.

person{...one, name: "bar"}

You can access field values by appending their names with . prefixes to expressions of record types.

john.name

Unions

It is a union of multiple types.

For example, the type below represents values that can be either number or none.

number | none

Any

Literally, it's an any type. Any values can be converted to the type.

any

Built-ins

Built-in types and functions are ones implicitly defined in every module.

Types

error

It is a special record type used for error handling. See also Error handling.

type error {
  ...
}

Functions

error

It creates an error with its source information.

\(info any) error

source

It extracts source information from an error.

\(e error) any

debug

It prints a debug message given as an argument if a PEN_DEBUG environment variable is set.

Note that behavior of this function can change among system packages. You may not even see any messages with system packages whose systems do not have any consoles.

\(message string) none

Modules

Modules are sets of functions and types. Using modules, you can split large programs into smaller chunks.

Each source file suffixed with a .pen file extension composes a module. Modules can import functions and types from other modules.

Exporting functions and types from modules

You can name functions and types in an upper camel case for them to be accessible from other modules.

type Foo {
  ...
}

type Bar = ...

Foo = \() number {
  ...
}

Importing functions and types from modules

In order to import functions and types from other modules, place import statements at the top of modules.

The first components in the statements are names of external packages you declare in package configuration files (Foo.) They are omitted if imported modules are in the same packages. The rest of the components are directory names where the modules exist (Bar) and the modules' filenames without their file extensions (Baz for Baz.pen.)

import Foo'Bar'Baz

Then, you can access exported members of the modules with their prefixes.

type foo = Baz'Type

bar = \(x number) number {
  Baz'Function(x)
}

Module names

Modules in the same package

Modules in the same package are referenced by their paths relative to a root directory of the package.

For example, a module of a file at <package directory>/Foo/Bar.pen is imported as below.

import 'Foo'Bar

Modules in other packages

Modules in other packages are referenced by their package names defined in package configuration files and module paths.

For example, a module of a file at <package directory>/Bar/Baz.pen in a package Foo is imported as below.

import Foo'Bar'Baz

Private modules

For modules to be private and not accessible from other packages, you can name them in lower camel case (e.g. fooBar.)

Custom prefixes

Imported modules can have custom prefixes given different names after the as keywords.

import Foo'Bar'Baz as Blah

Unqualified import

You can import functions and types without prefixes by putting their names between { and } in import statements. This is especially useful when module names and imported functions or types have the same names like import 'MyType { MyType }.

import Foo'Bar { Foo, Bar }

type Baz {
  foo Foo
}

Blah = \() number {
  Bar()
}

Packages

Packages are sets of modules. Like modules, packages can import other packages specifying them in their configurations.

What composes a package?

The following entities compose packages.

  • Standard packages bundled in installation of the language
  • Remote repositories managed by version control systems (VCS)
    • Currently, the language supports only Git as a VCS.
  • Directories with package configuration files on file systems

During builds of packages, the language's build system automatically download and initialize their dependency packages based on their URLs.

Package types

There are 3 package types: application, library, and system. Those types are specified in package configuration files.

Application packages

Application packages build applications often as executable files. Every application package must have a main.pen module file at its top directory. The main module has a main function that receives an argument of a context type and returns a none type. The context type is a record type containing context values of system packages with their field names of package names. For example, given system packages named Http and Os, a main function looks like the following.

main = \(ctx context) none {
  s = fetch(ctx.Http, "https://pen-lang.org/")
  print(ctx.Os, s)

  none
}

Every application package must specify one and only one system package that links applications (e.g. the Os standard system package) in its package configuration file. However, application packages can specify system packages that do not link applications (e.g. the Http system package in the example above) as many as possible.

Library packages

Library packages contain functions and types that have no side effects. They are imported and used by other packages.

System packages

System packages contain functions and types that have side effects to provide system interfaces to application packages. The language currently provides the two standard system packages of Os and OsSync.

Although they can be imported by library packages as well as application packages, then they are expected not to cause any side effects.

If you want to write your own system packages, see Writing system packages.

Package configuration

Each package has its configuration file named pen.json in a JSON format at its top directory. The JSON file has the following fields.

NameRequiredDescription
typeYesPackage type (either application, library, or system)
dependenciesYesMap of package names to their URLs

Package URLs have different protocol schemes depending on where they are located.

  • Standard packages: pen
  • Git repositories: git
  • Directories on file systems: none

Examples

Application

{
  "type": "application",
  "dependencies": {
    "Os": "pen:///os",
    "Core": "pen:///core",
    "Foo": "git://github.com/foo/foo",
    "Bar": "../bar"
  }
}

Library

{
  "type": "library",
  "dependencies": {
    "Core": "pen:///core",
    "Foo": "git://github.com/foo/foo",
    "Bar": "../bar"
  }
}

System

{
  "type": "system",
  "dependencies": {
    "Core": "pen:///core",
    "Foo": "git://github.com/foo/foo",
    "Bar": "../bar"
  }
}

Command line tools

The pen command has the following sub-commands.

For more information, see its help message by running pen --help.

build command

It builds a package in the current directory.

pen build

create command

It creates a package of a given kind in a specified directory.

Creating an application package

pen create foo

Creating a library package

pen create --library foo

Standard packages

Standard packages provide common utility functions and system interfaces. They are installed together with the pen command on your system. This chapter describes their APIs and how to use them.

Core package

This package provides common algorithms and data structures.

Install

{
  "dependencies": {
    "Core": "pen:///core"
  } 
}

Core'Bit module

This module provides bitwise operations.

Most functions defined in this module take arguments of 64-bit integers. They can be converted from and into integers represented in IEEE-754 of a number type.

Types

No types are defined.

Functions

And

Calculate bitwise "and" given two 64-bit integers.

\(x number, y number) number

Or

Calculate bitwise "or" given two 64-bit integers.

\(x number, y number) number

Xor

Calculate bitwise exclusive-"or" given two 64-bit integers.

\(x number, y number) number

Not

Calculate bitwise "not" given two 64-bit integers.

\(x number) number

LeftShift

Calculate unsigned left shift given a 64-bit integer.

\(x number, n number) number

RightShift

Calculate unsigned right shift given a 64-bit integer.

\(x number, n number) number

ToInteger64

Convert an integer in IEEE-754 to a 64-bit integer.

\(x number) number

FromInteger64

Convert a 64-bit integer to an integer in IEEE-754.

\(x number) number

Core'Boolean module

This module provides common boolean operations.

Types

No types are defined.

Functions

Any

Return true if any of given booleans are true or false otherwise.

\(bs [boolean]) boolean

All

Return true if all of given booleans are true or false otherwise.

\(bs [boolean]) boolean

Core'List module

This module provides common list operations.

Types

No types are defined.

Functions

Length

Calculate a length of a list.

\(xs [any]) number

First

Get the first element in a list. If a list is empty, it returns a fallback value.

\(xs [any], fallback any) any

Last

Get the last element in a list. If a list is empty, it returns a fallback value.

\(xs [any], fallback any) any

ToNumbers

Convert a list of an any type to one of a number type skipping non-number types.

\(xs [any]) [number]

ToStrings

Convert a list of an any type to one of a string type skipping non-string types.

\(xs [any]) [string]

ToBooleans

Convert a list of an any type to one of a boolean type skipping non-boolean types.

\(xs [any]) [boolean]

Core'Number module

This module provides common number operations.

Types

No types are defined.

Functions

String

Convert a number into its string representation.

\(x number) string

Sum

Sum up numbers.

\(xs [number]) number

Remainder

Calculate a remainder.

\(dividend number, divisor number) number

Power

Calculate a power.

\(base number, exponent number) number

SquareRoot

Calculate a square root.

\(x number) number

Sequence

Create a list of numbers from 1 to a maximum value.

\(maximum number) [number]

Range

Create a list of numbers from a minimum to a maximum.

\(minimum number, maximum number) [number]

IsNan

Check if a number is NaN.

\(x number) boolean

Ceil

Calculate a ceil value.

\(x number) number

Floor

Calculate a floor value.

\(x number) number

Nan

NaN

\() number

Infinity

Infinity

\() number

Minimum

Calculate a minimum.

\(x number, y number) number

Maximum

Calculate a maximum.

\(x number, y number) number

Core'String module

This module provides common string operations.

Types

No types are defined.

Functions

Concatenate

Concatenate strings.

\(strings [string]) string

Join

Join strings with a separator.

\(strings [string], separator string) string

HasPrefix

Check if a string has a prefix.

\(s string, prefix string) boolean

Core'String'Byte module

This module provides operations for strings as byte arrays.

Types

No types are defined.

Functions

Length

Return a length of a string.

\(s string) number

Slice

Slice a string.

\(s string, start number, end number) string

Core'String'Byte'View module

This module provides views of strings as byte arrays. By using those views, you can apply operations to strings without copying them.

Types

View

A view of a string

type View {
  # ...
}

Functions

New

Create a view of a string.

\(s string) View

Viewee

Get an original string.

\(v View) string

Start

Get a start index of a view.

\(v View) number

End

Get an end index of a view.

\(v View) number

HasPrefix

Check if a string has a prefix.

\(v View, s string) boolean

Length

Get a length of a view.

\(v View) number

Seek

Move a start index forward.

\(v View, n number) View

Slice

Slice a view.

\(v View, start number, end number) View

ToString

Convert a view into a string.

\(v View) string

Core'String'Utf8 module

This module provides operations for strings encoded in UTF-8.

Types

No types are defined.

Functions

Length

Return a length of a string.

\(s string) number

Slice

Slice a string.

\(s string, start number, end number) string

Flag package

This package provides command-line flag parsing.

Install

{
  "dependencies": {
    "Flag": "pen:///flag"
  } 
}

Flag'Flag module

This module provides command-line flag parsing.

The flags have the following formats:

  • -foo bar: a named flag of a key foo with a value bar
  • foo: a positional flag of a value foo

Named flags and positional flags are separated by -- optionally.

Types

Set

A set of flags

type Set {
  Named {string: string}
  Positional [string]
}

Functions

Parse

Parse flags.

\(ss [string]) Set | error

Json package

This package provides a JSON parser.

Install

{
  "dependencies": {
    "Json": "pen:///json"
  } 
}

Json'Decode module

Types

No types are defined.

Functions

Decode

Decode a string into a JSON value.

\(s string) Value | error

Json'Encode module

Types

No types are defined.

Functions

Encode

Encode a JSON value.

\(v Value) string

Json'Value module

Types

Value

A JSON value

type Value {
  # ...
}

Raw

A raw JSON value represented by built-in types

type Raw = boolean | none | number | string | [Value] | {string: Value}

Functions

New

Create a JSON value.

\(r Raw) Value

Raw

Get a raw value.

\(v Value) Raw

Http package

This package provides HTTP client and server.

Install

{
  "dependencies": {
    "Http": "pen:///http"
  } 
}

Http'Client module

This module provides an HTTP client.

Types

No types are defined.

Functions

Send

Send an HTTP request.

\(ctx Context, r Request) Response | error

Http'Context module

Types

Context

An HTTP context

type Context = context'Context

Functions

No functions are defined.

Http'Request module

Types

Request

An HTTP request

type Request {
  Method string
  Uri string
  Headers {string: string}
  Body string
}

Functions

No functions are defined.

Http'Response module

Types

Response

An HTTP response

type Response {
  Status number
  Headers {string: string}
  Body string
}

Functions

No functions are defined.

Http'Server module

This module provides an HTTP server.

Types

No types are defined.

Functions

Serve

Run an HTTP service.

\(ctx Context, address string, callback \(Request) Response) none | error

Os package

This package provides an interface for operating systems.

Install

{
  "dependencies": {
    "Os": "pen:///os"
  } 
}

Os'Context module

Types

Context

A context of an operating system.

type Context = context'Context

Functions

No functions are defined.

Os'Directory module

Types

No types are defined.

Functions

Read

Read a directory and return file paths it contains.

\(ctx Context, path string) [string] | error

Create

Create a directory.

\(ctx Context, path string) none | error

Remove

Remove a directory.

\(ctx Context, path string) none | error

Os'Environment module

Types

No types are defined.

Functions

Arguments

Get command line arguments.

\(ctx Context) [string]

Variable

Get an environment variable.

\(ctx Context, name string) string | error

Os'File module

Types

File

A file on a file system.

type File {
  # ...
}

Functions

StdIn

A file of standard input.

\() File

StdOut

A file of standard output.

\() File

StdErr

A file of standard error.

\() File

Open

Open a file for read-only.

\(ctx Context, path string) File | error

OpenWithOptions

Open a file with options.

\(ctx Context, path string, opt OpenOptions) File | error

Read

Read a file.

\(ctx Context, file File) string | error

ReadLimit

Read a file until a size limit.

\(ctx Context, file File, limit number) string | error

Write

Write data to a file.

\(ctx Context, file File, data string) number | error

Copy

Copy a file to another path.

\(ctx Context, src string, dest string) none | error

Move

Move a file to another path.

\(ctx Context, src string, dest string) none | error

Remove

Remove a file.

\(ctx Context, path string) none | error

Metadata

Get metadata of a file at a path.

\(ctx Context, path string) Metadata | error

Os'File'Metadata module

Types

Metadata

File metadata

type Metadata {
  Size number
}

Functions

No functions are defined.

Os'File'OpenOptions module

Types

OpenOptions

Options to open a file

  • Append allows appending data to the file.
  • Create creates a new file if the file doesn't exist or opens it otherwise.
  • CreateNew creates a new file. If the file already exists, it emits an error.
  • Read allows reading data from the file.
  • Truncate truncates the file to zero byte.
  • Write allows writing data to the file.
type OpenOptions {
  Append boolean
  Create boolean
  CreateNew boolean
  Read boolean
  Truncate boolean
  Write boolean
}

Functions

Default

Get default options to open a file. They do not include any permission.

\() OpenOptions

Os'Process module

Types

No types are defined.

Functions

Exit

Exit a current process.

\(ctx Context, code number) none

Os'Random module

Types

No types are defined.

Functions

Number

Generate a random number in a range of [0, 1).

\(ctx Context) number

Os'Tcp module

Types

No types are defined.

Functions

Bind

Create a listener bound to a server address.

\(ctx Context, address string) Listener | error

Connect

Create a stream connected to a peer address.

\(ctx Context, address string) Stream | error

Accept

Accept a client connection and create its stream.

\(ctx Context, l Listener) AcceptedStream | error

Receive

Receive data from a peer through a stream with a size limit in bytes.

\(ctx Context, s Stream, limit number) string | error

Send

Send data to a peer through a stream.

\(ctx Context, s Stream, data string) number | error

Os'Tcp'AcceptedStream module

Types

AcceptedStream

A TCP stream accepted on a server with a client address

type AcceptedStream {
  Stream Stream
  Address string
}

Functions

No functions are defined.

Os'Tcp'Listener module

Types

Listener

A TCP listener to listen for client connections

type Listener {
  # ...
}

Functions

No functions are defined.

Os'Tcp'Stream module

Types

Stream

A TCP stream

type Stream {
  # ...
}

Functions

No functions are defined.

Os'Time module

Types

No types are defined.

Functions

Now

Fetch a current system time in milliseconds.

\(ctx Context) number

Sleep

Pause a current execution context for a given amount of time.

\(ctx Context, milliseconds number) none

Os'Udp module

Types

No types are defined.

Functions

Bind

Bind a socket with a server address.

\(ctx Context, address string) Socket | error

Connect

Connect a socket to a peer address.

\(ctx Context, s Socket, address string) none | error

Receive

Receive a datagram from a connected address.

\(ctx Context, s Socket) string | error

ReceiveFrom

Receive a datagram from any address.

\(ctx Context, s Socket) Datagram | error

Send

Send a datagram to a connected address.

\(ctx Context, s Socket, data string) number | error

SendTo

Send a datagram to a specified address.

\(ctx Context, s Socket, data string, address string) number | error

Os'Udp'Datagram module

Types

Datagram

UDP datagram

type Datagram {
  Data string
  Address string
}

Functions

No functions are defined.

Os'Udp'Socket module

Types

Socket

UDP socket

type Socket {
  # ...
}

Functions

No functions are defined.

Sql package

This package provides a SQL database client.

Install

{
  "dependencies": {
    "Sql": "pen:///sql"
  } 
}

Sql'Context module

Types

Context

A SQL database context

type Context = context'Context

Functions

No functions are defined.

Sql'Pool module

Types

Pool

A connection pool

type Pool = pool'Pool

Functions

New

Create a connection pool.

\(context Context, uri string, options Options) Pool | error

Query

Run a query and return its rows.

\(context Context, pool Pool, query string, arguments [Value]) [[Value]] | error

Execute

Run a query and return a number of affected rows.

\(context Context, pool Pool, query string, arguments [Value]) number | error

Sql'Pool'Options module

Types

Options

Connection pool options

type Options {
  MinConnections number
  MaxConnections number
  ConnectTimeout number
}

Functions

No functions are defined.

Sql'Value module

Types

Value

A value in a column

type Value = boolean | none | number | string

Functions

No functions are defined.

Test package

This package provides test utilities.

Install

{
  "dependencies": {
    "Test": "pen:///test"
  } 
}

Test'Assert module

This module provides a collection of assertion logic for testing.

Types

No types are defined.

Functions

True

Assert that a condition is true.

\(x boolean) none | error

Error

Check if a value is an error.

\(x any) none | error

Fail

Fail with an error immediately. This function is useful to make unreachable codes fail.

\() error

Examples

These are examples on how to use the language, its command line tools, and libraries.

Types

These are examples on how to use types and their values in the language.

Number

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Use a number literal

Given a file named "Foo.pen" with:

f = \() number {
  42
}

When I run pen build

Then the exit status should be 0.

Use arithmetic operators

Given a file named "Foo.pen" with:

f = \() number {
  1 + 2 - 3 * 4 / 5
}

When I run pen build

Then the exit status should be 0.

Use equality operators

Given a file named "Foo.pen" with:

f = \() boolean {
  0 == 0
}

g = \() boolean {
  0 != 0
}

When I run pen build

Then the exit status should be 0.

Use order operators

Given a file named "Foo.pen" with:

f = \() boolean {
  0 < 0
}

g = \() boolean {
  0 <= 0
}

h = \() boolean {
  0 > 0
}

i = \() boolean {
  0 >= 0
}

When I run pen build

Then the exit status should be 0.

Boolean

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Use boolean literals

Given a file named "Foo.pen" with:

f = \() boolean {
  true
}

g = \() boolean {
  false
}

When I run pen build

Then the exit status should be 0.

Use an and operation

Given a file named "Foo.pen" with:

f = \() boolean {
  true & false
}

When I run pen build

Then the exit status should be 0.

Use an or operation

Given a file named "Foo.pen" with:

f = \() boolean {
  true | false
}

When I run pen build

Then the exit status should be 0.

Use a not operation

Given a file named "Foo.pen" with:

f = \() boolean {
  !true
}

When I run pen build

Then the exit status should be 0.

Use an if expression

Given a file named "Foo.pen" with:

f = \() number {
  if true {
    1
  } else {
    0
  }
}

When I run pen build

Then the exit status should be 0.

None

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Use a none literal

Given a file named "Foo.pen" with:

f = \() none {
  none
}

When I run pen build

Then the exit status should be 0.

String

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Use a string literal

Given a file named "Foo.pen" with:

f = \() string {
  "foo"
}

When I run pen build

Then the exit status should be 0.

Use equality operators

Given a file named "Foo.pen" with:

f = \() boolean {
  "" == ""
}

g = \() boolean {
  "" != ""
}

When I run pen build

Then the exit status should be 0.

Function

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Define a function

Given a file named "Foo.pen" with:

f = \(x number) number {
  x
}

When I run pen build

Then the exit status should be 0.

Call a function with no argument

Given a file named "Foo.pen" with:

f = \() number {
  f()
}

When I run pen build

Then the exit status should be 0.

Call a function with an argument

Given a file named "Foo.pen" with:

f = \(x number) number {
  f(x)
}

When I run pen build

Then the exit status should be 0.

Call a function with two arguments

Given a file named "Foo.pen" with:

f = \(x number, y number) number {
  f(x, y)
}

When I run pen build

Then the exit status should be 0.

Define a closure

Given a file named "Foo.pen" with:

f = \(x number) \(number) number {
  \(y number) number {
    x + y
  }
}

When I run pen build

Then the exit status should be 0.

Record

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Create a record with a field

Given a file named "Foo.pen" with:

type r {
  x number
}

f = \() r {
  r{x: 42}
}

When I run pen build

Then the exit status should be 0.

Create a record with two fields

Given a file named "Foo.pen" with:

type r {
  x number
  y none
}

f = \() r {
  r{x: 42, y: none}
}

When I run pen build

Then the exit status should be 0.

Create a record with no field

Given a file named "Foo.pen" with:

type r {}

f = \() r {
  r{}
}

When I run pen build

Then the exit status should be 0.

Update a record

Given a file named "Foo.pen" with:

type r {
  x number
  y none
}

f = \(x r) r {
  r{...x, y: none}
}

When I run pen build

Then the exit status should be 0.

Get a field in a record

Given a file named "Foo.pen" with:

type r {
  x number
}

f = \(x r) number {
  x.x
}

When I run pen build

Then the exit status should be 0.

Use an equal operator

Given a file named "Foo.pen" with:

type r {
  x number
}

f = \(x r, y r) boolean {
  x == y
}

When I run pen build

Then the exit status should be 0.

Use a not-equal operator

Given a file named "Foo.pen" with:

type r {
  x number
}

f = \(x r, y r) boolean {
  x == y
}

When I run pen build

Then the exit status should be 0.

Union

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Upcast a number into a union

Given a file named "Foo.pen" with:

f = \() number | none {
  42
}

When I run pen build

Then the exit status should be 0.

Upcast a function into a union

Given a file named "Foo.pen" with:

f = \() (\() number) | none {
  \() number {
    42
  }
}

When I run pen build

Then the exit status should be 0.

Upcast a list into a union

Given a file named "Foo.pen" with:

f = \() [number] | none {
  [number 42]
}

When I run pen build

Then the exit status should be 0.

Downcast a union type

Given a file named "Foo.pen" with:

f = \(x number | none) number {
  if x = x as number {
    x
  } else if none {
    0
  }
}

When I run pen build

Then the exit status should be 0.

Downcast a union type with an else block

Given a file named "Foo.pen" with:

f = \(x number | none) number {
  if x = x as none {
    0
  } else {
    x
  }
}

When I run pen build

Then the exit status should be 0.

Downcast a union type to another union type

Given a file named "Foo.pen" with:

f = \(x number | boolean | none) number | none {
  if x = x as number | none {
    x
  } else {
    none
  }
}

When I run pen build

Then the exit status should be 0.

List

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Create an empty list

Given a file named "Foo.pen" with:

f = \() [number] {
  [number]
}

When I run pen build

Then the exit status should be 0.

Create a list with an element

Given a file named "Foo.pen" with:

f = \() [number] {
  [number 1]
}

When I run pen build

Then the exit status should be 0.

Create a list with two elements

Given a file named "Foo.pen" with:

f = \() [number] {
  [number 1, 2]
}

When I run pen build

Then the exit status should be 0.

Join lists

Given a file named "Foo.pen" with:

f = \(xs [number]) [number] {
  [number ...xs, ...xs]
}

When I run pen build

Then the exit status should be 0.

Create a list of a union type

Given a file named "Foo.pen" with:

f = \() [number | none] {
  [number | none 1, none]
}

When I run pen build

Then the exit status should be 0.

Coerce elements of a spread list

Given a file named "Foo.pen" with:

f = \(xs [number]) [number | none] {
  [number | none ...xs]
}

When I run pen build

Then the exit status should be 0.

Use if-list expression

Given a file named "Foo.pen" with:

f = \(xs [number]) [number] {
  if [y, ...ys] = xs {
    [number y(), ...ys]
  } else {
    [number]
  }
}

When I run pen build

Then the exit status should be 0.

Use list comprehension

Given a file named "Foo.pen" with:

f = \(xs [number]) [number] {
  [number x() + 42 for x in xs]
}

When I run pen build

Then the exit status should be 0.

Error

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Call an error function

Given a file named "Foo.pen" with:

f = \() error {
  error(none)
}

When I run pen build

Then the exit status should be 0.

Call a source function

Given a file named "Foo.pen" with:

f = \(e error) any {
  source(e)
}

When I run pen build

Then the exit status should be 0.

Use a try operator

Given a file named "Foo.pen" with:

f = \(x number | error) number | error {
  x? + 1
}

When I run pen build

Then the exit status should be 0.

Use a try operator with a union type

Given a file named "Foo.pen" with:

f = \(x number | none | error) number | error {
  if x = x? as number {
    x + 1
  } else if none {
    0
  }
}

When I run pen build

Then the exit status should be 0.

Any

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Use an any type

Given a file named "Foo.pen" with:

f = \() any {
  42
}

When I run pen build

Then the exit status should be 0.

Downcast an any type

Given a file named "Foo.pen" with:

f = \(x any) number {
  if x = x as number {
    x
  } else {
    0
  }
}

When I run pen build

Then the exit status should be 0.

Polymorphism

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Use an equal operator

Given a file named "Foo.pen" with:

f = \(x number | none) boolean {
  x == none
}

When I run pen build

Then the exit status should be 0.

Use a not-equal operator

Given a file named "Foo.pen" with:

f = \(x number | none) boolean {
  x != none
}

When I run pen build

Then the exit status should be 0.

Compare unions

Given a file named "Foo.pen" with:

f = \(x number | none, y number | none) boolean {
  x == y
}

When I run pen build

Then the exit status should be 0.

Compare a union and none

Given a file named "Foo.pen" with:

f = \(x number | none) boolean {
  x == none
}

When I run pen build

Then the exit status should be 0.

Syntax

These are examples on how to compose programs in the language.

Block

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Define a variable

Given a file named "Foo.pen" with:

f = \(x number) number {
  y = x

  y
}

When I run pen build

Then the exit status should be 0.

Call a function

Given a file named "Foo.pen" with:

f = \() none {
  none
}

g = \() none {
  f()

  none
}

When I run pen build

Then the exit status should be 0.

Use if expression

Given a file named "Foo.pen" with:

f = \() none {
  none
}

g = \() none {
  none
}

h = \(x boolean) none {
  if x {
    f()
  } else {
    g()
  }

  none
}

When I run pen build

Then the exit status should be 0.

Modules

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Import a function from a module

Given a file named "Foo.pen" with:

Foo = \() number {
  42
}

And a file named "Bar.pen" with:

import 'Foo

Bar = \() number {
  Foo'Foo()
}

When I run pen build

Then the exit status should be 0.

Import a type alias from a module

Given a file named "Foo.pen" with:

type Foo = number

And a file named "Bar.pen" with:

import 'Foo

type Bar = Foo'Foo

When I run pen build

Then the exit status should be 0.

Import a function from a nested module

Given a file named "Foo/Foo.pen" with:

Foo = \() number {
  42
}

And a file named "Bar.pen" with:

import 'Foo'Foo

Bar = \() number {
  Foo'Foo()
}

When I run pen build

Then the exit status should be 0.

Import a module with a custom prefix

Given a file named "Foo.pen" with:

Foo = \() number {
  42
}

And a file named "Bar.pen" with:

import 'Foo as Bar

Bar = \() number {
  Bar'Foo()
}

When I run pen build

Then the exit status should be 0.

Import a type definition with no prefix

Given a file named "Foo.pen" with:

type Foo {}

And a file named "Bar.pen" with:

import 'Foo { Foo }

type Bar = Foo

When I run pen build

Then the exit status should be 0.

Import a type alias with no prefix

Given a file named "Foo.pen" with:

type Foo = number

And a file named "Bar.pen" with:

import 'Foo { Foo }

type Bar = Foo

When I run pen build

Then the exit status should be 0.

Import a function with no prefix

Given a file named "Foo.pen" with:

Foo = \() number {
  42
}

And a file named "Bar.pen" with:

import 'Foo { Foo }

Bar = \() number {
  Foo()
}

When I run pen build

Then the exit status should be 0.

Packages

Background

Given a file named "foo/pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

And a file named "foo/Foo.pen" with:

type Foo = number

Foo = \() number {
  42
}

And a file named "foo/Foo/Foo.pen" with:

Foo = \() number {
  42
}

And a directory named "bar"

And I cd to "bar"

And a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {
    "Foo": "../foo"
  }
}

Import a function from a module

Given a file named "Bar.pen" with:

import Foo'Foo

Bar = \() number {
  Foo'Foo()
}

When I run pen build

Then the exit status should be 0.

Import a type from a module

Given a file named "Bar.pen" with:

import Foo'Foo

type Bar = Foo'Foo

When I run pen build

Then the exit status should be 0.

Import a function from a nested module

Given a file named "Bar.pen" with:

import Foo'Foo'Foo

Bar = \() number {
  Foo'Foo()
}

When I run pen build

Then the exit status should be 0.

Commands

These are examples on how to use command line tools of the language.

Building packages

Build an application package

Given a file named "pen.json" with:

{
  "type": "application",
  "dependencies": {
    "Os": "pen:///os"
  }
}

And a file named "main.pen" with:

import Os'Context { Context }

main = \(ctx context) none {
  none
}

When I successfully run pen build

Then I successfully run ./app.

Build a library package

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

And a file named "Foo.pen" with:

f = \(x number) number {
  x
}

When I run pen build

Then the exit status should be 0.

Cross-build an application package

Given a file named "pen.json" with:

{
  "type": "application",
  "dependencies": {
    "Os": "pen:///os-sync"
  }
}

And a file named "main.pen" with:

import Os'Context { Context }

main = \(ctx context) none {
  none
}

And I successfully run rustup target add <target>

When I run pen build --target <target>

Then the exit status should be 0.

Examples

target
i686-unknown-linux-musl
x86_64-unknown-linux-musl
aarch64-unknown-linux-musl
wasm32-wasi

Cross-build a library package

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

And a file named "Foo.pen" with:

f = \(x number) number {
  x
}

And I successfully run rustup target add <target>

When I run pen build --target <target>

Then the exit status should be 0.

Examples

target
i686-unknown-linux-musl
x86_64-unknown-linux-musl
aarch64-unknown-linux-musl
wasm32-wasi

Creating packages

Create an application package

Given I successfully run pen create foo

And I cd to "foo"

When I successfully run pen build

Then I successfully run ./app.

Create a library package

Given I successfully run pen create --library foo

And I cd to "foo"

When I run pen build

Then the exit status should be 0.

Create an application package in a current directory

Given I successfully run pen create .

When I successfully run pen build

Then I successfully run ./app.

Create a library package in a current directory

Given I successfully run pen create --library .

When I run pen build

Then the exit status should be 0.

Testing packages

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {
    "Test": "pen:///test"
  }
}

And a file named "Foo.pen" with:

Add = \(x number, y number) number {
  x + y
}

Test a module

Given a file named "Foo.test.pen" with:

import Test'Assert
import 'Foo

Add = \() none | error {
  Assert'True(Foo'Add(41, 1) == 42)
}

When I run pen test

Then the exit status should be 0

And the stdout should contain "OK".

Fail to test a module

Given a file named "Foo.test.pen" with:

import Test'Assert
import 'Foo

Add = \() none | error {
  Assert'True(Foo'Add(41, 0) == 42)
}

When I run pen test

Then the exit status should not be 0

And the stdout should contain "FAIL".

Run no test

When I run pen test

Then the exit status should be 0.

Use a debug function in a test

Given a file named "Foo.test.pen" with:

Foo = \() none | error {
  debug("hello")
}

And I append "1" to the environment variable "PEN_DEBUG"

When I run pen_test_on_linux.sh

Then the exit status should be 0.

Formatting module files

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Format module files

Given a file named "Foo.pen" with:

Foo = \() none {

  none
}

When I successfully run pen format

Then a file named "Foo.pen" should contain exactly:

Foo = \() none {
  none
}

Check if module files are formatted

Given a file named "Foo.pen" with:

Foo = \() none {

  none
}

When I run pen format --checked

Then the exit status should not be 0.

Standard packages

These are examples on how to use standard packages.

Core

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {
    "Core": "pen:///core"
  }
}

Convert a number to a string

Given a file named "Foo.pen" with:

import Core'Number

f = \() string {
  Number'String(42)
}

When I run pen build

Then the exit status should be 0.

Sum numbers

Given a file named "Foo.pen" with:

import Core'Number

f = \() number {
  Number'Sum([number 1, 2, 3])
}

When I run pen build

Then the exit status should be 0.

Join strings

Given a file named "Foo.pen" with:

import Core'String

f = \() string {
  String'Join([string "hello", "world"], " ")
}

When I run pen build

Then the exit status should be 0.

Slice a string as bytes

Given a file named "Foo.pen" with:

import Core'String'Byte

f = \() string {
  Byte'Slice("foo", 1, 2)
}

When I run pen build

Then the exit status should be 0.

Get a length of a list

Given a file named "Foo.pen" with:

import Core'List

f = \() number {
  xs = [none]

  List'Length([any ...xs])
}

When I run pen build

Then the exit status should be 0.

Os

Background

Given a file named "pen.json" with:

{
  "type": "application",
  "dependencies": {
    "Os": "pen:///os",
    "Core": "pen:///core"
  }
}

Build an application

Given a file named "main.pen" with:

import Os'Context { Context }
import Os'File
import Os'Process

main = \(ctx context) none {
  if _ = run(ctx.Os) as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

run = \(ctx Context) none | error {
  File'Write(ctx, File'StdOut(), "Hello, world!")?

  none
}

When I successfully run pen build

Then I successfully run ./app

And the stdout from "./app" should contain exactly "Hello, world!".

Get arguments

Given a file named "main.pen" with:

import Core'String
import Os'Context { Context }
import Os'Environment
import Os'File
import Os'Process

main = \(ctx context) none {
  if _ = run(ctx.Os) as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

run = \(ctx Context) none | error {
  File'Write(ctx, File'StdOut(), String'Join(Environment'Arguments(ctx), " "))?

  none
}

When I successfully run pen build

Then I successfully run ./app foo bar

And stdout from "./app foo bar" should contain exactly "foo bar".

Get an environment variable

Given a file named "main.pen" with:

import Core'String
import Os'Context { Context }
import Os'File
import Os'Environment
import Os'Process

main = \(ctx context) none {
  if _ = run(ctx.Os) as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

run = \(ctx Context) none | error {
  File'Write(ctx, File'StdOut(), Environment'Variable(ctx, "FOO")?)?

  none
}

And I append "foo" to the environment variable "FOO"

When I successfully run pen build

Then I successfully run ./app

And stdout from "./app" should contain exactly "foo".

Open a file

Given a file named "main.pen" with:

import Os'Context { Context }
import Os'File { File }
import Os'Process

main = \(ctx context) none {
  if _ = run(ctx.Os) as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

run = \(ctx Context) none | error {
  File'Open(ctx, "./foo.txt")?

  none
}

And a file named "foo.txt" with ""

When I successfully run pen build

Then I successfully run ./app.

Read a file

Given a file named "main.pen" with:

import Os'Context { Context }
import Os'File
import Os'File'OpenOptions
import Os'Process

main = \(ctx context) none {
  if _ = run(ctx.Os) as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

run = \(ctx Context) none | error {
  f = File'Open(ctx, "foo.txt")?
  d = File'Read(ctx, f)?

  File'Write(ctx, File'StdOut(), d)?

  none
}

And a file named "foo.txt" with "foo"

When I successfully run pen build

Then I successfully run ./app

And the stdout from "./app" should contain exactly "foo".

Read a file until a limit

Given a file named "main.pen" with:

import Os'Context { Context }
import Os'File
import Os'Process

main = \(ctx context) none {
  if _ = run(ctx.Os) as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

run = \(ctx Context) none | error {
  f = File'Open(ctx, "foo.txt")?
  d = File'ReadLimit(ctx, f, 5)?
  File'Write(ctx, File'StdOut(), d)?

  none
}

And a file named "foo.txt" with "Hello, world!"

When I successfully run pen build

Then I successfully run ./app

And the stdout from "./app" should contain exactly "Hello".

Write a file

Given a file named "main.pen" with:

import Os'Context { Context }
import Os'File
import Os'File'OpenOptions { OpenOptions }
import Os'Process

main = \(ctx context) none {
  if _ = run(ctx.Os) as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

run = \(ctx Context) none | error {
  f = File'OpenWithOptions(
    ctx,
    "./foo.txt",
    OpenOptions{...OpenOptions'Default(), Write: true},
  )?

  File'Write(ctx, f, "foo")?

  none
}

And a file named "foo.txt" with ""

When I successfully run pen build

Then I successfully run ./app

And the file "foo.txt" should contain "foo".

Copy a file

Given a file named "main.pen" with:

import Os'Context { Context }
import Os'File
import Os'Process

main = \(ctx context) none {
  if _ = File'Copy(ctx.Os, "foo.txt", "bar.txt") as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

And a file named "foo.txt" with "foo"

When I successfully run pen build

Then I successfully run ./app

And the file "foo.txt" should contain "foo"

And the file "bar.txt" should contain "foo".

Move a file

Given a file named "main.pen" with:

import Os'Context { Context }
import Os'File
import Os'Process

main = \(ctx context) none {
  if _ = File'Move(ctx.Os, "foo.txt", "bar.txt") as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

And a file named "foo.txt" with "foo"

When I successfully run pen build

Then I successfully run ./app

And the file "foo.txt" does not exist

And the file "bar.txt" should contain "foo".

Remove a file

Given a file named "main.pen" with:

import Os'Context { Context }
import Os'File
import Os'Process

main = \(ctx context) none {
  if _ = File'Remove(ctx.Os, "foo.txt") as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

And a file named "foo.txt" with ""

When I successfully run pen build

Then I successfully run ./app

And the file "foo.txt" does not exist.

Read a directory

Given a file named "main.pen" with:

import Core'String
import Os'Context { Context }
import Os'File
import Os'Directory
import Os'Process

main = \(ctx context) none {
  if _ = run(ctx.Os) as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

run = \(ctx Context) none | error {
  File'Write(
    ctx,
    File'StdOut(),
    String'Join(Directory'Read(ctx, ".")?, "\n"),
  )?

  none
}

When I successfully run pen build

Then I successfully run ./app

And the stdout from "./app" should contain "main.pen"

And the stdout from "./app" should contain "pen.json".

Create a directory

Given a file named "main.pen" with:

import Os'Context { Context }
import Os'Directory
import Os'Process

main = \(ctx context) none {
  if _ = Directory'Create(ctx.Os, "foo") as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

When I successfully run pen build

Then I successfully run ./app

And a directory named "foo" should exist.

Remove a directory

Given a file named "main.pen" with:

import Os'Context { Context }
import Os'Directory
import Os'Process

main = \(ctx context) none {
  if _ = Directory'Remove(ctx.Os, "foo") as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

And a directory named "foo"

When I successfully run pen build

Then I successfully run ./app

And a directory named "foo" should not exist.

Get file metadata

Given a file named "main.pen" with:

import Os'Context { Context }
import Os'File
import Os'File'Metadata { Metadata }
import Os'Process

main = \(ctx context) none {
  m = File'Metadata(ctx.Os, "foo")

  c = if m = m as Metadata {
    if m.Size == 3 {
      0
    } else {
      1
    }
  } else {
    1
  }

  Process'Exit(ctx.Os, c)
}

And a file named "foo" with:

foo

When I successfully run pen build

Then I successfully run ./app.

Get os time

Given a file named "main.pen" with:

import Core'Number
import Os'Context { Context }
import Os'File
import Os'Process
import Os'Time

main = \(ctx context) none {
  if m = run(ctx.Os) as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

run = \(ctx Context) none | error {
  File'Write(ctx, File'StdOut(), Number'String(Time'Now(ctx)))?

  none
}

When I successfully run pen build

Then I successfully run ./app.

Sleep

Given a file named "main.pen" with:

import Os'Context { Context }
import Os'Time

main = \(ctx context) none {
  Time'Sleep(ctx.Os, 1)
}

When I successfully run pen build

Then I successfully run ./app.

Generate a random number

Given a file named "main.pen" with:

import Core'Number
import Os'Context { Context }
import Os'File
import Os'Process
import Os'Random

main = \(ctx context) none {
  if m = run(ctx.Os) as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

run = \(ctx Context) none | error {
  File'Write(ctx, File'StdOut(), Number'String(Random'Number(ctx)))?

  none
}

When I successfully run pen build

Then I successfully run ./app.

Exit a process

Given a file named "main.pen" with:

import Os'Context { Context }
import Os'Process

main = \(ctx context) none {
  Process'Exit(ctx.Os, 42)
}

When I successfully run pen build

Then I run ./app

And the exit status should be 42.

Test

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {
    "Test": "pen:///test"
  }
}

Check if a condition is true

Given a file named "foo.test.pen" with:

import Test'Assert

Foo = \() none | error {
  Assert'True(true)
}

When I run pen test

Then the exit status should be 0.

Check if a value is an error

Given a file named "foo.test.pen" with:

import Test'Assert

Foo = \() none | error {
  Assert'Error(error(none))
}

When I run pen test

Then the exit status should be 0.

Make a test fail

Given a file named "foo.test.pen" with:

import Test'Assert

Foo = \() none | error {
  Assert'Fail()
}

When I run pen test

Then the exit status should be 1.

FFI

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Import a foreign function of native calling convention

Given a file named "Foo.pen" with:

import foreign g \(number) number

f = \(x number) number {
  g(x)
}

When I run pen build

Then the exit status should be 0.

Import a foreign function of the C calling convention

Given a file named "Foo.pen" with:

import foreign "c" g \(number) number

f = \(x number) number {
  g(x)
}

When I run pen build

Then the exit status should be 0.

Export a foreign function

Given a file named "Foo.pen" with:

foreign f = \(x number) number {
  x
}

When I run pen build

Then the exit status should be 0.

Export a foreign function of the C calling convention

Given a file named "Foo.pen" with:

foreign "c" f = \(x number) number {
  x
}

When I run pen build

Then the exit status should be 0.

Advanced features

This chapter lists up several advanced features of the language and describe when and how to use them.

Cross compile

The language's compiler supports cross compile. To compile applications and libraries for different targets, specify the --target option of the pen build subcommand.

For example, run the following command to compile a wasm32 binary for the WASI platform.

pen build --target wasm32-wasi

Note that we currently support those targets via Rust's cross compiler toolchain. Please install a Rust compiler through rustup to enable installation of toolchains for different targets.

Supported targets

Run pen build --help to see all supported targets.

System package support

Cross compile support of system packages are totally up to their developers. For example, the Os standard system package supports most targets as long as their platforms expose OS-like APIs. However, some custom system packages might not support those targets because of their limited use cases.

Foreign Function Interface (FFI)

Using FFI, you can import or export functions in foreign languages, such as Rust and C.

Importing functions in foreign languages

You can import functions in foreign languages using foreign import statements. The statements specify the foreign functions' calling convention, names and types.

You might specify calling conventions of foreign functions in a format of string literals after import foreign keywords optionally. Currently, only the C calling convention is supported as "c". If the options are omitted, the functions are imported with the native calling convention of the language.

import foreign "c" foo \(number, number) number

Caveat: You can import foreign functions that might cause side effects only in system packages. See also Writing system packages.

Exporting functions to foreign languages

You can export functions to foreign languages using foreign function definitions, which have foreign keywords in front of normal function definitions.

You might specify calling conventions of exported foreign functions optionally after foreign keywords as well as imported foreign functions.

foreign "c" foo = \(x number, y number) number {
  ...
}

Building libraries of foreign languages

During builds of your packages, you might want to build libraries of foreign languages so that you can use their functions. If that's your case, you can set up pen-ffi scripts in your packages. The script files run on every build and output absolute paths to .a archive files of your libraries in foreign languages built by the scripts. The script files may or may not have file extensions.

The pen-ffi scripts should accept the following command line arguments.

ArgumentRequiredDescription
-t <target>NoCustom target triple of builds

One of examples in practice is a pen-ffi.sh file in the Core library.

Native calling convention in Pen

WIP

Writing system packages

Using existing system packages covers most use cases in application development. However, by writing your own system packages, you can achieve the following:

  • Define your own system interfaces of functions and types with side effects.
  • Link applications in arbitrary file formats.

This page assumes that you have already read Packages.

Caveat: Providing bad system packages which do not conform to conventions can break the ecosystem of the language! In the worst cases, they might make applications malfunction. Please be careful to follow the conventions to keep applications maintainable and portable.

Functionalities of system packages

System packages have the following functionalities:

  • Define context types.
  • Provide system interfaces as functions and types.
  • Link application files.

Defining context types

Every system package must have a module named Context at the top level. The module defines a Context type and an UnsafeNew function that returns a Context value with no argument.

For example, a system package for command line applications might have the following Context module:

...

type Context {
  print \(string) none
}

UnsafeNew = \() Context {
  Context{
    print: ...
  }
}

The language's compiler uses these type and function to compose a context type passed to main functions in main modules in application packages.

Providing system functions and types

System packages are the only places where you can define functions that have side effects. Thanks to system packages, applications can perform effects to interact with the world, such as:

  • Console input/output
  • File system operations
  • Networking
  • Random number generation

Note that system packages should never expose side effects directly through their functions; all public functions in system packages must be purely functional. Instead, you need to pass a Context type to every effect-ful function for it to make side effects.

For example, a system package for command line applications might have the following types and functions:

# Define a foreign function to output a string in console.
import foreign _pen_cli_print \(string) none

type Context {
  print: _pen_cli_print,
}

Print = \(ctx Context, s string) none {
  ctx.print(s)
}

rather than:

import foreign _pen_cli_print \(string) none

Print = \(s string) none {
  # Oh, no! We make side effects in a public function directly.
  _pen_cli_print(s)
}

Linking application files (optional)

System packages might have optional script files named pen-link at their top directories. On every build of application packages using the system packages, the script files run given object files specified as command line arguments to link application files. The script files may or may not have file extensions.

The scripts should accept the following command line arguments.

ArgumentRequiredDescription
-t <target>NoTarget triple
-o <application>YesPath of an application file
<archive>...YesPaths of archive files sorted topologically from main packages

At the liking phase, compiled main functions are available under a symbol named _pen_main with the language's native calling convention.

Examples

The Os standard package is an example of system packages.

Roadmap

Items are ordered by priority.

  • Basic syntax
  • CPS transformation
  • Capability-based effect system
  • Performant GC
    • Automatic reference counting
  • Foreign function interface
  • Basic OS interface
  • WASM backend
  • Stream-based list type
  • Testing framework
  • TCP/UDP sockets
  • System time
  • Asynchronous operations
  • Parallel computation
  • Binary operations
  • List comprehension
  • Multiple system packages
  • Map type
  • Code formatter
  • Documentation generator
  • IDE/editor support
    • Language server
  • Reflection
  • Serialization / deserialization
  • Process management
  • Mutable state
    • Thread safety
  • Vector type
  • Web browser interface
  • Self-hosting!?

The Zen

  • Integrated values over spontaneous values
  • Beautiful is better than ugly.
  • Explicit is better than implicit.
  • Simple is better than complex.
  • One way to do one thing
  • Single solution to solve multiple problems

Language qualities

In the order of priority,

  • Simplicity
  • Readability
  • Consistency
  • Writability

References