Pen programming language

Pen is the parallel, concurrent, and functional programming language for scalable software development, focused on software maintainability and portability.

import Core'Number
import Os'File

# The `\` prefix for λ denotes a function.
findAnswer = \(kind string) number {
  # Secret source...

  21
}

main = \(ctx context) none {
  # The `go` function runs a given function in parallel.
  # `x` is a future for the computed value.
  x = go(\() number { findAnswer("humanity") })
  y = findAnswer("dolphins")

  _ = File'Write(ctx, File'StdOut(), Number'String(x() + y))

  none
}

Vision

Pen aims to make large-scale software development efficient where many engineers develop software together for a long time. To realize that, it focuses on software maintainability and portability.

• Maintainability
  • Simplicity: The language is small and easy to learn yet full featured.
  • Testability: Tests are always fast and reliable.
  • Flexibility: Developers can change codes easily without regression.
• Portability
  • Programs written in Pen can be ported to different platforms including WebAssembly.

Features

Minimal language

• Its syntax and type system are small, simple, and easy to learn.
• Yet, the language supports all the modern features.

Concurrent/parallel computation

• The language and its runtime enables thread-safe concurrent/parallel computation.
• For more information, see Concurrency and parallelism.

Reliable testing

• Tests are always deterministic and fast.
• Tests are side-effect free and independent from test environment.

No built-in system library

• There is no built-in system library dependent on platforms.
• Developers choose system packages suitable for their applications.
• System packages encapsulate platform-dependent codes and side effects.
• No other kind of package causes side effects without explicit injection.

Security

• No runtime exception
• No undefined behavior
• No data race

Even more...

• Static typing
• Immutable values
• Pure functions by default
• Errors as values
• Asynchronous I/O
• Cross compile
• Rust/C Foreign Function Interface (FFI)

License

Pen is dual-licensed under MIT and Apache 2.0.

Install

Via Homebrew

On Linux, macOS and WSL, you can install Pen through Homebrew.

1. Install Homebrew.

2. Run the following command in your terminal.

   brew install pen-lang/pen/pen
   

Now, you should be able to run a pen command in your shell.

pen --version

Building from source

You can also build Pen from source on your local machine.

1. Install the following software using a package manager of your choice (e.g. apt for Ubuntu and Homebrew for macOS.)

   • Rust
   • LLVM 14
   • Git
   • Ninja

2. Clone the Git repository.

   git clone https://github.com/pen-lang/pen
   

3. Run a cargo command in the repository's directory.

   cargo install --path cmd/pen
   

4. Set a PEN_ROOT environment variable to the directory.

   export PEN_ROOT=<directory>
   

Now, you are ready to use the pen command built manually.

Getting started

Install

See Install.

Creating a package

To create your first package, run a pen create command with your package's name in your terminal.

pen create foo

Then, you should see a directory named foo in your current directory. When you go into the directory, you should see a main.pen source file and a pen.json file for package configuration.

Building a package

To build the package, run a pen build command in the foo directory.

pen build

Then, you will see an executable file named app in the directory. Run it to see your first "Hello, world!"

./app # -> Hello, world!

Now, you can start editing source files and build your own application in Pen!

Next steps

• To use other library packages, see Using a library.
• For more code examples, see Examples.
• For the language syntax, see Syntax and Types.
• For usage of the standard packages, see Standard packages.
• To add more modules in your package, see Modules.

Guides

This chapter contains how-to guides for software development in Pen.

Building an executable

This page describes how to build an executable of a program written in Pen. It consists of the following steps:

1. Create an application package.
2. Build the package into an executable.

Creating an application package

Application packages are packages that are built into executables. To create it, you run a pen create command with your application's name (e.g. foo) in your terminal.

pen create foo

Then, you should see a foo directory in your current directory. When you go there, you should see a main.pen source file and a pen.json file for package configuration.

main.pen:

import Os'File

main = \(ctx context) none {
  _ = File'Write(ctx.Os, File'StdOut(), "Hello, world!\n")

  none
}

pen.json:

{
  "type": "application",
  "dependencies": {
    "Os": "pen:///os"
  }
}

In this example, the main.pen file contains a program that outputs a text, "Hello, world!" in a terminal. And the pen.json configuration file defines a package type of application and its dependencies. Here, it has only a dependency of the Os system package.

Building a package into an executable

To build the package, you run a pen build command in the package's directory.

pen build

Then, you will see an executable file named app in the directory. Now, you can run it to see its output, "Hello, world!"

./app # -> Hello, world!

Next steps

• Creating a library
• Using a library
• Language syntax

Creating a library

This page describes how to create a library in Pen. It consists of the following steps:

1. Create a library package.
2. Publish the package.

Creating a library package

Library packages are packages imported and used by other packages. To create it, you run a pen create --library command with your library's name (e.g. foo) in your terminal.

pen create --library foo

Then, you should see a foo directory in your current directory. When you go there, you should see a Foo.pen source file and a pen.json file for package configuration.

Foo.pen:

Add = \(x number, y number) number {
  x + y
}

pen.json:

{
  "type": "library",
  "dependencies": {}
}

In this example, the Foo.pen file contains an Add function that adds two numbers. And the pen.json configuration file defines a package type of library and its dependencies of none.

Publishing a package

The easiest way to publish your library package is to push the package as a Git repository onto one of Git repository hosting services, such as GitHub.

git add .
git commit
git remote add origin ssh://git@github.com/your-name/foo
git push

Now, your package is ready for use by other packages!

Next steps

• Building an executable
• Using a library
• Language syntax

Using a library

This page describes how to use a library in Pen. It consists of the following steps:

1. Add a library package as a dependency in another package.
2. Import functions and types from the library package.

Modifying package configuration

To use a library package, you need to add the package as a dependency in another package. To add the dependency, you modify a pen.json configuration file in the package adding the library package's name (e.g. Foo) and URL (e.g. git://github.com/your-name/foo) in a dependencies field like the following example. Note that you need to specify a git protocol scheme for library packages published as Git repositories. For other kinds of library packages, see Package configuration.

{
  "type": "application", // This can be any type.
  "dependencies": {
    "Foo": "git://github.com/your-name/foo"
  }
}

Importing functions and types from a library package

To import functions and types from the library package, you use import statements in a source file of your module with a name of the library package (e.g. Foo) and a module name (e.g. Math) where functions or types you want to use are defined.

import Foo'Math

Then, you are ready to use those functions and types with a prefix of the module name! For example, to call a function named Add in the Math module, you can write Math'Add(x, y).

type MyType = Math'Order

MyFunction = \(x number, y number) number {
  Math'Add(x, y)
}

Next steps

• Building an executable
• Creating a library
• Language syntax
  • Import statement

Testing

This page describes how to write and run unit tests for programs written in Pen.

Testing codes consists of the following steps:

1. Write tests as test functions in test modules.
2. Run the tests with a pen test command.

Writing tests

You can write tests as test functions in test modules. All modules with the .test.pen file extension are test modules. And, all public functions in test modules are test functions. The test functions need to have a type of \() none | error and should return error values when they fail.

For example, to test a Foo function in a Foo.pen module, write a Foo.test.pen test module with the following contents.

import Test'Assert
import 'Foo

CheckFoo = \() none | error {
  Assert'True(Foo'Foo() == 42)
}

The Test package

The Test standard package includes some utilities which helps you to write tests.

Running tests

To run tests, you can run a pen test command in your package's directory. Then, you should see test results of test functions in test modules. The pen test command exits with a non-zero status code if some tests fail.

Concurrency and parallelism

Concurrent programs use CPU time efficiently without being blocked on I/O or data synchronization. Parallel programs leverage multi-core CPUs to compute things in parallel faster than sequential programs.

This page describes how to write concurrent and/or parallel programs in Pen.

Built-ins

Pen provides several built-in functions for concurrent and parallel programming.

go function

The go built-in function runs a given function concurrently, and possibly in parallel.

future = go(\() number {
  computeExpensive(x, y, z)
})

The go function returns a function of the same type as the given argument. The returned function returns a resulting value of the function execution. In other languages, such functions returning values computed concurrently when they are ready are also known as futures or promises.

The go function may or may not run a given function immediately depending on its implementation. For example, the standard Os system package runs the given function in parallel if multiple CPU cores are available.

race function

The race built-in function takes multiple lists and merge them into one by evaluating elements in each list concurrently and possibly in parallel. The resulting list contains the elements in the original lists in order of their finished times of computation. Remember that elements in lists are evaluated lazily.

zs = race([[number] xs, ys])

This functionality is similar to concurrent queues in other imperative languages, such as channels in Go. Input lists to the race function correspond to producers of elements into the queue, and a consumer of the queue is codes that use elements in the output list.

Patterns

Task parallelism

The go function can run different codes concurrently. For example, the following code runs the functions, computeA and computeB concurrently. Runtimes of applications might execute those functions even in parallel if their system packages allow that.

compute = \(x number, y number) number {
  z = go(\() number { computeA(x) })
  v = computeB(y)

  v + z
}

Data parallelism

To run the same computation against many values of the same type, you can use recursion and the go function.

computeMany = \(xs [number]) [number] {
  if [x, ...xs] = xs {
    y = go(\() number { foo(x()) })
    ys = computeMany(xs)

    [number y(), ...ys]
  } else {
    [number]
  }
}

The example above computes things in order of elements in the original list. However, you might want to see output values of concurrent computation in order of their finished times. By doing that, you can start using the output values as fast as possible without waiting for all computation to be completed. In this case, you can use the race function to reorder elements in the output list by their finished times.

compute = \(xs [number]) [number] {
  race([[number] [number x()] for x in computeMany(xs)])
}

If you want to evaluate elements in multiple lists concurrently, you can simply pass the lists as an argument to the race function. Note that elements in the same lists are not evaluated concurrently although elements in different lists are evaluated concurrently.

compute = \(xs [number], ys [number]) [number] {
  race([[number] computeMany(xs), computeMany(ys)])
}

Coding style

This page describes the common coding style of programs written in Pen.

Spacing

Use 2 space characters for indentation.

Foo = \(x number) number {
  if x == 0 {
    "Succeeded!"
  } else {
    "Failed..."
  }
}

Naming convention

Naming is important to keep codes consistent. The language currently has the following naming conventions.

Global and local names

You should use descriptive names for global functions and types. But, on the other hand, you are encouraged to use abbreviated names for local variables as long as that doesn't incur ambiguity. For example, you might use the following abbreviated names:

• i for index
• c for requestCount
• sys for system
• ctx for context

Acronyms

Acronyms are treated as single words.

• Cpu
• Ast

References

This chapter details language features, command line tools, and standard packages.

Language

This chapter describes the Pen's syntax, and type, module and package systems.

Syntax

This page describes syntax of Pen. You can compose programs building up those language constructs. See also Types about syntax for specific data types.

Module

Modules are sets of type and function definitions. Syntactically, a module consists of statements. See Modules about how modules themselves interact with each other.

Statements

Statements are constructs that declare functions and types in modules.

Import statement

It imports types and functions from another module from the same or another package.

See Modules for more details.

import Foo'Bar

Foreign import statement

It imports a function from a foreign language.

See Foreign Function Interface (FFI) for more details.

import foreign "c" foo \(number, number) number

Record type definition

It defines a record type.

See Records for more details.

type foo {
  bar number
  baz string
}

Type alias

It gives another name to a type.

type foo = number | none

Function definition

It defines a function with a given name. The right-hand side of = signs must be function expressions.

foo = \(x number, y number) number {
  x + y
}

Foreign function definition

It defines a function exported to foreign languages.

See Foreign Function Interface (FFI) for more details.

foreign "c" foo = \(x number, y number) number {
  x + y
}

Block

A block consists of 1 or more expressions wrapped in { and }. Values of the last expressions are treated as resulting values of the blocks.

{
  foo(ctx, z)

  x + y + z
}

If you want to keep values of intermediate expressions for later use, you can define variables putting their names and = operators in front of the expressions.

{
  x = 42

  ...
}

Expressions

Expressions represent some computation. Expressions can be nested; expressions often contain other expressions inside.

Function call

It calls a function to evaluate it with given arguments returning its result value.

f(x, y)

Operators

Arithmetic

Arithmetic operators add, subtract, multiply, or divide a number with another.

1 + 1
1 - 1
1 * 1
1 / 1

Comparison

Equality

Equal (==) and not-equal (!=) operators compare two values and return a boolean value indicating if they are equal or not.

1 == 1
1 != 1

The operators can compare any types except functions and types containing them.

"foo" == "bar"
foo{x: 0} == foo{x: 1}
42 != none

Ordering

Order operators compare two numbers and return a boolean value indicating if the condition is correct or not.

1 < 1
1 <= 1
1 > 1
1 >= 1

Boolean

A not operator flips a boolean value.

!true

An and operator returns true if both operands are true, or false otherwise.

true & false

An or operator returns true if either operand is true, or false otherwise.

true | false

Error handling

? suffix operators immediately exit the current functions with operands if they are of the error type. Both the operands and result values of functions where the operators are used must be a union type containing the error type.

x?

Function

It creates a function.

First, functions declare their argument names and types (x number and y number) and their result types (number). After that, function bodies of blocks describe how the functions compute result values.

\(x number, y number) number {
  x + y
}

Conditionals

If expression

It evaluates one of blocks depending on a condition of an expression of a boolean type.

• It evaluates the first block if a given boolean value is true.
• Otherwise, it evaluates the second block.

if x {
  ...
} else {
  ...
}

If-type expression

It evaluates one of blocks depending on the type of a given expression. The expression (foo()) needs to be bound to a variable (x) and, in each block, the variable is treated as its specified type.

if x = foo() as number {
  ...
} else if string | none {
  ...
} else {
  ...
}

If-list expression

It deconstructs a list and evaluates one of two blocks depending on if the list is empty or not.

• If a given list has 1 or more element, it evaluates the first block with a function that returns its first element (x) and rest of elements as a list (xs).
• If the list has no element, it evaluates the second block.

if [x, ...xs] = ... {
  ...
} else {
  ...
}

If-map expression

It gets a value for a key in a map and evaluates one of two blocks depending on if the map has the key or not.

• If a value for a key (key) is found, it evaluates the first block with the value (value).
• If the map has no such key, it evaluates the second block.

if value = xs[key] {
  ...
} else {
  ...
}

Loop

List comprehension

It iterates over elements in a given list and generates a new list with elements of a given expression.

[number f(x) for x in xs]

You can also iterate keys and values in a map.

[number f(key, value) for key, value in map]

Comment

Comments start with # and end with new-line characters.

# This is a comment.

Types

This page describes different data types in Pen.

Number

It represents a real number. It is implemented as a 64-bit floating point number of IEEE 754.

number

Literals

3.14
-42

Boolean

It is a boolean value of true or false.

boolean

Literals

true
false

None

It represents a missing value. It has only a single value of none.

none

Literals

none

String

It is a byte array.

string

Literals

String literals are sequences of bytes. They are often used to represent texts encoded in UTF-8.

"foo"

Escape sequences

String literals can contain the following escape sequences.

Functions

A function represents reusable computation with arguments and a result.

Functions represent not only pure computation but may also execute side effects, such as I/O.

\(number, number) number

Literals

\(x number, y number) number {
  x + y
}

Lists

It is a list of values. Its type contains an element type between [ and ].

[number]

Literals

A list literal contains its element type and elements as expressions.

[number]
[number 1]
[number 1, 2, 3]

You can create new lists from existing ones by spreading elements of the old ones prefixed by ... into the new ones.

[number x, ...xs]

Note that expressions within list literals are evaluated lazily; they are evaluated only if their values are required.

Maps

It is a map from keys to values. Its type contains key and value types between { and } separated by :.

{string: number}

Literals

A map literal contains its key and value types and key-value pairs as expressions.

{string: number}
{string: number "foo": 1}
{string: number "foo": 1, "bar": 2}

You can create new maps from existing ones by spreading entries of the old ones prefixed by ... into the new ones.

{string: number ...map, "foo": 1}

You can also delete a key from a map omitting its value.

{string: number ...map, "foo"}

Records

It combines multiple types into a single type. Each field of a record type is composed of its name and type.

Fields are not accessible outside modules where they are defined by default.

type person {
  name string
  age number
}

To expose fields as well as the type itself to other modules, you need to capitalize their names.

type Person {
  Name string
  Age number
}

Literals

Record values are constructed using record literals containing their field names and values separated by commas.

person{name: "foo", age: 42}

You can also create new records from existing ones spreading fields of the old ones into the new ones.

person{...one, name: "bar"}

You can access field values by appending their names with . prefixes to expressions of record types.

john.name

Unions

It is a union of multiple types.

For example, the type below represents values that can be either number or none.

number | none

Any

Literally, it's an any type. Any values can be converted to the type.

any

Error

It is an error. You can create error values by calling the error built-in function. See also Error handling.

error

Built-ins

Built-in types and functions are ones implicitly defined in every module.

Types

See Types.

Functions

size

It calculates a size of a list or a map. It is generic and you can apply it to any list and map types.

Its time complexity is O(n) for lists and O(1) for maps.

\(list [a]) number
\(map {a: b}) number

error

It creates an error with its source information.

\(s any) error

source

It extracts source information from an error.

\(e error) any

debug

It prints a debug message given as an argument.

Note that behavior of this function can change among system packages. You may not even see any messages with system packages whose systems do not have any consoles.

\(message string) none

go

It executes a function concurrently. Its return value is a future represented as a function that returns a result of the executed function.

\(\() a) \() a

race

It merges multiple lists into one by evaluating elements in the lists concurrently. Elements in each list are evaluated sequentially in order.

This function corresponds to the fan-in concurrency pattern in other languages where results of concurrent computation in multiple queues are merged into a queue.

\([[a]]) [a]

Modules

Modules are sets of functions and types. Using modules, you can split large programs into smaller chunks.

Each source file suffixed with a .pen file extension composes a module. Modules can import functions and types from other modules.

Exporting functions and types from modules

You can name functions and types in an upper camel case for them to be accessible from other modules.

type Foo {
  ...
}

type Bar = ...

Foo = \() number {
  ...
}

Importing functions and types from modules

In order to import functions and types from other modules, place import statements at the top of modules.

The first components in the statements are names of external packages you declare in package configuration files (Foo.) They are omitted if imported modules are in the same packages. The rest of the components are directory names where the modules exist (Bar) and the modules' filenames without their file extensions (Baz for Baz.pen.)

import Foo'Bar'Baz

Then, you can access exported members of the modules with their prefixes.

type foo = Baz'Type

bar = \(x number) number {
  Baz'Function(x)
}

Module names

Modules in the same package

Modules in the same package are referenced by their paths relative to a root directory of the package.

For example, a module of a file at <package directory>/Foo/Bar.pen is imported as below.

import 'Foo'Bar

Modules in other packages

Modules in other packages are referenced by their package names defined in package configuration files and module paths.

For example, a module of a file at <package directory>/Bar/Baz.pen in a package Foo is imported as below.

import Foo'Bar'Baz

Private modules

For modules to be private and not accessible from other packages, you can name them in lower camel case (e.g. fooBar.)

Custom prefixes

Imported modules can have custom prefixes given different names after the as keywords.

import Foo'Bar'Baz as Blah

Unqualified import

You can import functions and types without prefixes by putting their names between { and } in import statements. This is especially useful when module names and imported functions or types have the same names like import 'MyType { MyType }.

import Foo'Bar { Foo, Bar }

type Baz {
  foo Foo
}

Blah = \() number {
  Bar()
}

Packages

Packages are sets of modules. Like modules, packages can import other packages specifying them in their configurations.

What composes a package?

The following entities compose packages.

• Standard packages bundled in installation of the language
• Remote repositories managed by version control systems (VCS)
  • Currently, Pen supports only Git as a VCS.
• Directories with package configuration files on file systems

During builds of packages, Pen's build system automatically download and initialize their dependency packages based on their URLs.

Package types

There are 3 package types: application, library, and system. Those types are specified in package configuration files.

Application packages

Application packages build applications often as executable files. Every application package must have a main.pen module file at its top directory. The main module has a main function that receives an argument of a context type and returns a none type. The context type is a record type containing context values of system packages with their field names of package names. For example, given system packages named Http and Os, a main function looks like the following.

main = \(ctx context) none {
  s = fetch(ctx.Http, "https://pen-lang.org/")
  print(ctx.Os, s)

  none
}

Every application package must specify one and only one system package that links applications (e.g. the Os standard system package) in its package configuration file. However, application packages can specify system packages that do not link applications (e.g. the Http system package in the example above) as many as possible.

Library packages

Library packages contain functions and types that have no side effects. They are imported and used by other packages.

System packages

System packages contain functions and types that have side effects to provide system interfaces to application packages. The language currently provides the two standard system packages of Os and OsSync.

Although they can be imported by library packages as well as application packages, then they are expected not to cause any side effects.

If you want to write your own system packages, see Writing system packages.

Package configuration

Each package has its configuration file named pen.json in a JSON format at its top directory. The JSON file has the following fields.

Package URLs have different protocol schemes depending on where they are located.

• Standard packages: pen
• Git repositories: git
• Directories on file systems: none

Examples

Application

{
  "type": "application",
  "dependencies": {
    "Os": "pen:///os",
    "Core": "pen:///core",
    "Foo": "git://github.com/foo/foo",
    "Bar": "../bar"
  }
}

Library

{
  "type": "library",
  "dependencies": {
    "Core": "pen:///core",
    "Foo": "git://github.com/foo/foo",
    "Bar": "../bar"
  }
}

System

{
  "type": "system",
  "dependencies": {
    "Core": "pen:///core",
    "Foo": "git://github.com/foo/foo",
    "Bar": "../bar"
  }
}

Command line tools

The pen command has the following sub-commands.

For more information, see its help message by running pen --help.

build command

It builds a package in the current directory.

pen build

create command

It creates a package of a given kind in a specified directory.

Creating an application package

pen create foo

Creating a library package

pen create --library foo

Standard packages

Standard packages provide common utility functions and system interfaces. They are installed together with the pen command on your system. This chapter describes their APIs and how to use them.

Core package

This package provides common algorithms and data structures.

{
  "type": "library"
}

Install

{
  "dependencies": {
    "Core": "pen:///core"
  } 
}

Core'Bit module

This module provides bitwise operations.

Most functions defined in this module take arguments of 64-bit integers. They can be converted from and into integers represented in IEEE-754 of a number type.

Types

No types are defined.

Functions

And

Calculate bitwise "and" given two 64-bit integers.

\(x number, y number) number

Or

Calculate bitwise "or" given two 64-bit integers.

\(x number, y number) number

Xor

Calculate bitwise exclusive-"or" given two 64-bit integers.

\(x number, y number) number

Not

Calculate bitwise "not" given two 64-bit integers.

\(x number) number

LeftShift

Calculate unsigned left shift given a 64-bit integer.

\(x number, n number) number

RightShift

Calculate unsigned right shift given a 64-bit integer.

\(x number, n number) number

ToInteger64

Convert an integer in IEEE-754 to a 64-bit integer.

\(x number) number

FromInteger64

Convert a 64-bit integer to an integer in IEEE-754.

\(x number) number

Core'Boolean module

This module provides common boolean operations.

Types

No types are defined.

Functions

Any

Return true if any of given booleans are true or false otherwise.

\(bs [boolean]) boolean

All

Return true if all of given booleans are true or false otherwise.

\(bs [boolean]) boolean

Core'Character module

Types

No types are defined.

Functions

FromCodePoint

\(n number) string

ToCodePoint

\(s string) number

Core'List module

This module provides common list operations.

Types

No types are defined.

Functions

First

Get the first element in a list. If a list is empty, it returns a fallback value.

\(xs [any], fallback any) any

Last

Get the last element in a list. If a list is empty, it returns a fallback value.

\(xs [any], fallback any) any

ToNumbers

Convert a list of an any type to one of a number type skipping non-number types.

\(xs [any]) [number]

ToStrings

Convert a list of an any type to one of a string type skipping non-string types.

\(xs [any]) [string]

ToBooleans

Convert a list of an any type to one of a boolean type skipping non-boolean types.

\(xs [any]) [boolean]

Core'Number module

This module provides common number operations.

Types

No types are defined.

Functions

Absolute

Calculate an absolute value.

\(x number) number

Ceil

Calculate a ceil value.

\(x number) number

Epsilon

Machine epsilon

\() number

Exponential

Calculate an exponential of a number.

\(x number) number

Floor

Calculate a floor value.

\(x number) number

Fraction

Calculate a fraction value of a number.

\(x number) number

Infinity

Infinity

\() number

IsNan

Check if a number is NaN.

\(x number) boolean

Maximum

Calculate a maximum.

\(x number, y number) number

Minimum

Calculate a minimum.

\(x number, y number) number

Nan

NaN

\() number

Parse

Parse a number.

\(s string) number | error

Power

Calculate a power.

\(base number, exponent number) number

Range

Create a list of numbers from a minimum to a maximum.

\(minimum number, maximum number) [number]

Remainder

Calculate a remainder.

\(dividend number, divisor number) number

Round

Round a number.

\(x number) number

Sequence

Create a list of numbers from 1 to a maximum value.

\(maximum number) [number]

SquareRoot

Calculate a square root.

\(x number) number

String

Convert a number into its string representation.

\(x number) string

Sum

Sum up numbers.

\(xs [number]) number

Truncate

Return an integer value of a number.

\(x number) number

Core'String module

This module provides common string operations.

Types

No types are defined.

Functions

Concatenate

Concatenate strings.

\(strings [string]) string

Join

Join strings with a separator.

\(strings [string], separator string) string

HasPrefix

Check if a string has a prefix.

\(s string, prefix string) boolean

Core'String'Byte module

This module provides operations for strings as byte arrays.

Types

No types are defined.

Functions

Length

Return a length of a string.

\(s string) number

Slice

Slice a string.

\(s string, start number, end number) string

Core'String'Byte'View module

This module provides views of strings as byte arrays. By using those views, you can apply operations to strings without copying them.

Types

View

A view of a string

type View {
  # ...
}

Functions

New

Create a view of a string.

\(s string) View

Viewee

Get an original string.

\(v View) string

Start

Get a start index of a view.

\(v View) number

End

Get an end index of a view.

\(v View) number

HasPrefix

Check if a string has a prefix.

\(v View, s string) boolean

Length

Get a length of a view.

\(v View) number

Seek

Move a start index forward.

\(v View, n number) View

Slice

Slice a view.

\(v View, start number, end number) View

ToString

Convert a view into a string.

\(v View) string

Core'String'Utf8 module

This module provides operations for strings encoded in UTF-8.

Types

No types are defined.

Functions

Contains

Check if a string contains a pattern.

\(s string, pat string) boolean

Find

Find an index for a pattern in a string.

\(s string, pat string) number | none

HasPrefix

Checks if a string has a prefix..

\(s string, pat string) boolean

HasSuffix

Checks if a string has a suffix..

\(s string, pat string) boolean

Length

Return a length of a string.

\(s string) number

Replace

Replace a pattern in a string.

\(s string, pattern string, replacement string) string

Slice

Slice a string.

\(s string, start number, end number) string

Split

Split a string by a pattern.

\(s string, pat string) [string]

ToLowercase

Convert a string into lowercase.

\(s string) string

ToUppercase

Convert a string into uppercase.

\(s string) string

Trim

Trim leading and trailing spaces.

\(s string) string

TrimEnd

Trim trailing spaces.

\(s string) string

TrimEndMatches

Trim trailing patterns.

\(s string, pat string) string

TrimMatches

Trim leading and trailing patterns.

\(s string, pat string) string

TrimStart

Trim leading spaces.

\(s string) string

TrimStartMatches

Trim leading patterns.

\(s string, pat string) string

Flag package

This package provides command-line flag parsing.

{
  "type": "library"
}

Install

{
  "dependencies": {
    "Flag": "pen:///flag"
  } 
}

Flag'Flag module

This module provides command-line flag parsing.

The flags have the following formats:

• -foo bar: a named flag of a key foo with a value bar
• foo: a positional flag of a value foo

Named flags and positional flags are separated by -- optionally.

Types

Set

A set of flags

type Set {
  Named {string: string}
  Positional [string]
}

Functions

Parse

Parse flags.

\(ss [string]) Set | error

Json package

This package provides a JSON parser.

{
  "type": "library"
}

Install

{
  "dependencies": {
    "Json": "pen:///json"
  } 
}

Json'Decode module

Types

No types are defined.

Functions

Decode

Decode a string into a JSON value.

\(s string) Value | error

Json'Encode module

Types

No types are defined.

Functions

Encode

Encode a JSON value.

\(v Value) string

Json'Value module

Types

Value

A JSON value

type Value {
  # ...
}

Raw

A raw JSON value represented by built-in types

type Raw = boolean | none | number | string | [Value] | {string: Value}

Functions

New

Create a JSON value.

\(r Raw) Value

Raw

Get a raw value.

\(v Value) Raw

Html package

This package provides HTML rendering logic.

{
  "type": "library"
}

Install

{
  "dependencies": {
    "Html": "pen:///html"
  } 
}

Html'Node module

Types

Node

An HTML node

type Node = Element | string

Element

An HTML element

type Element {
  Tag string
  Attributes [Attribute]
  Children [Node]
}

Attribute

An HTML attribute

type Attribute {
  Key string
  Value string
}

Functions

No functions are defined.

Html'Render module

Types

No types are defined.

Functions

Render

Render an HTML node.

\(n Node) string | error

Http package

This package provides HTTP client and server.

{
  "type": "system"
}

Install

{
  "dependencies": {
    "Http": "pen:///http"
  } 
}

Http'Client module

This module provides an HTTP client.

Types

No types are defined.

Functions

Send

Send an HTTP request.

\(ctx Context, r Request) Response | error

Http'Context module

Types

Context

An HTTP context

type Context = context'Context

Functions

No functions are defined.

Http'Request module

Types

Request

An HTTP request

type Request {
  Method string
  Uri string
  Headers {string: string}
  Body string
}

Functions

No functions are defined.

Http'Response module

Types

Response

An HTTP response

type Response {
  Status number
  Headers {string: string}
  Body string
}

Functions

No functions are defined.

Http'Server module

This module provides an HTTP server.

Types

No types are defined.

Functions

Serve

Run an HTTP service.

\(ctx Context, address string, callback \(Request) Response) none | error

Os package

This package provides an interface for operating systems.

{
  "type": "system"
}

Install

{
  "dependencies": {
    "Os": "pen:///os"
  } 
}

Os'Context module

Types

Context

A context of an operating system.

type Context = context'Context

Functions

No functions are defined.

Os'Directory module

Types

No types are defined.

Functions

Read

Read a directory and return file paths it contains.

\(ctx Context, path string) [string] | error

Create

Create a directory.

\(ctx Context, path string) none | error

Remove

Remove a directory.

\(ctx Context, path string) none | error

Os'Environment module

Types

No types are defined.

Functions

Arguments

Get command line arguments.

\(ctx Context) [string]

Variable

Get an environment variable.

\(ctx Context, name string) string | error

Os'File module

Types

File

A file on a file system.

type File {
  # ...
}

Functions

StdIn

A file of standard input.

\() File

StdOut

A file of standard output.

\() File

StdErr

A file of standard error.

\() File

Open

Open a file for read-only.

\(ctx Context, path string) File | error

OpenWithOptions

Open a file with options.

\(ctx Context, path string, opt OpenOptions) File | error

Read

Read a file.

\(ctx Context, file File) string | error

ReadLimit

Read a file until a size limit.

\(ctx Context, file File, limit number) string | error

Write

Write data to a file.

\(ctx Context, file File, data string) number | error

Copy

Copy a file to another path.

\(ctx Context, src string, dest string) none | error

Move

Move a file to another path.

\(ctx Context, src string, dest string) none | error

Remove

Remove a file.

\(ctx Context, path string) none | error

Metadata

Get metadata of a file at a path.

\(ctx Context, path string) Metadata | error

Os'File'Metadata module

Types

Metadata

File metadata

type Metadata {
  Size number
}

Functions

No functions are defined.

Os'File'OpenOptions module

Types

OpenOptions

Options to open a file

• Append allows appending data to the file.
• Create creates a new file if the file doesn't exist or opens it otherwise.
• CreateNew creates a new file. If the file already exists, it emits an error.
• Read allows reading data from the file.
• Truncate truncates the file to zero byte.
• Write allows writing data to the file.

type OpenOptions {
  Append boolean
  Create boolean
  CreateNew boolean
  Read boolean
  Truncate boolean
  Write boolean
}

Functions

Default

Get default options to open a file. They do not include any permission.

\() OpenOptions

Os'Process module

Types

No types are defined.

Functions

Exit

Exit a current process.

\(ctx Context, code number) none

Os'Tcp module

Types

No types are defined.

Functions

Bind

Create a listener bound to a server address.

\(ctx Context, address string) Listener | error

Connect

Create a stream connected to a peer address.

\(ctx Context, address string) Stream | error

Accept

Accept a client connection and create its stream.

\(ctx Context, l Listener) AcceptedStream | error

Receive

Receive data from a peer through a stream with a size limit in bytes.

\(ctx Context, s Stream, limit number) string | error

Send

Send data to a peer through a stream.

\(ctx Context, s Stream, data string) number | error

Os'Tcp'AcceptedStream module

Types

AcceptedStream

A TCP stream accepted on a server with a client address

type AcceptedStream {
  Stream Stream
  Address string
}

Functions

No functions are defined.

Os'Tcp'Listener module

Types

Listener

A TCP listener to listen for client connections

type Listener {
  # ...
}

Functions

No functions are defined.

Os'Tcp'Stream module

Types

Stream

A TCP stream

type Stream {
  # ...
}

Functions

No functions are defined.

Os'Time module

Types

No types are defined.

Functions

Now

Fetch a current system time in milliseconds.

\(ctx Context) number

Sleep

Pause a current execution context for a given amount of time.

\(ctx Context, milliseconds number) none

Os'Udp module

Types

No types are defined.

Functions

Bind

Bind a socket with a server address.

\(ctx Context, address string) Socket | error

Connect

Connect a socket to a peer address.

\(ctx Context, s Socket, address string) none | error

Receive

Receive a datagram from a connected address.

\(ctx Context, s Socket) string | error

ReceiveFrom

Receive a datagram from any address.

\(ctx Context, s Socket) Datagram | error

Send

Send a datagram to a connected address.

\(ctx Context, s Socket, data string) number | error

SendTo

Send a datagram to a specified address.

\(ctx Context, s Socket, data string, address string) number | error

Os'Udp'Datagram module

Types

Datagram

UDP datagram

type Datagram {
  Data string
  Address string
}

Functions

No functions are defined.

Os'Udp'Socket module

Types

Socket

UDP socket

type Socket {
  # ...
}

Functions

No functions are defined.

Random package

This package provides random number generation.

{
  "type": "system"
}

Install

{
  "dependencies": {
    "Random": "pen:///random"
  } 
}

Random'Context module

Types

Context

A context of random number generation.

type Context = context'Context

Functions

No functions are defined.

Random'Random module

Types

No types are defined.

Functions

Number

Generate a random number in a range of [0, 1).

\(ctx Context) number

Regex package

This package provides regular expressions.

{
  "type": "library"
}

Install

{
  "dependencies": {
    "Regex": "pen:///regex"
  } 
}

Regex'Expression module

This module provides functions to compile and match regular expressions.

Types

Expression

A regular expression.

type Expression {
  # ...
}

Functions

New

Compile a regular expression.

\(s string) Expression | error

IsMatch

Check if a regular expression matches with a string or not.

\(e Expression, s string) boolean

Match

Match a regular expression with a string and return matched groups.

\(e Expression, s string) [string | none] | none

Sql package

This package provides a SQL database client.

{
  "type": "system"
}

Install

{
  "dependencies": {
    "Sql": "pen:///sql"
  } 
}

Sql'Context module

Types

Context

A SQL database context

type Context = context'Context

Functions

No functions are defined.

Sql'Pool module

Types

Pool

A connection pool

type Pool = pool'Pool

Functions

New

Create a connection pool.

\(context Context, uri string, options Options) Pool | error

Query

Run a query and return its rows.

\(context Context, pool Pool, query string, arguments [Value]) [[Value]] | error

Execute

Run a query and return a number of affected rows.

\(context Context, pool Pool, query string, arguments [Value]) number | error

Sql'Pool'Options module

Types

Options

Connection pool options

type Options {
  MinConnections number
  MaxConnections number
  ConnectTimeout number
}

Functions

No functions are defined.

Sql'Value module

Types

Value

A value in a column

type Value = boolean | none | number | string

Functions

No functions are defined.

Test package

This package provides test utilities.

{
  "type": "library"
}

Install

{
  "dependencies": {
    "Test": "pen:///test"
  } 
}

Test'Assert module

This module provides a collection of assertion logic for testing.

Types

No types are defined.

Functions

True

Assert that a condition is true.

\(x boolean) none | error

Error

Check if a value is an error.

\(x any) none | error

Fail

Fail with an error immediately. This function is useful to make unreachable codes fail.

\() error

Examples

These are examples on how to write programs in Pen, and use its command line tools and standard packages.

See also our code repository for examples of full applications and libraries.

Types

These are examples on how to use types and their values in Pen.

Number

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Use a number literal

Given a file named "Foo.pen" with:

f = \() number {
  42
}

When I run pen build

Then the exit status should be 0.

Use arithmetic operators

Given a file named "Foo.pen" with:

f = \() number {
  1 + 2 - 3 * 4 / 5
}

When I run pen build

Then the exit status should be 0.

Use equality operators

Given a file named "Foo.pen" with:

f = \() boolean {
  0 == 0
}

g = \() boolean {
  0 != 0
}

When I run pen build

Then the exit status should be 0.

Use order operators

Given a file named "Foo.pen" with:

f = \() boolean {
  0 < 0
}

g = \() boolean {
  0 <= 0
}

h = \() boolean {
  0 > 0
}

i = \() boolean {
  0 >= 0
}

When I run pen build

Then the exit status should be 0.

Boolean

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Use boolean literals

Given a file named "Foo.pen" with:

f = \() boolean {
  true
}

g = \() boolean {
  false
}

When I run pen build

Then the exit status should be 0.

Use an and operation

Given a file named "Foo.pen" with:

f = \() boolean {
  true & false
}

When I run pen build

Then the exit status should be 0.

Use an or operation

Given a file named "Foo.pen" with:

f = \() boolean {
  true | false
}

When I run pen build

Then the exit status should be 0.

Use a not operation

Given a file named "Foo.pen" with:

f = \() boolean {
  !true
}

When I run pen build

Then the exit status should be 0.

Use an if expression

Given a file named "Foo.pen" with:

f = \() number {
  if true {
    1
  } else {
    0
  }
}

When I run pen build

Then the exit status should be 0.

None

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Use a none literal

Given a file named "Foo.pen" with:

f = \() none {
  none
}

When I run pen build

Then the exit status should be 0.

String

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Use a string literal

Given a file named "Foo.pen" with:

f = \() string {
  "foo"
}

When I run pen build

Then the exit status should be 0.

Use equality operators

Given a file named "Foo.pen" with:

f = \() boolean {
  "" == ""
}

g = \() boolean {
  "" != ""
}

When I run pen build

Then the exit status should be 0.

Function

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Define a function

Given a file named "Foo.pen" with:

f = \(x number) number {
  x
}

When I run pen build

Then the exit status should be 0.

Call a function with no argument

Given a file named "Foo.pen" with:

f = \() number {
  f()
}

When I run pen build

Then the exit status should be 0.

Call a function with an argument

Given a file named "Foo.pen" with:

f = \(x number) number {
  f(x)
}

When I run pen build

Then the exit status should be 0.

Call a function with two arguments

Given a file named "Foo.pen" with:

f = \(x number, y number) number {
  f(x, y)
}

When I run pen build

Then the exit status should be 0.

Define a closure

Given a file named "Foo.pen" with:

f = \(x number) \(number) number {
  \(y number) number {
    x + y
  }
}

When I run pen build

Then the exit status should be 0.

Record

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Create a record with a field

Given a file named "Foo.pen" with:

type r {
  x number
}

f = \() r {
  r{x: 42}
}

When I run pen build

Then the exit status should be 0.

Create a record with two fields

Given a file named "Foo.pen" with:

type r {
  x number
  y none
}

f = \() r {
  r{x: 42, y: none}
}

When I run pen build

Then the exit status should be 0.

Create a record with no field

Given a file named "Foo.pen" with:

type r {}

f = \() r {
  r{}
}

When I run pen build

Then the exit status should be 0.

Update a record

Given a file named "Foo.pen" with:

type r {
  x number
  y none
}

f = \(x r) r {
  r{...x, y: none}
}

When I run pen build

Then the exit status should be 0.

Get a field in a record

Given a file named "Foo.pen" with:

type r {
  x number
}

f = \(x r) number {
  x.x
}

When I run pen build

Then the exit status should be 0.

Use an equal operator

Given a file named "Foo.pen" with:

type r {
  x number
}

f = \(x r, y r) boolean {
  x == y
}

When I run pen build

Then the exit status should be 0.

Use a not-equal operator

Given a file named "Foo.pen" with:

type r {
  x number
}

f = \(x r, y r) boolean {
  x == y
}

When I run pen build

Then the exit status should be 0.

Propagate openness of a record

Given a file named "Foo.pen" with:

type Foo {
  X number
}

And a file named "Bar.pen" with:

import 'Foo

Bar = \() Foo'Foo {
  Foo'Foo{X: 42}
}

And a file named "Baz.pen" with:

import 'Bar

f = \() number {
  Bar'Bar().X
}

When I run pen build

Then the exit status should be 0.

Union

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Upcast a number into a union

Given a file named "Foo.pen" with:

f = \() number | none {
  42
}

When I run pen build

Then the exit status should be 0.

Upcast a function into a union

Given a file named "Foo.pen" with:

f = \() (\() number) | none {
  \() number {
    42
  }
}

When I run pen build

Then the exit status should be 0.

Upcast a list into a union

Given a file named "Foo.pen" with:

f = \() [number] | none {
  [number 42]
}

When I run pen build

Then the exit status should be 0.

Downcast a union type

Given a file named "Foo.pen" with:

f = \(x number | none) number {
  if x = x as number {
    x
  } else if none {
    0
  }
}

When I run pen build

Then the exit status should be 0.

Downcast a union type with an else block

Given a file named "Foo.pen" with:

f = \(x number | none) number {
  if x = x as none {
    0
  } else {
    x
  }
}

When I run pen build

Then the exit status should be 0.

Downcast a union type to another union type

Given a file named "Foo.pen" with:

f = \(x number | boolean | none) number | none {
  if x = x as number | none {
    x
  } else {
    none
  }
}

When I run pen build

Then the exit status should be 0.

List

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Create an empty list

Given a file named "Foo.pen" with:

f = \() [number] {
  [number]
}

When I run pen build

Then the exit status should be 0.

Create a list with an element

Given a file named "Foo.pen" with:

f = \() [number] {
  [number 1]
}

When I run pen build

Then the exit status should be 0.

Create a list with two elements

Given a file named "Foo.pen" with:

f = \() [number] {
  [number 1, 2]
}

When I run pen build

Then the exit status should be 0.

Join lists

Given a file named "Foo.pen" with:

f = \(xs [number]) [number] {
  [number ...xs, ...xs]
}

When I run pen build

Then the exit status should be 0.

Create a list of a union type

Given a file named "Foo.pen" with:

f = \() [number | none] {
  [number | none 1, none]
}

When I run pen build

Then the exit status should be 0.

Coerce elements of a spread list

Given a file named "Foo.pen" with:

f = \(xs [number]) [number | none] {
  [number | none ...xs]
}

When I run pen build

Then the exit status should be 0.

Use if-list expression

Given a file named "Foo.pen" with:

f = \(xs [number]) [number] {
  if [y, ...ys] = xs {
    [number y(), ...ys]
  } else {
    [number]
  }
}

When I run pen build

Then the exit status should be 0.

Use list comprehension

Given a file named "Foo.pen" with:

f = \(xs [number]) [number] {
  [number x() + 42 for x in xs]
}

When I run pen build

Then the exit status should be 0.

Get a size of a list

Given a file named "Foo.pen" with:

f = \(xs [none]) number {
  size(xs)
}

When I run pen build

Then the exit status should be 0.

Error

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Call an error function

Given a file named "Foo.pen" with:

f = \() error {
  error(none)
}

When I run pen build

Then the exit status should be 0.

Call a source function

Given a file named "Foo.pen" with:

f = \(e error) any {
  source(e)
}

When I run pen build

Then the exit status should be 0.

Use a try operator

Given a file named "Foo.pen" with:

f = \(x number | error) number | error {
  x? + 1
}

When I run pen build

Then the exit status should be 0.

Use a try operator with a union type

Given a file named "Foo.pen" with:

f = \(x number | none | error) number | error {
  if x = x? as number {
    x + 1
  } else if none {
    0
  }
}

When I run pen build

Then the exit status should be 0.

Any

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Use an any type

Given a file named "Foo.pen" with:

f = \() any {
  42
}

When I run pen build

Then the exit status should be 0.

Downcast an any type

Given a file named "Foo.pen" with:

f = \(x any) number {
  if x = x as number {
    x
  } else {
    0
  }
}

When I run pen build

Then the exit status should be 0.

Polymorphism

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Use an equal operator

Given a file named "Foo.pen" with:

f = \(x number | none) boolean {
  x == none
}

When I run pen build

Then the exit status should be 0.

Use a not-equal operator

Given a file named "Foo.pen" with:

f = \(x number | none) boolean {
  x != none
}

When I run pen build

Then the exit status should be 0.

Compare unions

Given a file named "Foo.pen" with:

f = \(x number | none, y number | none) boolean {
  x == y
}

When I run pen build

Then the exit status should be 0.

Compare a union and none

Given a file named "Foo.pen" with:

f = \(x number | none) boolean {
  x == none
}

When I run pen build

Then the exit status should be 0.

Syntax

These are examples on how to write programs in Pen.

Block

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Define a variable

Given a file named "Foo.pen" with:

f = \(x number) number {
  y = x

  y
}

When I run pen build

Then the exit status should be 0.

Call a function

Given a file named "Foo.pen" with:

f = \() none {
  none
}

g = \() none {
  f()

  none
}

When I run pen build

Then the exit status should be 0.

Use if expression

Given a file named "Foo.pen" with:

f = \() none {
  none
}

g = \() none {
  none
}

h = \(x boolean) none {
  if x {
    f()
  } else {
    g()
  }

  none
}

When I run pen build

Then the exit status should be 0.

Modules

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Import a function from a module

Given a file named "Foo.pen" with:

Foo = \() number {
  42
}

And a file named "Bar.pen" with:

import 'Foo

Bar = \() number {
  Foo'Foo()
}

When I run pen build

Then the exit status should be 0.

Import a type alias from a module

Given a file named "Foo.pen" with:

type Foo = number

And a file named "Bar.pen" with:

import 'Foo

type Bar = Foo'Foo

When I run pen build

Then the exit status should be 0.

Import a function from a nested module

Given a file named "Foo/Foo.pen" with:

Foo = \() number {
  42
}

And a file named "Bar.pen" with:

import 'Foo'Foo

Bar = \() number {
  Foo'Foo()
}

When I run pen build

Then the exit status should be 0.

Import a module with a custom prefix

Given a file named "Foo.pen" with:

Foo = \() number {
  42
}

And a file named "Bar.pen" with:

import 'Foo as Bar

Bar = \() number {
  Bar'Foo()
}

When I run pen build

Then the exit status should be 0.

Import a type definition with no prefix

Given a file named "Foo.pen" with:

type Foo {}

And a file named "Bar.pen" with:

import 'Foo { Foo }

type Bar = Foo

When I run pen build

Then the exit status should be 0.

Import a type alias with no prefix

Given a file named "Foo.pen" with:

type Foo = number

And a file named "Bar.pen" with:

import 'Foo { Foo }

type Bar = Foo

When I run pen build

Then the exit status should be 0.

Import a function with no prefix

Given a file named "Foo.pen" with:

Foo = \() number {
  42
}

And a file named "Bar.pen" with:

import 'Foo { Foo }

Bar = \() number {
  Foo()
}

When I run pen build

Then the exit status should be 0.

Packages

Background

Given a file named "foo/pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

And a file named "foo/Foo.pen" with:

type Foo = number

Foo = \() number {
  42
}

And a file named "foo/Foo/Foo.pen" with:

Foo = \() number {
  42
}

And a directory named "bar"

And I cd to "bar"

And a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {
    "Foo": "../foo"
  }
}

Import a function from a module

Given a file named "Bar.pen" with:

import Foo'Foo

Bar = \() number {
  Foo'Foo()
}

When I run pen build

Then the exit status should be 0.

Import a type from a module

Given a file named "Bar.pen" with:

import Foo'Foo

type Bar = Foo'Foo

When I run pen build

Then the exit status should be 0.

Import a function from a nested module

Given a file named "Bar.pen" with:

import Foo'Foo'Foo

Bar = \() number {
  Foo'Foo()
}

When I run pen build

Then the exit status should be 0.

Commands

These are examples on how to use command line tools of Pen.

Building packages

Build an application package

Given a file named "pen.json" with:

{
  "type": "application",
  "dependencies": {
    "Os": "pen:///os"
  }
}

And a file named "main.pen" with:

import Os'Context { Context }

main = \(ctx context) none {
  none
}

When I successfully run pen build

Then I successfully run ./app.

Build a library package

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

And a file named "Foo.pen" with:

f = \(x number) number {
  x
}

When I run pen build

Then the exit status should be 0.

Cross-build an application package

Given a file named "pen.json" with:

{
  "type": "application",
  "dependencies": {
    "Os": "pen:///os-sync"
  }
}

And a file named "main.pen" with:

import Os'Context { Context }

main = \(ctx context) none {
  none
}

And I successfully run rustup target add <target>

When I run pen build --target <target>

Then the exit status should be 0.

Examples

Cross-build a library package

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

And a file named "Foo.pen" with:

f = \(x number) number {
  x
}

And I successfully run rustup target add <target>

When I run pen build --target <target>

Then the exit status should be 0.

Examples

Creating packages

Create an application package

Given I successfully run pen create foo

And I cd to "foo"

When I successfully run pen build

Then I successfully run ./app.

Create a library package

Given I successfully run pen create --library foo

And I cd to "foo"

When I run pen build

Then the exit status should be 0.

Create an application package in a current directory

Given I successfully run pen create .

When I successfully run pen build

Then I successfully run ./app.

Create a library package in a current directory

Given I successfully run pen create --library .

When I run pen build

Then the exit status should be 0.

Testing packages

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {
    "Test": "pen:///test"
  }
}

And a file named "Foo.pen" with:

Add = \(x number, y number) number {
  x + y
}

Test a module

Given a file named "Foo.test.pen" with:

import Test'Assert
import 'Foo

Add = \() none | error {
  Assert'True(Foo'Add(41, 1) == 42)
}

When I run pen test

Then the exit status should be 0

And the stdout should contain "OK".

Fail to test a module

Given a file named "Foo.test.pen" with:

import Test'Assert
import 'Foo

Add = \() none | error {
  Assert'True(Foo'Add(41, 0) == 42)
}

When I run pen test

Then the exit status should not be 0

And the stdout should contain "FAIL".

Run multiple tests

Given a file named "Foo.test.pen" with:

import Test'Assert
import 'Foo

Add = \() none | error {
  Assert'True(Foo'Add(41, 1) == 42)
}

AddMore = \() none | error {
  Assert'True(Foo'Add(40, 2) == 42)
}

When I successfully run pen test

Then the exit status should be 0.

Run no test

When I run pen test

Then the exit status should be 0.

Use a debug function in a test

Given a file named "Foo.test.pen" with:

Foo = \() none | error {
  debug("hello")
}

When I run pen test

Then the exit status should be 0.

Formatting module files

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Format module files

Given a file named "Foo.pen" with:

Foo = \() none {

  none
}

When I successfully run pen format

Then a file named "Foo.pen" should contain exactly:

Foo = \() none {
  none
}

Check if module files are formatted

Given a file named "Foo.pen" with:

Foo = \() none {

  none
}

When I run pen format --checked

Then the exit status should not be 0.

Standard packages

These are examples on how to use standard packages.

Core

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {
    "Core": "pen:///core"
  }
}

Convert a number to a string

Given a file named "Foo.pen" with:

import Core'Number

f = \() string {
  Number'String(42)
}

When I run pen build

Then the exit status should be 0.

Sum numbers

Given a file named "Foo.pen" with:

import Core'Number

f = \() number {
  Number'Sum([number 1, 2, 3])
}

When I run pen build

Then the exit status should be 0.

Join strings

Given a file named "Foo.pen" with:

import Core'String

f = \() string {
  String'Join([string "hello", "world"], " ")
}

When I run pen build

Then the exit status should be 0.

Slice a string as bytes

Given a file named "Foo.pen" with:

import Core'String'Byte

f = \() string {
  Byte'Slice("foo", 1, 2)
}

When I run pen build

Then the exit status should be 0.

Os

Background

Given a file named "pen.json" with:

{
  "type": "application",
  "dependencies": {
    "Os": "pen:///os",
    "Core": "pen:///core"
  }
}

Build an application

Given a file named "main.pen" with:

import Os'Context { Context }
import Os'File
import Os'Process

main = \(ctx context) none {
  if _ = run(ctx.Os) as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

run = \(ctx Context) none | error {
  File'Write(ctx, File'StdOut(), "Hello, world!")?

  none
}

When I successfully run pen build

Then I successfully run ./app

And the stdout from "./app" should contain exactly "Hello, world!".

Get arguments

Given a file named "main.pen" with:

import Core'String
import Os'Context { Context }
import Os'Environment
import Os'File
import Os'Process

main = \(ctx context) none {
  if _ = run(ctx.Os) as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

run = \(ctx Context) none | error {
  File'Write(ctx, File'StdOut(), String'Join(Environment'Arguments(ctx), " "))?

  none
}

When I successfully run pen build

Then I successfully run ./app foo bar

And stdout from "./app foo bar" should contain exactly "foo bar".

Get an environment variable

Given a file named "main.pen" with:

import Core'String
import Os'Context { Context }
import Os'File
import Os'Environment
import Os'Process

main = \(ctx context) none {
  if _ = run(ctx.Os) as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

run = \(ctx Context) none | error {
  File'Write(ctx, File'StdOut(), Environment'Variable(ctx, "FOO")?)?

  none
}

And I append "foo" to the environment variable "FOO"

When I successfully run pen build

Then I successfully run ./app

And stdout from "./app" should contain exactly "foo".

Open a file

Given a file named "main.pen" with:

import Os'Context { Context }
import Os'File { File }
import Os'Process

main = \(ctx context) none {
  if _ = run(ctx.Os) as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

run = \(ctx Context) none | error {
  File'Open(ctx, "./foo.txt")?

  none
}

And a file named "foo.txt" with ""

When I successfully run pen build

Then I successfully run ./app.

Read a file

Given a file named "main.pen" with:

import Os'Context { Context }
import Os'File
import Os'File'OpenOptions
import Os'Process

main = \(ctx context) none {
  if _ = run(ctx.Os) as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

run = \(ctx Context) none | error {
  f = File'Open(ctx, "foo.txt")?
  d = File'Read(ctx, f)?

  File'Write(ctx, File'StdOut(), d)?

  none
}

And a file named "foo.txt" with "foo"

When I successfully run pen build

Then I successfully run ./app

And the stdout from "./app" should contain exactly "foo".

Read a file until a limit

Given a file named "main.pen" with:

import Os'Context { Context }
import Os'File
import Os'Process

main = \(ctx context) none {
  if _ = run(ctx.Os) as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

run = \(ctx Context) none | error {
  f = File'Open(ctx, "foo.txt")?
  d = File'ReadLimit(ctx, f, 5)?
  File'Write(ctx, File'StdOut(), d)?

  none
}

And a file named "foo.txt" with "Hello, world!"

When I successfully run pen build

Then I successfully run ./app

And the stdout from "./app" should contain exactly "Hello".

Write a file

Given a file named "main.pen" with:

import Os'Context { Context }
import Os'File
import Os'File'OpenOptions { OpenOptions }
import Os'Process

main = \(ctx context) none {
  if _ = run(ctx.Os) as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

run = \(ctx Context) none | error {
  f = File'OpenWithOptions(
    ctx,
    "./foo.txt",
    OpenOptions{...OpenOptions'Default(), Write: true},
  )?

  File'Write(ctx, f, "foo")?

  none
}

And a file named "foo.txt" with ""

When I successfully run pen build

Then I successfully run ./app

And the file "foo.txt" should contain "foo".

Copy a file

Given a file named "main.pen" with:

import Os'Context { Context }
import Os'File
import Os'Process

main = \(ctx context) none {
  if _ = File'Copy(ctx.Os, "foo.txt", "bar.txt") as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

And a file named "foo.txt" with "foo"

When I successfully run pen build

Then I successfully run ./app

And the file "foo.txt" should contain "foo"

And the file "bar.txt" should contain "foo".

Move a file

Given a file named "main.pen" with:

import Os'Context { Context }
import Os'File
import Os'Process

main = \(ctx context) none {
  if _ = File'Move(ctx.Os, "foo.txt", "bar.txt") as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

And a file named "foo.txt" with "foo"

When I successfully run pen build

Then I successfully run ./app

And the file "foo.txt" does not exist

And the file "bar.txt" should contain "foo".

Remove a file

Given a file named "main.pen" with:

import Os'Context { Context }
import Os'File
import Os'Process

main = \(ctx context) none {
  if _ = File'Remove(ctx.Os, "foo.txt") as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

And a file named "foo.txt" with ""

When I successfully run pen build

Then I successfully run ./app

And the file "foo.txt" does not exist.

Read a directory

Given a file named "main.pen" with:

import Core'String
import Os'Context { Context }
import Os'File
import Os'Directory
import Os'Process

main = \(ctx context) none {
  if _ = run(ctx.Os) as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

run = \(ctx Context) none | error {
  File'Write(
    ctx,
    File'StdOut(),
    String'Join(Directory'Read(ctx, ".")?, "\n"),
  )?

  none
}

When I successfully run pen build

Then I successfully run ./app

And the stdout from "./app" should contain "main.pen"

And the stdout from "./app" should contain "pen.json".

Create a directory

Given a file named "main.pen" with:

import Os'Context { Context }
import Os'Directory
import Os'Process

main = \(ctx context) none {
  if _ = Directory'Create(ctx.Os, "foo") as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

When I successfully run pen build

Then I successfully run ./app

And a directory named "foo" should exist.

Remove a directory

Given a file named "main.pen" with:

import Os'Context { Context }
import Os'Directory
import Os'Process

main = \(ctx context) none {
  if _ = Directory'Remove(ctx.Os, "foo") as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

And a directory named "foo"

When I successfully run pen build

Then I successfully run ./app

And a directory named "foo" should not exist.

Get file metadata

Given a file named "main.pen" with:

import Os'Context { Context }
import Os'File
import Os'File'Metadata { Metadata }
import Os'Process

main = \(ctx context) none {
  m = File'Metadata(ctx.Os, "foo")

  c = if m = m as Metadata {
    if m.Size == 3 {
      0
    } else {
      1
    }
  } else {
    1
  }

  Process'Exit(ctx.Os, c)
}

And a file named "foo" with:

foo

When I successfully run pen build

Then I successfully run ./app.

Get os time

Given a file named "main.pen" with:

import Core'Number
import Os'Context { Context }
import Os'File
import Os'Process
import Os'Time

main = \(ctx context) none {
  if m = run(ctx.Os) as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

run = \(ctx Context) none | error {
  File'Write(ctx, File'StdOut(), Number'String(Time'Now(ctx)))?

  none
}

When I successfully run pen build

Then I successfully run ./app.

Sleep

Given a file named "main.pen" with:

import Os'Context { Context }
import Os'Time

main = \(ctx context) none {
  Time'Sleep(ctx.Os, 1)
}

When I successfully run pen build

Then I successfully run ./app.

Exit a process

Given a file named "main.pen" with:

import Os'Context { Context }
import Os'Process

main = \(ctx context) none {
  Process'Exit(ctx.Os, 42)
}

When I successfully run pen build

Then I run ./app

And the exit status should be 42.

Random

Background

Given a file named "pen.json" with:

{
  "type": "application",
  "dependencies": {
    "Core": "pen:///core",
    "Os": "pen:///os",
    "Random": "pen:///random"
  }
}

Generate a random number

Given a file named "main.pen" with:

import Core'Number
import Os'Context { Context }
import Os'File
import Os'Process
import Random'Random

main = \(ctx context) none {
  if m = run(ctx) as none {
    none
  } else {
    Process'Exit(ctx.Os, 1)
  }
}

run = \(ctx context) none | error {
  File'Write(ctx.Os, File'StdOut(), Number'String(Random'Number(ctx.Random)))?

  none
}

When I successfully run pen build

Then I successfully run ./app.

Test

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {
    "Test": "pen:///test"
  }
}

Check if a condition is true

Given a file named "foo.test.pen" with:

import Test'Assert

Foo = \() none | error {
  Assert'True(true)
}

When I run pen test

Then the exit status should be 0.

Check if a value is an error

Given a file named "foo.test.pen" with:

import Test'Assert

Foo = \() none | error {
  Assert'Error(error(none))
}

When I run pen test

Then the exit status should be 0.

Make a test fail

Given a file named "foo.test.pen" with:

import Test'Assert

Foo = \() none | error {
  Assert'Fail()
}

When I run pen test

Then the exit status should be 1.

FFI

Background

Given a file named "pen.json" with:

{
  "type": "library",
  "dependencies": {}
}

Import a foreign function of native calling convention

Given a file named "Foo.pen" with:

import foreign g \(number) number

f = \(x number) number {
  g(x)
}

When I run pen build

Then the exit status should be 0.

Import a foreign function of the C calling convention

Given a file named "Foo.pen" with:

import foreign "c" g \(number) number

f = \(x number) number {
  g(x)
}

When I run pen build

Then the exit status should be 0.

Export a foreign function

Given a file named "Foo.pen" with:

foreign f = \(x number) number {
  x
}

When I run pen build

Then the exit status should be 0.

Export a foreign function of the C calling convention

Given a file named "Foo.pen" with:

foreign "c" f = \(x number) number {
  x
}

When I run pen build

Then the exit status should be 0.

Advanced features

This chapter lists advanced features of Pen and describe when and how to use them.

Cross compile

The language's compiler supports cross compile. To compile applications and libraries for different targets, specify the --target option of the pen build subcommand.

For example, run the following command to compile a wasm32 binary for the WASI platform.

pen build --target wasm32-wasi

Note that we currently support those targets via Rust's cross compiler toolchain. Please install a Rust compiler through rustup to enable installation of toolchains for different targets.

Supported targets

Run pen build --help to see all supported targets.

System package support

Cross compile support of system packages are totally up to their developers. For example, the Os standard system package supports most targets as long as their platforms expose OS-like APIs. However, some custom system packages might not support those targets because of their limited use cases.

Foreign Function Interface (FFI)

Using FFI, you can import or export functions in foreign languages, such as Rust and C.

Importing functions in foreign languages

You can import functions in foreign languages using foreign import statements. The statements specify the foreign functions' calling convention, names and types.

You might specify calling conventions of foreign functions in a format of string literals after import foreign keywords optionally. Currently, only the C calling convention is supported as "c". If the options are omitted, the functions are imported with the native calling convention of Pen.

import foreign "c" foo \(number, number) number

Caveat: You can import foreign functions that might cause side effects only in system packages. See also Writing system packages.

Exporting functions to foreign languages

You can export functions to foreign languages using foreign function definitions, which have foreign keywords in front of normal function definitions.

You might specify calling conventions of exported foreign functions optionally after foreign keywords as well as imported foreign functions.

foreign "c" foo = \(x number, y number) number {
  ...
}

Building libraries of foreign languages

During builds of your packages, you might want to build libraries of foreign languages so that you can use their functions. If that's your case, you can set up pen-ffi scripts in your packages. The script files run on every build and output absolute paths to .a archive files of your libraries in foreign languages built by the scripts. The script files may or may not have file extensions.

The pen-ffi scripts should accept the following command line arguments.

One of examples in practice is a pen-ffi.sh file in the Core library.

Native calling convention in Pen

TBD

Writing system packages

Using existing system packages covers most use cases in application development. However, by writing your own system packages, you can achieve the following:

• Define your own system interfaces of functions and types with side effects.
• Link applications in arbitrary file formats.

This page assumes that you have already read Packages.

Caveat: Providing bad system packages which do not conform to conventions can break the ecosystem of the language! In the worst cases, they might make applications malfunction. Please be careful to follow the conventions to keep applications maintainable and portable.

Functionalities of system packages

System packages have the following functionalities:

• Define context types.
• Provide system interfaces as functions and types.
• Link application files.

Defining context types

Every system package must have a module named Context at the top level. The module defines a Context type and an UnsafeNew function that returns a Context value with no argument.

For example, a system package for command line applications might have the following Context module:

...

type Context {
  print \(string) none
}

UnsafeNew = \() Context {
  Context{
    print: ...
  }
}

The language's compiler uses these type and function to compose a context type passed to main functions in main modules in application packages.

Providing system functions and types

System packages are the only places where you can define functions that have side effects. Thanks to system packages, applications can perform effects to interact with the world, such as:

• Console input/output
• File system operations
• Networking
• Random number generation

Note that system packages should never expose side effects directly through their functions; all public functions in system packages must be purely functional. Instead, you need to pass a Context type to every effect-ful function for it to make side effects.

For example, a system package for command line applications might have the following types and functions:

# Define a foreign function to output a string in console.
import foreign _pen_cli_print \(string) none

type Context {
  print: _pen_cli_print,
}

Print = \(ctx Context, s string) none {
  ctx.print(s)
}

rather than:

import foreign _pen_cli_print \(string) none

Print = \(s string) none {
  # Oh, no! We make side effects in a public function directly.
  _pen_cli_print(s)
}

Linking application files (optional)

System packages might have optional script files named pen-link at their top directories. On every build of application packages using the system packages, the script files run given object files specified as command line arguments to link application files. The script files may or may not have file extensions.

The scripts should accept the following command line arguments.

At the liking phase, compiled main functions are available under a symbol named _pen_main with Pen's native calling convention.

Examples

The Os standard package is an example of system packages.

Roadmap

Items are ordered by priority.

• Basic syntax
• CPS transformation
• Capability-based effect system
• Performant GC
  • Automatic reference counting
• Foreign function interface
• Basic OS interface
• WASM backend
• Stream-based list type
• Testing framework
• TCP/UDP sockets
• System time
• Asynchronous operations
• Parallel computation
• Binary operations
• List comprehension
• Multiple system packages
• Map type
• Code formatter
• Documentation generator
• IDE/editor support
  • Language server
• Reflection
• Serialization / deserialization
• Process management
• Mutable state
  • Thread safety
• Vector type
• Web browser interface

The Zen

• Beautiful is better than ugly.
• Explicit is better than implicit.
• Simple is better than complex.
• One way to do one thing
• Single solution for multiple problems
• Steady value over volatile value

Language qualities

In the order of priority,

• Simplicity
• Readability
• Consistency
• Writability

References

• The Zen of Python
• Zen | The Zig programming language