Development documentation

This section will collect how-to, notes and development documentation

Contributing

We use conventional commits and semantic versioning. Please follow the conventional commits specification when writing commit messages.

Creating a gRPC backend

LocalAI backends are gRPC servers.

In order to create a new backend you need:

• If there are changes required to the protobuf code, modify the proto file and re-generate the code with make protogen.
• Modify the Makefile to add your new backend and re-generate the client code with make protogen if necessary.
• Create a new gRPC server in extra/grpc if it’s not written in go: link, and create the specific implementation.
  • Golang gRPC servers should be added in the pkg/backend directory given their type. See piper as an example.
  • Golang servers needs a respective cmd/grpc binary that must be created too, see also cmd/grpc/piper as an example, update also the Makefile accordingly to build the binary during build time.
• Update the Dockerfile: if the backend is written in another language, update the Dockerfile default EXTERNAL_GRPC_BACKENDS variable by listing the new binary link.

Once you are done, you can either re-build LocalAI with your backend or you can try it out by running the gRPC server manually and specifying the host and IP to LocalAI with --external-grpc-backends or using (EXTERNAL_GRPC_BACKENDS environment variable, comma separated list of name:host:port tuples, e.g. my-awesome-backend:host:port):

./local-ai --debug --external-grpc-backends "my-awesome-backend:host:port" ...