Custom Dep Resources

Resources Compatible with versions 3.7.0 and Above

Starting with version 3.7.0, Rebar3 published a new API for custom resources, which gives access to the project’s local configuration to enable more powerful custom dependency formats. They can use contextual information from the current build to customize how dependencies might be fetched.

🚧

The new Interface is not backwards compatible

This new interface is unknown and unsupported in versions prior to 3.7.0. If you are writing libraries that should work with all Rebar3 copies, skip to the next section, where resources compatible with all Rebar3 versions are documented. Old interfaces however are still compatible with all versions and no support for existing project has been broken in adding the new API.

The new callback API is defined as follows:

%% Type declarations
-type resource() :: #resource{}. % an opaque record generated by an API call described below
-type source() :: {type(), location(), ref()} | {type(), location(), ref(), binary()}.
-type type() :: atom().
-type location() :: string().
-type ref() :: any().
-type resource_state() :: term().

%% and the following callbacks
-callback init(type(), rebar_state:t()) -> {ok, resource()}.
-callback lock(rebar_app_info:t(), resource_state()) -> source().
-callback download(file:filename_all(), rebar_app_info:t(), rebar_state:t(), resource_state()) ->
    ok | {error, any()}.
-callback needs_update(rebar_app_info:t(), resource_state()) -> boolean().
-callback make_vsn(rebar_app_info:t(), resource_state()) ->
    {plain, string()} | {error, string()}.
Erlang

The callbacks allow the resource plugin to have access to the rebar_state:t() data structure, which lets you access and manipulate the Rebar3 state, find application state, and generally work with the rebar_state, rebar_app_info, rebar_dir, and the new rebar_paths modules.

An example of a plugin making use of this functionality is rebar3_path_deps. Rebar3’s own hex package resource uses this API.

A resource plugin is initialized the same way as any other plugin would:

-module(my_rebar_plugin).

-export([init/1]).

-spec init(rebar_state:t()) -> {ok, rebar_state:t()}.
init(State) ->
    {ok, rebar_state:add_resource(State, {Tag, Module})}.
Erlang

Where Tag stands for the type in the deps configuration value (git, hg, etc.), and Module is the callback module.

The callback module may look as follows:

-module(my_rebar_plugin_resource).

-export([init/2,
         lock/2,
         download/4,
         needs_update/2,
         make_vsn/1]).

%% Initialize the custom dep resource plugin
init(Type, _RebarState) ->
   CustomState = #{},
   Resource = rebar_resource_v2:new(
       Type,         % type tag such as 'git' or 'hg'
       ?MODULE,      % this callback module
       CustomState   % anything you want to carry around for next calls
   ),
   {ok, Resource}.

lock(AppInfo, CustomState) ->
  %% Extract info such as {Type, ResourcePath, ...} as declared
  %% in rebar.config
  SourceTuple = rebar_app_info:source(AppInfo),
  %% Annotate and modify the source tuple to make it absolutely
  %% and indeniably unambiguous (for example, with git this means
  %% transforming a branch name into an immutable ref)
  ...
  %% Return the unambiguous source tuple
  ModifiedSource.

download(TmpDir, AppInfo, RebarState, CustomState) ->
  %% Extract info such as {Type, ResourcePath, ...} as declared
  %% in rebar.config
  SourceTuple = rebar_app_info:source(AppInfo)),
  %% Download the resource defined by SourceTuple, which should be
  %% an OTP application or library, into TmpDir
  ...
  ok.

make_vsn(Dir, CustomState) ->
  %% Extract a version number from the application. This is useful
  %% when defining the version in the .app.src file as `{version, Type}',
  %% which means it should be derived from the build information. For
  %% the `git' resource, this means looking for the last tag and adding
  %% commit-specific information
  ...
  {plain, "0.1.2"}.


needs_update(AppInfo, CustomState) ->
  %% Extract the Source tuple if needed
  SourceTuple = rebar_app_info:source(AppInfo),
  %% Base version in the current file
  OriginalVsn = rebar_app_info:original_vsn(AppInfo)
  %% Check if the copy in the current install matches
  %% the defined value in the source tuple. On a conflict,
  %% return `true', otherwise `false'
  ...,
    Bool.
Erlang

Resources Compatible with all versions

Prior to version 3.7.0, the dependency resource framework was a bit more restricted. It had to essentially work context-free, with only the deps information from the rebar.config and rebar.lock to work from. It was not possible to have any information relative to the project configuration, which essentially restricted what could be done by each resource.

These custom resources are still supported in Rebar3 versions higher than 3.7.0, and so if you have users running older build, we recommend that you develop this kind of resources only.

Each dependency resource must implement the rebar_resource behaviour.

-module(rebar_resource).

-export_type([resource/0
             ,type/0
             ,location/0
             ,ref/0]).

-type resource() :: {type(), location(), ref()}.
-type type() :: atom().
-type location() :: string().
-type ref() :: any().

-callback lock(file:filename_all(), tuple()) ->
    rebar_resource:resource().
-callback download(file:filename_all(), tuple(), rebar_state:t()) ->
    {tarball, file:filename_all()} | {ok, any()} | {error, any()}.
-callback needs_update(file:filename_all(), tuple()) ->
    boolean().
-callback make_vsn(file:filename_all()) ->
    {plain, string()} | {error, string()}.
Erlang

Included with rebar3 are rebar_git_resource, rebar_hg_resource and rebar_pkg_resource.

A custom resource can be included the same way as a plugin. An example of this can be seen in Kelly McLaughlin’s rebar3_tidy_deps resource:

-module(rebar_tidy_deps).

-export([init/1]).

-spec init(rebar_state:t()) -> {ok, rebar_state:t()}.
init(State) ->
    {ok, rebar_state:add_resource(State, {github, rebar_github_resource})}.
Erlang

This resource rebar_github_resource which implements the rebar3 resource behaviour is added to the list of resources available in rebar_state. Adding the repo as a plugin to rebar.config allows this resource to be used:

{mydep, {github, "kellymclauglin/mydep.git", {tag, "1.0.1"}}}.

{plugins, [
    {rebar_tidy_deps, ".*", {git, "https://github.com/kellymclaughlin/rebar3-tidy-deps-plugin.git", {tag, "0.0.2"}}}
]}.
Erlang

Writing a Plugin working with both versions

If you want to write a custom resource plugin that works with both versions, you can dynamically detect arguments to provide backwards-compatible functionality. In the example below, the new API disregards all new information and just plugs itself back in the old API:

-module(my_rebar_plugin_resource).

-export([init/2,
         lock/2,
         download/4, download/3,
         needs_update/2,
         make_vsn/1]).

init(Type, _RebarState) ->
   CustomState = #{},
   Resource = rebar_resource_v2:new(Type, ?MODULE, CustomState),
   {ok, Resource}.

%% Old API
lock(Dir, Source) when is_tuple(Source) ->
  lock_(Dir, Source);
%% New API
lock(AppInfo, _ResourceState) ->
  %%      extract info for dir               extract info for source
  lock_(rebar_app_info:dir(AppInfo), rebar_app_info:source(AppInfo)).

%% Function handling normalized case
lock_(Dir, Path) ->
  ...

%% Old Version
download(TmpDir, SourceTuple, RebarState) ->
  download_(TmpDir, SourceTuple, State).

%% New Version
download(TmpDir, AppInfo, RebarState, _ResourceState) ->
  %%                            extract source tuple
  download_(TmpDir, rebar_app_info:source(AppInfo), RebarState).

%% Function handling normalized case
download_(TmpDir, {MyTag, ...}, _State) ->
  ...

%% Old version
make_vsn(Dir) ->
  ...
%% New version
make_vsn(Dir, _ResourceState) ->
  make_vsn(Dir).

%% Old Version
needs_update(Dir, {MyTag, Path, _}) ->
  needs_update_(Dir, {MyTag, Path});
%% New Version
needs_update(AppInfo, _) ->
  needs_update_(rebar_app_info:dir(AppInfo), rebar_app_info:source(AppInfo)).

%% Function handling normalized case
needs_update_(Dir, {Tag, Path}) ->
  ...
Erlang

Note that if you resource really needs the new API to work, backwards compatibility will be difficult to achieve since whenever it will be called, it won’t have all the information of the new API.

This approach is mostly useful when you can provide an acceptable (even if degraded) user experience with the old API.

Last modified December 9, 2022: Fix broken source code links (33b768f)