caml-list - the Caml user's mailing list
 help / Atom feed
* [Caml-list] (dune/opam) Proper way of vendoring a library inside an application?
@ 2020-01-31  8:49 François Pottier
  2020-01-31 13:08 ` François Bobot
                   ` (2 more replies)
  0 siblings, 3 replies; 16+ messages in thread
From: François Pottier @ 2020-01-31  8:49 UTC (permalink / raw)
  To: caml users


Dear fellow OCaml users,

I am trying to figure out how to "vendor" (embark) a private
copy of a library inside a project. (Concretely, I would like
to vendor Fix inside Menhir.) I want this fact to remain
invisible, that is, installing Menhir should not result
in installing or upgrading the user's public copy of Fix.

I have copied the source tree for Fix inside the source tree
for Menhir, and added a (vendored_dirs fix) stanza in Menhir's
toplevel "dune" file.

Compilation works fine, but installation does not seem to work
the way I wish:

* "dune install" installs Menhir *and Fix*, though I would like
   it to install Menhir only.

* Attempting to pin Menhir using "opam pin" fails because opam
   cannot find Fix (inded, Fix is not globally installed).

I have tried removing (public_name fix) from fix/src/dune, but
then the compilation of Menhir fails (Fix cannot be found).

I am confused... what's the proper way of embarking a private
library inside an application without installing it globally?

Thanks!

-- 
François Pottier
francois.pottier@inria.fr
http://cambium.inria.fr/~fpottier/

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [Caml-list] (dune/opam) Proper way of vendoring a library inside an application?
  2020-01-31  8:49 [Caml-list] (dune/opam) Proper way of vendoring a library inside an application? François Pottier
@ 2020-01-31 13:08 ` François Bobot
  2020-01-31 14:53   ` François Pottier
  2020-02-03 10:27 ` Jeremie Dimino
  2020-02-29  8:41 ` [Caml-list] (dune/opam) Proper way of vendoring a library inside an application? Richard W.M. Jones
  2 siblings, 1 reply; 16+ messages in thread
From: François Bobot @ 2020-01-31 13:08 UTC (permalink / raw)
  To: caml-list

Le 31/01/2020 à 09:48, François Pottier a écrit :
> I am trying to figure out how to "vendor" (embark) a private
> copy of a library inside a project. (Concretely, I would like
> to vendor Fix inside Menhir.) I want this fact to remain
> invisible, that is, installing Menhir should not result
> in installing or upgrading the user's public copy of Fix.
> 
> I have copied the source tree for Fix inside the source tree
> for Menhir, and added a (vendored_dirs fix) stanza in Menhir's
> toplevel "dune" file.

> I am confused... what's the proper way of embarking a private
> library inside an application without installing it globally?


Will Fix be used for MenhirLib? It is just to understand the impact on users of the library.

Best,

-- 
François

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [Caml-list] (dune/opam) Proper way of vendoring a library inside an application?
  2020-01-31 13:08 ` François Bobot
@ 2020-01-31 14:53   ` François Pottier
  0 siblings, 0 replies; 16+ messages in thread
From: François Pottier @ 2020-01-31 14:53 UTC (permalink / raw)
  To: François Bobot, caml-list

On 31/01/2020 14:08, François Bobot wrote:
> Will Fix be used for MenhirLib? It is just to understand the impact on users of the library.

No, Fix will be used only by the menhir executable,
not by MenhirLib.

-- 
François Pottier
francois.pottier@inria.fr
http://cambium.inria.fr/~fpottier/

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [Caml-list] (dune/opam) Proper way of vendoring a library inside  an application?
  2020-01-31  8:49 [Caml-list] (dune/opam) Proper way of vendoring a library inside an application? François Pottier
  2020-01-31 13:08 ` François Bobot
@ 2020-02-03 10:27 ` Jeremie Dimino
  2020-02-03 14:17   ` Jeremie Dimino
  2020-02-29  8:41 ` [Caml-list] (dune/opam) Proper way of vendoring a library inside an application? Richard W.M. Jones
  2 siblings, 1 reply; 16+ messages in thread
From: Jeremie Dimino @ 2020-02-03 10:27 UTC (permalink / raw)
  To: francois.pottier, caml users

Hi François,

On Fri, Jan 31, 2020 at 8:48 AM François Pottier
<francois.pottier@inria.fr> wrote:
> * "dune install" installs Menhir *and Fix*, though I would like
>    it to install Menhir only.
>
> * Attempting to pin Menhir using "opam pin" fails because opam
>    cannot find Fix (inded, Fix is not globally installed).

These look like bugs in Dune, we'll fix them for the next release.

Cheers,

Jeremie

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [Caml-list] (dune/opam) Proper way of vendoring a library inside  an application?
  2020-02-03 10:27 ` Jeremie Dimino
@ 2020-02-03 14:17   ` Jeremie Dimino
  2020-02-03 14:53     ` François Pottier
  0 siblings, 1 reply; 16+ messages in thread
From: Jeremie Dimino @ 2020-02-03 14:17 UTC (permalink / raw)
  To: francois.pottier, caml users

Here are the PR fixing these:

https://github.com/ocaml/dune/pull/3074
https://github.com/ocaml/dune/pull/3075

On Mon, Feb 3, 2020 at 10:26 AM Jeremie Dimino <jdimino@janestreet.com> wrote:
>
> Hi François,
>
> On Fri, Jan 31, 2020 at 8:48 AM François Pottier
> <francois.pottier@inria.fr> wrote:
> > * "dune install" installs Menhir *and Fix*, though I would like
> >    it to install Menhir only.
> >
> > * Attempting to pin Menhir using "opam pin" fails because opam
> >    cannot find Fix (inded, Fix is not globally installed).
>
> These look like bugs in Dune, we'll fix them for the next release.
>
> Cheers,
>
> Jeremie



-- 
Jeremie

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [Caml-list] (dune/opam) Proper way of vendoring a library inside an application?
  2020-02-03 14:17   ` Jeremie Dimino
@ 2020-02-03 14:53     ` François Pottier
  2020-02-03 23:13       ` Jeremie Dimino
  0 siblings, 1 reply; 16+ messages in thread
From: François Pottier @ 2020-02-03 14:53 UTC (permalink / raw)
  To: Jeremie Dimino, caml users

On 03/02/2020 15:16, Jeremie Dimino wrote:
> Here are the PR fixing these:
> 
> https://github.com/ocaml/dune/pull/3074
> https://github.com/ocaml/dune/pull/3075

Thanks! I will try these changes as soon as possible
(I guess I can create a private fork of dune, merge
these PRs, and test?).

-- 
François Pottier
francois.pottier@inria.fr
http://cambium.inria.fr/~fpottier/

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [Caml-list] (dune/opam) Proper way of vendoring a library inside  an application?
  2020-02-03 14:53     ` François Pottier
@ 2020-02-03 23:13       ` Jeremie Dimino
  2020-02-04  9:11         ` François Pottier
  0 siblings, 1 reply; 16+ messages in thread
From: Jeremie Dimino @ 2020-02-03 23:13 UTC (permalink / raw)
  To: François Pottier; +Cc: caml users

On Mon, Feb 3, 2020 at 2:53 PM François Pottier
<francois.pottier@inria.fr> wrote:
> Thanks! I will try these changes as soon as possible
> (I guess I can create a private fork of dune, merge
> these PRs, and test?).

The PRs have now been merged, testing would be welcome!

-- 
Jeremie

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [Caml-list] (dune/opam) Proper way of vendoring a library inside an application?
  2020-02-03 23:13       ` Jeremie Dimino
@ 2020-02-04  9:11         ` François Pottier
  2020-02-06 22:33           ` Jeremie Dimino
  0 siblings, 1 reply; 16+ messages in thread
From: François Pottier @ 2020-02-04  9:11 UTC (permalink / raw)
  To: Jeremie Dimino; +Cc: caml users

On 04/02/2020 00:13, Jeremie Dimino wrote:
> The PRs have now been merged, testing would be welcome!

Tested, and seems to work -- both problems fixed. Thanks!
Looking forward to the next release (2.1.4?).

-- 
François Pottier
francois.pottier@inria.fr
http://cambium.inria.fr/~fpottier/

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [Caml-list] (dune/opam) Proper way of vendoring a library inside  an application?
  2020-02-04  9:11         ` François Pottier
@ 2020-02-06 22:33           ` Jeremie Dimino
  2020-02-10  9:37             ` [Caml-list] dune for OCaml < 4.07? François Pottier
  0 siblings, 1 reply; 16+ messages in thread
From: Jeremie Dimino @ 2020-02-06 22:33 UTC (permalink / raw)
  To: François Pottier; +Cc: caml users

Thanks for testing!
We just submitted the release of Dune 2.2.0:
https://github.com/ocaml/opam-repository/pull/15795

On Tue, Feb 4, 2020 at 10:11 AM François Pottier
<francois.pottier@inria.fr> wrote:
>
> On 04/02/2020 00:13, Jeremie Dimino wrote:
> > The PRs have now been merged, testing would be welcome!
>
> Tested, and seems to work -- both problems fixed. Thanks!
> Looking forward to the next release (2.1.4?).
>
> --
> François Pottier
> francois.pottier@inria.fr
> http://cambium.inria.fr/~fpottier/



-- 
Jeremie

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [Caml-list] dune for OCaml < 4.07?
  2020-02-06 22:33           ` Jeremie Dimino
@ 2020-02-10  9:37             ` François Pottier
  2020-02-10 10:19               ` David Allsopp
  0 siblings, 1 reply; 16+ messages in thread
From: François Pottier @ 2020-02-10  9:37 UTC (permalink / raw)
  To: Jeremie Dimino; +Cc: caml users


On 06/02/2020 23:33, Jeremie Dimino wrote:
> We just submitted the release of Dune 2.2.0:
> https://github.com/ocaml/opam-repository/pull/15795

Thanks Jérémie. This seems to work great.

However, I realize that recent versions of dune require OCaml 4.07.
Menhir is supposed to be compatible with OCaml all the way back to
4.02.3. Is there any way to install dune 2.2.0 with OCaml < 4.07?

--
François Pottier
francois.pottier@inria.fr
http://cambium.inria.fr/~fpottier/

^ permalink raw reply	[flat|nested] 16+ messages in thread

* RE: [Caml-list] dune for OCaml < 4.07?
  2020-02-10  9:37             ` [Caml-list] dune for OCaml < 4.07? François Pottier
@ 2020-02-10 10:19               ` David Allsopp
  2020-02-10 10:33                 ` François Pottier
  0 siblings, 1 reply; 16+ messages in thread
From: David Allsopp @ 2020-02-10 10:19 UTC (permalink / raw)
  To: François Pottier, Jeremie Dimino; +Cc: caml users

François Pottier wrote:
> On 06/02/2020 23:33, Jeremie Dimino wrote:
> > We just submitted the release of Dune 2.2.0:
> > https://github.com/ocaml/opam-repository/pull/15795
> 
> Thanks Jérémie. This seems to work great.
> 
> However, I realize that recent versions of dune require OCaml 4.07.
> Menhir is supposed to be compatible with OCaml all the way back to 4.02.3.
> Is there any way to install dune 2.2.0 with OCaml < 4.07?

TL;DR for installing Menhir on OCaml 4.02-4.06 using opam, you don't need to do anything.

Dune 2.0.0+ requires at least OCaml 4.07 to build, but the dune package in opam does not. More detail in case it's relevant to be able to do that without opam...

For OCaml 4.02-4.06, Dune depends on the ocaml-secondary-compiler package[1], which installs a separate copy of OCaml 4.08.1 in your switch which is accessed using `ocamlfind -toolchain secondary` and that's configured in the ocamlfind-secondary package. Dune's bootstrap natively looks for the ocamlfind secondary toolchain if the OCaml version is < 4.07, so opam is not required, you just have to configure ocamlfind manually if not using it. So installing without opam requires building (or having) a later of version of OCaml in a different path, and then creating the configuration file for ocamlfind so that it can use it.

HTH,


David


[1] https://github.com/ocaml/opam-repository/tree/master/packages/ocaml-secondary-compiler/ocaml-secondary-compiler.4.08.1
[2] https://github.com/ocaml/opam-repository/tree/master/packages/ocamlfind-secondary/ocamlfind-secondary.1.8.1


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [Caml-list] dune for OCaml < 4.07?
  2020-02-10 10:19               ` David Allsopp
@ 2020-02-10 10:33                 ` François Pottier
  2020-02-10 11:27                   ` David Allsopp
  0 siblings, 1 reply; 16+ messages in thread
From: François Pottier @ 2020-02-10 10:33 UTC (permalink / raw)
  To: David Allsopp, Jeremie Dimino; +Cc: caml users


Hello David,

> TL;DR for installing Menhir on OCaml 4.02-4.06 using opam, you don't need to do anything.

Thanks for your prompt reply, which sounds reassuring!

However, attempting to install dune 2.2.0 on a pre-4.07 switch
(for instance, 4.02.3) results in the following failure:

   $ opam install dune.2.2.0
   The following dependencies couldn't be met:
     - dune → ocaml >= 4.07 → ocaml-variants >= 4.07.0
         conflict with the base packages of this switch
     - dune → ocaml >= 4.07 → ocaml-system >= 4.07.0
         unmet availability conditions, e.g. sys-ocaml-version = "4.07.0"
     - dune → ocaml >= 4.07 → ocaml-base-compiler >= 4.07.0
         base of this switch (use `--unlock-base' to force)

I have run "opam update" first. Is there something else I should do
to convince opam that it is possible to install dune?

--
François Pottier
francois.pottier@inria.fr
http://cambium.inria.fr/~fpottier/

^ permalink raw reply	[flat|nested] 16+ messages in thread

* RE: [Caml-list] dune for OCaml < 4.07?
  2020-02-10 10:33                 ` François Pottier
@ 2020-02-10 11:27                   ` David Allsopp
  0 siblings, 0 replies; 16+ messages in thread
From: David Allsopp @ 2020-02-10 11:27 UTC (permalink / raw)
  To: François Pottier, Jeremie Dimino; +Cc: caml users

François Pottier wrote:
> > TL;DR for installing Menhir on OCaml 4.02-4.06 using opam, you don't
> > need to do anything.
> 
> Thanks for your prompt reply, which sounds reassuring!
> 
> However, attempting to install dune 2.2.0 on a pre-4.07 switch (for
> instance, 4.02.3) results in the following failure:
> 
>    $ opam install dune.2.2.0
>    The following dependencies couldn't be met:
>      - dune → ocaml >= 4.07 → ocaml-variants >= 4.07.0
>          conflict with the base packages of this switch
>      - dune → ocaml >= 4.07 → ocaml-system >= 4.07.0
>          unmet availability conditions, e.g. sys-ocaml-version = "4.07.0"
>      - dune → ocaml >= 4.07 → ocaml-base-compiler >= 4.07.0
>          base of this switch (use `--unlock-base' to force)
> 
> I have run "opam update" first. Is there something else I should do to
> convince opam that it is possible to install dune?

Oops :$ - https://github.com/ocaml/opam-repository/pull/15820 !

Best,


D



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [Caml-list] (dune/opam) Proper way of vendoring a library inside an application?
  2020-01-31  8:49 [Caml-list] (dune/opam) Proper way of vendoring a library inside an application? François Pottier
  2020-01-31 13:08 ` François Bobot
  2020-02-03 10:27 ` Jeremie Dimino
@ 2020-02-29  8:41 ` Richard W.M. Jones
  2020-02-29 11:20   ` François Pottier
  2 siblings, 1 reply; 16+ messages in thread
From: Richard W.M. Jones @ 2020-02-29  8:41 UTC (permalink / raw)
  To: francois.pottier, caml users

On Fri, Jan 31, 2020 at 09:48:48AM +0100, François Pottier wrote:
> 
> Dear fellow OCaml users,
> 
> I am trying to figure out how to "vendor" (embark) a private
> copy of a library inside a project. (Concretely, I would like
> to vendor Fix inside Menhir.) I want this fact to remain
> invisible, that is, installing Menhir should not result
> in installing or upgrading the user's public copy of Fix.
> 
> I have copied the source tree for Fix inside the source tree
> for Menhir, and added a (vendored_dirs fix) stanza in Menhir's
> toplevel "dune" file.
> 
> Compilation works fine, but installation does not seem to work
> the way I wish:
> 
> * "dune install" installs Menhir *and Fix*, though I would like
>   it to install Menhir only.
> 
> * Attempting to pin Menhir using "opam pin" fails because opam
>   cannot find Fix (inded, Fix is not globally installed).
> 
> I have tried removing (public_name fix) from fix/src/dune, but
> then the compilation of Menhir fails (Fix cannot be found).
> 
> I am confused... what's the proper way of embarking a private
> library inside an application without installing it globally?

I wish you wouldn't do this.  Bundling libraries like this causes
problems for packagers because we can never be sure what (possibly out
of date or insecure) version of a library that we package elsewhere
has been bundled in another place.

As is this is already in opam, what are you trying to achieve here?

Rich.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [Caml-list] (dune/opam) Proper way of vendoring a library inside an application?
  2020-02-29  8:41 ` [Caml-list] (dune/opam) Proper way of vendoring a library inside an application? Richard W.M. Jones
@ 2020-02-29 11:20   ` François Pottier
  2020-03-02  9:33     ` François Bobot
  0 siblings, 1 reply; 16+ messages in thread
From: François Pottier @ 2020-02-29 11:20 UTC (permalink / raw)
  To: Richard W.M. Jones, caml users


Hi,

On 29/02/2020 09:41, Richard W.M. Jones wrote:
> As is this is already in opam, what are you trying to achieve here?

I want to maintain the property that Menhir depends on no external
libraries, as this makes it easier for some people (who do not use
opam) to install.

Also, this allows me to know exactly which version of fix I am
using, so two versions of Menhir that have the same version
number are guaranteed to work in the same way; that wouldn't
necessarily be true otherwise.

I don't see how it could cause any packaging problem; it should
be transparent. The copy of Fix embedded inside Menhir is used
when Menhir is installed and is immediately thrown away.

-- 
François Pottier
francois.pottier@inria.fr
http://cambium.inria.fr/~fpottier/

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [Caml-list] (dune/opam) Proper way of vendoring a library inside an application?
  2020-02-29 11:20   ` François Pottier
@ 2020-03-02  9:33     ` François Bobot
  0 siblings, 0 replies; 16+ messages in thread
From: François Bobot @ 2020-03-02  9:33 UTC (permalink / raw)
  To: caml-list

Le 29/02/2020 à 12:20, François Pottier a écrit :
> I don't see how it could cause any packaging problem; it should
> be transparent. The copy of Fix embedded inside Menhir is used
> when Menhir is installed and is immediately thrown away.
> 

Even if it is perhaps not applicable for Fix which is a small library, without attack surface.
Generally if there is a security bug in Fix, distributions don't want to need to patch it in all the
package which vendor Fix. Patching Fix once is simpler, more efficient and safer.

But for a distribution removing this vendor directory just mean to remove it, no other modifications
are needed; dune will then used the installed dependency. Package creator could look at
`(vendored_dirs vendor)` to find those directories. Of course the version can be different from the
last version of Fix. But to choose common version is usually the hurdle of packagers (which we
should strive not to burden more!).

Best,

-- 
François

^ permalink raw reply	[flat|nested] 16+ messages in thread

end of thread, back to index

Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-01-31  8:49 [Caml-list] (dune/opam) Proper way of vendoring a library inside an application? François Pottier
2020-01-31 13:08 ` François Bobot
2020-01-31 14:53   ` François Pottier
2020-02-03 10:27 ` Jeremie Dimino
2020-02-03 14:17   ` Jeremie Dimino
2020-02-03 14:53     ` François Pottier
2020-02-03 23:13       ` Jeremie Dimino
2020-02-04  9:11         ` François Pottier
2020-02-06 22:33           ` Jeremie Dimino
2020-02-10  9:37             ` [Caml-list] dune for OCaml < 4.07? François Pottier
2020-02-10 10:19               ` David Allsopp
2020-02-10 10:33                 ` François Pottier
2020-02-10 11:27                   ` David Allsopp
2020-02-29  8:41 ` [Caml-list] (dune/opam) Proper way of vendoring a library inside an application? Richard W.M. Jones
2020-02-29 11:20   ` François Pottier
2020-03-02  9:33     ` François Bobot

caml-list - the Caml user's mailing list

Archives are clonable: git clone --mirror https://inbox.ocaml.org/caml-list

AGPL code for this site: git clone https://public-inbox.org/ public-inbox