Is the build-system section of pyproject.toml a maintained list of the build dependencies?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Is the build-system section of pyproject.toml a maintained list of the build dependencies?

Warren Weckesser-2
I'm doing some work on the travis-ci scripts, and I'd like to remove
some redundant calls of 'pip install'.  The scripts should get the
build dependencies from a configuration file instead of having
hard-coded pip commands.  Is pyproject.toml the appropriate file to
use for this?  (Note: we also have test_requirements.txt, but as the
name says, those are the dependencies for running the tests, not for
building numpy.)

Warren
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Is the build-system section of pyproject.toml a maintained list of the build dependencies?

Warren Weckesser-2
On 1/8/20, Warren Weckesser <[hidden email]> wrote:
> I'm doing some work on the travis-ci scripts, and I'd like to remove
> some redundant calls of 'pip install'.  The scripts should get the
> build dependencies from a configuration file instead of having
> hard-coded pip commands.  Is pyproject.toml the appropriate file to
> use for this?  (Note: we also have test_requirements.txt, but as the
> name says, those are the dependencies for running the tests, not for
> building numpy.)
>

Updating my question:  `pyproject.toml` lists numpy's build
dependencies in the `build_system` section of the file:

[build-system]
# Minimum requirements for the build system to execute.
requires = [
    "setuptools",
    "wheel",
    "Cython>=0.29.14",  # Note: keep in sync with tools/cythonize.py
]

So the file serves the equivalent purpose of a `requirements.txt`
file.  Is there an option to pip that would allow something like

    pip install -r pyproject.toml

(with some other flag or option as needed) to install the build
requirements found in pyproject.toml?  In
https://github.com/numpy/numpy/pull/15275, I wrote a few lines of
Python to get the dependencies from pyproject.toml, but it seems like
that shouldn't be necessary.

Warren


> Warren
>
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Is the build-system section of pyproject.toml a maintained list of the build dependencies?

bashtage
With recent versions of pip it will read the pyproject.toml file to get the dependencies, and then install these in an isolated environment to build the wheel, and then install the wheel.  The requires=[...] in the pyproject is not installed in the original environment, so that when run on NumPy, you would only end up with NumPy installed. Are you trying to get Cython installed after an install of NumPy?  If you want this then it needs to be listed in the setup as a dependency.

On Wed, Jan 8, 2020 at 4:38 PM Warren Weckesser <[hidden email]> wrote:
On 1/8/20, Warren Weckesser <[hidden email]> wrote:
> I'm doing some work on the travis-ci scripts, and I'd like to remove
> some redundant calls of 'pip install'.  The scripts should get the
> build dependencies from a configuration file instead of having
> hard-coded pip commands.  Is pyproject.toml the appropriate file to
> use for this?  (Note: we also have test_requirements.txt, but as the
> name says, those are the dependencies for running the tests, not for
> building numpy.)
>

Updating my question:  `pyproject.toml` lists numpy's build
dependencies in the `build_system` section of the file:

[build-system]
# Minimum requirements for the build system to execute.
requires = [
    "setuptools",
    "wheel",
    "Cython>=0.29.14",  # Note: keep in sync with tools/cythonize.py
]

So the file serves the equivalent purpose of a `requirements.txt`
file.  Is there an option to pip that would allow something like

    pip install -r pyproject.toml

(with some other flag or option as needed) to install the build
requirements found in pyproject.toml?  In
https://github.com/numpy/numpy/pull/15275, I wrote a few lines of
Python to get the dependencies from pyproject.toml, but it seems like
that shouldn't be necessary.

Warren


> Warren
>
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Is the build-system section of pyproject.toml a maintained list of the build dependencies?

Warren Weckesser-2
On 1/8/20, Kevin Sheppard <[hidden email]> wrote:
> With recent versions of pip it will read the pyproject.toml file to get the
> dependencies, and then install these in an isolated environment to build
> the wheel, and then install the wheel.  The requires=[...] in the pyproject
> is not installed in the original environment, so that when run on NumPy,
> you would only end up with NumPy installed. Are you trying to get Cython
> installed after an install of NumPy?  If you want this then it needs to be
> listed in the setup as a dependency


Thanks Kevin.  I'm cleaning up the shell scripts that we use on
Travis-CI.  There were several redundant uses of `pip install`, some
of which were installing build requirements.  The pull request is
https://github.com/numpy/numpy/pull/15275, but I think the code can be
further simplified.

Does building with setup.py (instead of pip) use pyproject.toml?

Warren



>
> On Wed, Jan 8, 2020 at 4:38 PM Warren Weckesser
> <[hidden email]>
> wrote:
>
>> On 1/8/20, Warren Weckesser <[hidden email]> wrote:
>> > I'm doing some work on the travis-ci scripts, and I'd like to remove
>> > some redundant calls of 'pip install'.  The scripts should get the
>> > build dependencies from a configuration file instead of having
>> > hard-coded pip commands.  Is pyproject.toml the appropriate file to
>> > use for this?  (Note: we also have test_requirements.txt, but as the
>> > name says, those are the dependencies for running the tests, not for
>> > building numpy.)
>> >
>>
>> Updating my question:  `pyproject.toml` lists numpy's build
>> dependencies in the `build_system` section of the file:
>>
>> [build-system]
>> # Minimum requirements for the build system to execute.
>> requires = [
>>     "setuptools",
>>     "wheel",
>>     "Cython>=0.29.14",  # Note: keep in sync with tools/cythonize.py
>> ]
>>
>> So the file serves the equivalent purpose of a `requirements.txt`
>> file.  Is there an option to pip that would allow something like
>>
>>     pip install -r pyproject.toml
>>
>> (with some other flag or option as needed) to install the build
>> requirements found in pyproject.toml?  In
>> https://github.com/numpy/numpy/pull/15275, I wrote a few lines of
>> Python to get the dependencies from pyproject.toml, but it seems like
>> that shouldn't be necessary.
>>
>> Warren
>>
>>
>> > Warren
>> >
>> _______________________________________________
>> NumPy-Discussion mailing list
>> [hidden email]
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
>
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Is the build-system section of pyproject.toml a maintained list of the build dependencies?

ralfgommers


On Wed, Jan 8, 2020 at 6:04 PM Warren Weckesser <[hidden email]> wrote:
On 1/8/20, Kevin Sheppard <[hidden email]> wrote:
> With recent versions of pip it will read the pyproject.toml file to get the
> dependencies, and then install these in an isolated environment to build
> the wheel, and then install the wheel.  The requires=[...] in the pyproject
> is not installed in the original environment, so that when run on NumPy,
> you would only end up with NumPy installed. Are you trying to get Cython
> installed after an install of NumPy?  If you want this then it needs to be
> listed in the setup as a dependency


Thanks Kevin.  I'm cleaning up the shell scripts that we use on
Travis-CI.  There were several redundant uses of `pip install`, some
of which were installing build requirements.  The pull request is
https://github.com/numpy/numpy/pull/15275, but I think the code can be
further simplified.

Does building with setup.py (instead of pip) use pyproject.toml?

No it doesn't.

Ralf


Warren



>
> On Wed, Jan 8, 2020 at 4:38 PM Warren Weckesser
> <[hidden email]>
> wrote:
>
>> On 1/8/20, Warren Weckesser <[hidden email]> wrote:
>> > I'm doing some work on the travis-ci scripts, and I'd like to remove
>> > some redundant calls of 'pip install'.  The scripts should get the
>> > build dependencies from a configuration file instead of having
>> > hard-coded pip commands.  Is pyproject.toml the appropriate file to
>> > use for this?  (Note: we also have test_requirements.txt, but as the
>> > name says, those are the dependencies for running the tests, not for
>> > building numpy.)
>> >
>>
>> Updating my question:  `pyproject.toml` lists numpy's build
>> dependencies in the `build_system` section of the file:
>>
>> [build-system]
>> # Minimum requirements for the build system to execute.
>> requires = [
>>     "setuptools",
>>     "wheel",
>>     "Cython>=0.29.14",  # Note: keep in sync with tools/cythonize.py
>> ]
>>
>> So the file serves the equivalent purpose of a `requirements.txt`
>> file.  Is there an option to pip that would allow something like
>>
>>     pip install -r pyproject.toml
>>
>> (with some other flag or option as needed) to install the build
>> requirements found in pyproject.toml?  In
>> https://github.com/numpy/numpy/pull/15275, I wrote a few lines of
>> Python to get the dependencies from pyproject.toml, but it seems like
>> that shouldn't be necessary.
>>
>> Warren
>>
>>
>> > Warren
>> >
>> _______________________________________________
>> NumPy-Discussion mailing list
>> [hidden email]
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
>
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion