pytest and degrees of separation.

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

pytest and degrees of separation.

Charles R Harris
Hi All,

Just looking for opinions and feedback on the need to keep NumPy from having a hard nose/pytest dependency. The options as I see them are:

  1. pytest is never imported until the tests are run -- current practice with nose
  2. pytest is never imported unless the testfiles are imported -- what I would like
  3. pytest is imported together when numpy is -- what we need to avoid.

Currently the approach has been 1), but I think 2) makes more sense and allows more flexibility.

Thoughts?

Chuck


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pytest and degrees of separation.

Sebastian Berg
On Tue, 2017-07-11 at 14:49 -0600, Charles R Harris wrote:

> Hi All,
>
> Just looking for opinions and feedback on the need to keep NumPy from
> having a hard nose/pytest dependency. The options as I see them are:
>
> pytest is never imported until the tests are run -- current practice
> with nose
> pytest is never imported unless the testfiles are imported -- what I
> would like 
> pytest is imported together when numpy is -- what we need to avoid.
> Currently the approach has been 1), but I think 2) makes more sense
> and allows more flexibility.

I am not quite sure about everything here. My guess is we can do
whatever we want when it comes to our own tests, and I don't mind just
switching everything to pytest (I for one am happy as long as I can run
`runtests.py` ;)).
When it comes to the utils we provide, those should keep working
without nose/pytest if they worked before without it I think.

My guess is that all your options do that, so I think we should take
the one that gives the nicest maintainable code :). Though can't say I
looked enough into it to really make a well educated decision, that
probably means your option 2.

- Sebastian



> Thoughts?
> Chuck
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion

signature.asc (817 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pytest and degrees of separation.

Thomas Caswell
Going with option 2 is probably the best option so that you can use pytest fixtures and parameterization.

Might be worth looking at how Matplotlib re-arranged things on our master branch to maintain back-compatibility with nose-specific tools that were used by down-stream projects.

Tom

On Tue, Jul 11, 2017 at 4:22 PM Sebastian Berg <[hidden email]> wrote:
On Tue, 2017-07-11 at 14:49 -0600, Charles R Harris wrote:
> Hi All,
>
> Just looking for opinions and feedback on the need to keep NumPy from
> having a hard nose/pytest dependency. The options as I see them are:
>
> pytest is never imported until the tests are run -- current practice
> with nose
> pytest is never imported unless the testfiles are imported -- what I
> would like 
> pytest is imported together when numpy is -- what we need to avoid.
> Currently the approach has been 1), but I think 2) makes more sense
> and allows more flexibility.


I am not quite sure about everything here. My guess is we can do
whatever we want when it comes to our own tests, and I don't mind just
switching everything to pytest (I for one am happy as long as I can run
`runtests.py` ;)).
When it comes to the utils we provide, those should keep working
without nose/pytest if they worked before without it I think.

My guess is that all your options do that, so I think we should take
the one that gives the nicest maintainable code :). Though can't say I
looked enough into it to really make a well educated decision, that
probably means your option 2.

- Sebastian



> Thoughts?
> Chuck
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/numpy-discussion_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pytest and degrees of separation.

Chris Barker - NOAA Federal


On Tue, Jul 11, 2017 at 5:04 PM, Thomas Caswell <[hidden email]> wrote:
Going with option 2 is probably the best option so that you can use pytest fixtures and parameterization.

I agree -- those are worth a lot!

-CHB

 
Might be worth looking at how Matplotlib re-arranged things on our master branch to maintain back-compatibility with nose-specific tools that were used by down-stream projects.

Tom

On Tue, Jul 11, 2017 at 4:22 PM Sebastian Berg <[hidden email]> wrote:
On Tue, 2017-07-11 at 14:49 -0600, Charles R Harris wrote:
> Hi All,
>
> Just looking for opinions and feedback on the need to keep NumPy from
> having a hard nose/pytest dependency. The options as I see them are:
>
> pytest is never imported until the tests are run -- current practice
> with nose
> pytest is never imported unless the testfiles are imported -- what I
> would like 
> pytest is imported together when numpy is -- what we need to avoid.
> Currently the approach has been 1), but I think 2) makes more sense
> and allows more flexibility.


I am not quite sure about everything here. My guess is we can do
whatever we want when it comes to our own tests, and I don't mind just
switching everything to pytest (I for one am happy as long as I can run
`runtests.py` ;)).
When it comes to the utils we provide, those should keep working
without nose/pytest if they worked before without it I think.

My guess is that all your options do that, so I think we should take
the one that gives the nicest maintainable code :). Though can't say I
looked enough into it to really make a well educated decision, that
probably means your option 2.

- Sebastian



> Thoughts?
> Chuck
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/numpy-discussion_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion




--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

[hidden email]

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pytest and degrees of separation.

ralfgommers


On Wed, Jul 12, 2017 at 11:06 AM, Chris Barker <[hidden email]> wrote:


On Tue, Jul 11, 2017 at 5:04 PM, Thomas Caswell <[hidden email]> wrote:
Going with option 2 is probably the best option so that you can use pytest fixtures and parameterization.

I agree -- those are worth a lot!

Maybe I'm dense, but I don't quite see the difference between 1 and 2. Test files should never be imported unless tests are run, they're not part of any public API nor do they currently have __init__.py files.

Ralf

 

-CHB

 
Might be worth looking at how Matplotlib re-arranged things on our master branch to maintain back-compatibility with nose-specific tools that were used by down-stream projects.

Tom

On Tue, Jul 11, 2017 at 4:22 PM Sebastian Berg <[hidden email]> wrote:
On Tue, 2017-07-11 at 14:49 -0600, Charles R Harris wrote:
> Hi All,
>
> Just looking for opinions and feedback on the need to keep NumPy from
> having a hard nose/pytest dependency. The options as I see them are:
>
> pytest is never imported until the tests are run -- current practice
> with nose
> pytest is never imported unless the testfiles are imported -- what I
> would like 
> pytest is imported together when numpy is -- what we need to avoid.
> Currently the approach has been 1), but I think 2) makes more sense
> and allows more flexibility.


I am not quite sure about everything here. My guess is we can do
whatever we want when it comes to our own tests, and I don't mind just
switching everything to pytest (I for one am happy as long as I can run
`runtests.py` ;)).
When it comes to the utils we provide, those should keep working
without nose/pytest if they worked before without it I think.

My guess is that all your options do that, so I think we should take
the one that gives the nicest maintainable code :). Though can't say I
looked enough into it to really make a well educated decision, that
probably means your option 2.

- Sebastian



> Thoughts?
> Chuck
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/numpy-discussion_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion




--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            <a href="tel:(206)%20526-6959" value="+12065266959" target="_blank">(206) 526-6959   voice
7600 Sand Point Way NE   <a href="tel:(206)%20526-6329" value="+12065266329" target="_blank">(206) 526-6329   fax
Seattle, WA  98115       <a href="tel:(206)%20526-6317" value="+12065266317" target="_blank">(206) 526-6317   main reception

[hidden email]

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion



_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pytest and degrees of separation.

Charles R Harris


On Wed, Jul 12, 2017 at 1:26 AM, Ralf Gommers <[hidden email]> wrote:


On Wed, Jul 12, 2017 at 11:06 AM, Chris Barker <[hidden email]> wrote:


On Tue, Jul 11, 2017 at 5:04 PM, Thomas Caswell <[hidden email]> wrote:
Going with option 2 is probably the best option so that you can use pytest fixtures and parameterization.

I agree -- those are worth a lot!

Maybe I'm dense, but I don't quite see the difference between 1 and 2. Test files should never be imported unless tests are run, they're not part of any public API nor do they currently have __init__.py files.

Ralf

In practice, that would generally be true, but the nose testing tools were 1, all nose imports were buried in functions that ran during testing. Whether or not that was by intent I don't know. But having an explicit consensus on 2, which seems to be the case here, is helpful because it allows better use of pytest fixtures.

<snip>

Chuck


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pytest and degrees of separation.

Pauli Virtanen-3
Charles R Harris kirjoitti 12.07.2017 klo 13:53:
> In practice, that would generally be true, but the nose testing tools
> were 1, all nose imports were buried in functions that ran during
> testing. Whether or not that was by intent I don't know. But having an
> explicit consensus on 2, which seems to be the case here, is helpful
> because it allows better use of pytest fixtures.

I guess the question is about shipping new pytest fixtures as a part of
the public API of numpy.testing, for use by 3rd party projects.

If the issue is only with Numpy's own tests, they can import stuff from
a private submodule that's not imported by "import numpy.testing", so it
does not introduce a dependency.

(Similar thing for the public API might also be possible e.g. "import
numpy.testing.pytest_fixtures" but it comes at the cost of a new submodule.)

So I guess a main question actually is: how much of the public API in
numpy.testing should be ported to pytest for use by 3rd projects?

The numerical assert functions are obviously useful.

The warnings suppression (pytest warning stuff IIRC doesn't deal with
warning registries nor work around the bugs in warnings.catch_warnings)
similarly --- it could make sense to actually upstream it...

But I'm not so clear about the rest.

        Pauli
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pytest and degrees of separation.

Pauli Virtanen-3
In reply to this post by Charles R Harris
Charles R Harris kirjoitti 12.07.2017 klo 13:53:
> In practice, that would generally be true, but the nose testing tools
> were 1, all nose imports were buried in functions that ran during
> testing. Whether or not that was by intent I don't know. But having an
> explicit consensus on 2, which seems to be the case here, is helpful
> because it allows better use of pytest fixtures.

I guess the question is about shipping new pytest fixtures as a part of
the public API of numpy.testing, for use by 3rd party projects.

If the issue is only with Numpy's own tests, they can import stuff from
a private submodule that's not imported by "import numpy.testing", so it
does not introduce a dependency.

(Similar thing for the public API might also be possible e.g. "import
numpy.testing.pytest_fixtures" but it comes at the cost of a new submodule.)

So I guess a main question actually is: how much of the public API in
numpy.testing should be ported to pytest for use by 3rd projects?

The numerical assert functions are obviously useful.

The warnings suppression (pytest warning stuff IIRC doesn't deal with
warning registries nor work around the bugs in warnings.catch_warnings)
similarly --- it could make sense to actually upstream it...

But I'm not so clear about the rest.

        Pauli
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pytest and degrees of separation.

Pauli Virtanen-3
In reply to this post by Charles R Harris
Charles R Harris kirjoitti 12.07.2017 klo 13:53:
> In practice, that would generally be true, but the nose testing tools
> were 1, all nose imports were buried in functions that ran during
> testing. Whether or not that was by intent I don't know. But having an
> explicit consensus on 2, which seems to be the case here, is helpful
> because it allows better use of pytest fixtures.

I guess the question is about shipping new pytest fixtures as a part of
the public API of numpy.testing, for use by 3rd party projects.

If the issue is only with Numpy's own tests, they can import stuff from
a private submodule that's not imported by "import numpy.testing", so it
does not introduce a dependency.

(Similar thing for the public API might also be possible e.g. "import
numpy.testing.pytest_fixtures" but it comes at the cost of a new submodule.)

So I guess a main question actually is: how much of the public API in
numpy.testing should be ported to pytest for use by 3rd projects?

The numerical assert functions are obviously useful.

The warnings suppression (pytest warning stuff IIRC doesn't deal with
warning registries nor work around the bugs in warnings.catch_warnings)
similarly --- it could make sense to actually upstream it...

But I'm not so clear about the rest.

        Pauli
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pytest and degrees of separation.

ralfgommers
In reply to this post by Pauli Virtanen-3


On Thu, Jul 13, 2017 at 8:14 AM, Pauli Virtanen <[hidden email]> wrote:
Charles R Harris kirjoitti 12.07.2017 klo 13:53:
> In practice, that would generally be true, but the nose testing tools
> were 1, all nose imports were buried in functions that ran during
> testing. Whether or not that was by intent I don't know. But having an
> explicit consensus on 2, which seems to be the case here, is helpful
> because it allows better use of pytest fixtures.

I guess the question is about shipping new pytest fixtures as a part of
the public API of numpy.testing, for use by 3rd party projects.

Agreed. That's a different question, and I'd prefer to keep things as they are in that respect. Otherwise it's basically a hard dependency of numpy itself on pytest.
 
If the issue is only with Numpy's own tests, they can import stuff from
a private submodule that's not imported by "import numpy.testing", so it
does not introduce a dependency.

(Similar thing for the public API might also be possible e.g. "import
numpy.testing.pytest_fixtures" but it comes at the cost of a new submodule.)

So I guess a main question actually is: how much of the public API in
numpy.testing should be ported to pytest for use by 3rd projects?

The numerical assert functions are obviously useful.

The warnings suppression (pytest warning stuff IIRC doesn't deal with
warning registries nor work around the bugs in warnings.catch_warnings)
similarly --- it could make sense to actually upstream it...

But I'm not so clear about the rest.

Agreed, nothing in the decorators that obviously needs a pytest-based implementation. The Tester class may be the one thing.

Ralf



_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Loading...