Re: NumPy-Discussion Digest, Vol 171, Issue 39

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: NumPy-Discussion Digest, Vol 171, Issue 39

Brian Soto
I'm still learning proper mailing list etiquette so I'm not sure if this is where I should respond. 
But users just getting into debugging might also benefit from knowing this:

You can turn off optimizations when compiling numpy by passing CFLAGS to setup.py like so:

CFLAGS="-O0 -g3" python setup.py build_ext -i 
*Assuming you have the source code and setup.py available 

This will remove optimizations while compiling and will make it easier to see more variables.
That took me a long time to figure out so I wanted to share the knowledge 

Thanks!

On Mon, Dec 28, 2020 at 10:38 PM <[hidden email]> wrote:
Send NumPy-Discussion mailing list submissions to
        [hidden email]

To subscribe or unsubscribe via the World Wide Web, visit
        https://mail.python.org/mailman/listinfo/numpy-discussion
or, via email, send a message with subject or body 'help' to
        [hidden email]

You can reach the person managing the list at
        [hidden email]

When replying, please edit your Subject line so it is more specific
than "Re: Contents of NumPy-Discussion digest..."


Today's Topics:

   1. Re: Addition of new distributions: Polya-gamma (Robert Kern)
   2. Help needed GDB (Amardeep Singh)
   3. ANN: NumExpr 2.7.2 (Robert McLeod)


----------------------------------------------------------------------

Message: 1
Date: Mon, 28 Dec 2020 13:06:33 -0500
From: Robert Kern <[hidden email]>
To: Discussion of Numerical Python <[hidden email]>
Subject: Re: [Numpy-discussion] Addition of new distributions:
        Polya-gamma
Message-ID:
        <[hidden email]>
Content-Type: text/plain; charset="utf-8"

My view is that we will not add more non-uniform distribution (i.e. "named"
statistical probability distributions like Polya-Gamma) methods to
`Generator`. I think that we might add a couple more methods to handle some
more fundamental issues (like sampling from the unit interval with control
over whether each boundary is open or closed, maybe one more variation on
shuffling) that helps write randomized algorithms. Now that we have the C
and Cython APIs which allow one to implement non-uniform distributions in
other packages, we strongly encourage that.

As I commented on the linked PR, `scipy.stats` would be a reasonable place
for a Polya-Gamma sampling function, even if it's not feasible to implement
an `rv_continuous` class for it. You have convinced me that the nature of
the Polya-Gamma distribution warrants this. The only issue is that scipy
still depends on a pre-`Generator` version of numpy. So I recommend
implementing this function in your own package with an eye towards
contributing it to scipy later.

On Sun, Dec 27, 2020 at 6:05 AM Zolisa Bleki <[hidden email]> wrote:

> Hi All,
>
> I would like to know if Numpy accepts addition of new distributions since
> the implementation of the Generator interface. If so, what is the criteria
> for a particular distribution to be accepted? The reason why i'm asking is
> because I would like to propose adding the Polya-gamma distribution to
> numpy, for the following reasons:
>
> 1) Polya-gamma random variables are commonly used as auxiliary variables
> during data augmentation in Bayesian sampling algorithms, which have
> wide-spread usage in Statistics and recently, Machine learning.
> 2) Since this distribution is mostly useful for random sampling, it since
> appropriate to have it in numpy and not projects like scipy [1].
> 3) The only python/C++ implementation of the sampler available is licensed
> under GPLv3 which I believe limits copying into packages that choose to use
> a different license [2].
> 4) Numpy's random API makes adding the distribution painless.
>
> I have done preliminary work on this by implementing the distribution
> sampler as decribed in [3]; see:
> https://github.com/numpy/numpy/compare/master...zoj613:polyagamma .
> There is a more efficient sampling algorithm described in a later paper
> [4], but I chose not to start with that one unless I know it is worth
> investing time in.
>
> I would appreciate your thoughts on this proposal.
>
> Regards,
> Zolisa
>
>
> Refs:
> [1] https://github.com/scipy/scipy/issues/11009
> [2] https://github.com/slinderman/pypolyagamma
> [3] https://arxiv.org/pdf/1205.0310v1.pdf
> [4] https://arxiv.org/pdf/1405.0506.pdf
>
>
>
> Disclaimer - University of Cape Town This email is subject to UCT policies
> and email disclaimer published on our website at
> http://www.uct.ac.za/main/email-disclaimer or obtainable from +27 21 650
> 9111. If this email is not related to the business of UCT, it is sent by
> the sender in an individual capacity. Please report security incidents or
> abuse via https://csirt.uct.ac.za/page/report-an-incident.php.
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/numpy-discussion
>


--
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/numpy-discussion/attachments/20201228/f4bbd564/attachment-0001.html>

------------------------------

Message: 2
Date: Tue, 29 Dec 2020 13:55:03 +0800
From: Amardeep Singh <[hidden email]>
To: [hidden email]
Subject: [Numpy-discussion] Help needed GDB
Message-ID:
        <[hidden email]>
Content-Type: text/plain; charset="utf-8"

Hi All,

I am trying to debug c code of numpy via gdb.Can someone help me with this?
i am getting " Python scripting is not supported in this copy of GDB". How
to install python supported gdb on win10?


https://numpy.org/doc/stable/dev/development_environment.html

I am following the steps in the docs. machine is windows 10.

Debugging
<https://numpy.org/doc/stable/dev/development_environment.html#debugging>

Another frequently asked question is ?How do I debug C code inside NumPy??.
First, ensure that you have gdb installed on your system with the Python
extensions (often the default on Linux). You can see which version of
Python is running inside gdb to verify your setup:

(gdb) python>import
sys>print(sys.version_info)>endsys.version_info(major=3, minor=7,
micro=0, releaselevel='final', serial=0)




$ gdb -v
GNU gdb (GDB) 7.6.1
This GDB was configured as "mingw32".

$ gdb
(gdb) python
>import sys
>print(sys.version_info)
>end
(gdb) Python scripting is not supported in this copy of GDB.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/numpy-discussion/attachments/20201229/268f82ab/attachment-0001.html>

------------------------------

Message: 3
Date: Mon, 28 Dec 2020 22:38:07 -0800
From: Robert McLeod <[hidden email]>
To: [hidden email],  Discussion of Numerical Python
        <[hidden email]>, [hidden email]
Subject: [Numpy-discussion] ANN: NumExpr 2.7.2
Message-ID:
        <[hidden email]>
Content-Type: text/plain; charset="utf-8"

========================
Announcing NumExpr 2.7.2
========================

Hi everyone,

It's been awhile since the last update to NumExpr, mostly as the existing
scientific
Python tool chain for building wheels on PyPi became defunct and we have
had to
redevelop a new one based on `cibuildwheel` and GitHub Actions. This
release also
brings us support (and wheels for) Python 3.9.

There have been a number of changes to enhance how NumExpr works when NumPy
uses MKL as a backend.

Project documentation is available at:

http://numexpr.readthedocs.io/

Changes from 2.7.1 to 2.7.2
---------------------------

- Support for Python 2.7 and 3.5 is deprecated and will be discontinued
when
  `cibuildwheels` and/or GitHub Actions no longer support these versions.
- Wheels are now provided for Python 3.7, 3.5, 3.6, 3.7, 3.8, and 3.9 via
  GitHub Actions.
- The block size is now exported into the namespace as
`numexpr.__BLOCK_SIZE1__`
  as a read-only value.
- If using MKL, the number of threads for VML is no longer forced to 1 on
loading
  the module. Testing has shown that VML never runs in multi-threaded mode
for
  the default BLOCKSIZE1 of 1024 elements, and forcing to 1 can have
deleterious
  effects on NumPy functions when built with MKL. See issue #355 for
details.
- Use of `ndarray.tostring()` in tests has been switch to
`ndarray.tobytes()`
  for future-proofing deprecation of `.tostring()`, if the version of NumPy
is
  greater than 1.9.
- Added a utility method `get_num_threads` that returns the (maximum)
number of
  threads currently in use by the virtual machine. The functionality of
  `set_num_threads` whereby it returns the previous value has been
deprecated
  and will be removed in 2.8.X.

What's Numexpr?
---------------

Numexpr is a fast numerical expression evaluator for NumPy.  With it,
expressions that operate on arrays (like "3*a+4*b") are accelerated
and use less memory than doing the same calculation in Python.

It has multi-threaded capabilities, as well as support for Intel's
MKL (Math Kernel Library), which allows an extremely fast evaluation
of transcendental functions (sin, cos, tan, exp, log...) while
squeezing the last drop of performance out of your multi-core
processors.  Look here for a some benchmarks of numexpr using MKL:

https://github.com/pydata/numexpr/wiki/NumexprMKL

Its only dependency is NumPy (MKL is optional), so it works well as an
easy-to-deploy, easy-to-use, computational engine for projects that
don't want to adopt other solutions requiring more heavy dependencies.

Where I can find Numexpr?
-------------------------

The project is hosted at GitHub in:

https://github.com/pydata/numexpr

You can get the packages from PyPI as well (but not for RC releases):

http://pypi.python.org/pypi/numexpr

Documentation is hosted at:

http://numexpr.readthedocs.io/en/latest/

Share your experience
---------------------

Let us know of any bugs, suggestions, gripes, kudos, etc. you may
have.

Enjoy data!


--
Robert McLeod
[hidden email]
[hidden email]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/numpy-discussion/attachments/20201228/f4240dab/attachment.html>

------------------------------

Subject: Digest Footer

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion


------------------------------

End of NumPy-Discussion Digest, Vol 171, Issue 39
*************************************************

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: NumPy-Discussion Digest, Vol 171, Issue 39

Robert Kern-2
On Tue, Dec 29, 2020 at 5:35 PM Brian Soto <[hidden email]> wrote:
I'm still learning proper mailing list etiquette so I'm not sure if this is where I should respond. 

FWIW, if you want to reply to conversations regularly, it's more ergonomic all around to subscribe regularly and not use the digests. But if you only occasionally want to jump in, using the digest is fine. It helps everyone if you change the subject line back to that of the thread that you are replying to and trimming the digest to just the email you are responding to.

Thank you! I hope this is helpful.
 
--
Robert Kern

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion