Better way to create multiple independent random number generators

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Better way to create multiple independent random number generators

Sebastian Berg
Hi all,

since this is going to be a new addition as part of the randomgen, I
thought I would just mention it on the mailing list. The Pull Request:

https://github.com/numpy/numpy/pull/13780

Implements a new SeedSequence object based on Robert Kern's proposal
and especially the work by Prof. O'Neill's which is included in C++.

This new API allows to create many independent random generators/random
number streams. For now this will be exposed by a new object:

```
entropy = {None, int, sequence[int]}  # None or "seed"
seed_seq = np.random.SeedSequence(entropy=entropy)

# Run 100 predictable independend streams:
for spawned_seed_seq in seed_seq.spawn(100):
    run_parallel_task(spawned_seed_seq)


# where `run_parallel_task` will do:
def run_parallel_task(seed_seq):
    # Create a BitGenerator and a Generator [1]
    bit_rng = np.random.PCG64(seed_seq)
    rng = np.random.Generator(bit_rng)
```

The beauty is that `run_parellel_task` can again use `seed_seq.spawn()`
to create another set of independent streams.

I am very happy with this new API. Right now we decided to opt for a
SeedSequence object. In the future we may opt to adding a `.spawn()`
method directly to the Generator or BitGenerator. This is mostly a
heads up, since it is a new set of API, which I believe has never been
mentioned/proposed on the mailing list.

Best,

Sebastian


[1] The new API separates the BitGenerator which creates the random
streams from the Generator which uses the random stream to give sample
distributions of random numbers providing `uniform`, or `normal`, etc.


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Better way to create multiple independent random number generators

Robert Kern-2
On Fri, Jun 21, 2019 at 11:39 AM Sebastian Berg <[hidden email]> wrote:
I am very happy with this new API. Right now we decided to opt for a
SeedSequence object. In the future we may opt to adding a `.spawn()`
method directly to the Generator or BitGenerator. This is mostly a
heads up, since it is a new set of API, which I believe has never been
mentioned/proposed on the mailing list.

We talked about the general desire for this kind of API a few years ago:


The strength of the algorithm provided by Prof O'Neill is what enables this kind of API, which is a sea change in how one can safely build reproducible parallel stochastic programs and libraries.

--
Robert Kern

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion