objax.privacy.dpsgd package¶

Computes differentially private gradients as required by DPSGD. 

Compute and print results of DPSGD analysis. 

Compute RDP of the Sampled Gaussian Mechanism. 

Compute delta (or eps) for given eps (or delta) from RDP values. 
 class objax.privacy.dpsgd.PrivateGradValues(f, vc, noise_multiplier, l2_norm_clip, microbatch, batch_axis=(0,), keygen=objax.random.Generator(seed=0), use_norm_accumulation=False)[source]¶
Computes differentially private gradients as required by DPSGD. This module can be used in place of GradVals, and automatically makes the optimizer differentially private.
 Parameters
f (Callable) –
vc (objax.variable.VarCollection) –
noise_multiplier (float) –
l2_norm_clip (float) –
microbatch (int) –
batch_axis (Tuple[Optional[int], ...]) –
keygen (objax.random.random.Generator) –
use_norm_accumulation (bool) –
 __init__(f, vc, noise_multiplier, l2_norm_clip, microbatch, batch_axis=(0,), keygen=objax.random.Generator(seed=0), use_norm_accumulation=False)[source]¶
Constructs a PrivateGradValues instance.
 Parameters
f (Callable) – the function for which to compute gradients.
vc (objax.variable.VarCollection) – the variables for which to compute gradients.
noise_multiplier (float) – scale of standard deviation for added noise in DPSGD.
l2_norm_clip (float) – value of clipping norm for DPSGD.
microbatch (int) – the size of each microbatch.
batch_axis (Tuple[Optional[int], ...]) – the axis to use as batch during vectorization. Should be a tuple of 0s.
keygen (objax.random.random.Generator) – a Generator for random numbers. Defaults to objax.random.DEFAULT_GENERATOR.
use_norm_accumulation (bool) – whether to use norm accumulation as a first step to compute the clipped gradients (more details further). This is more memory efficient and often faster when the model is very deep and uses a large batchsize.
 reshape_microbatch(x)[source]¶
Reshapes examples into microbatches. DPSGD requires that perexample gradients are clipped and noised, however this can be inefficient. To speed this up, it is possible to clip and noise a microbatch of examples, at a sight cost to privacy. If speed is not an issue, the microbatch size should be set to 1.
If x has shape [D0, D1, …, Dn], the reshaped output will have shape [number_of_microbatches, microbatch_size, D1, …, DN].
 Parameters
x (Union[jax._src.numpy.lax_numpy.ndarray, jaxlib.xla_extension.DeviceArrayBase, jaxlib.xla_extension.pmap_lib.ShardedDeviceArrayBase]) – items to be reshaped.
 Returns
The reshaped items.
 Return type
Union[jax._src.numpy.lax_numpy.ndarray, jaxlib.xla_extension.DeviceArrayBase, jaxlib.xla_extension.pmap_lib.ShardedDeviceArrayBase]
 objax.privacy.dpsgd.analyze_dp(q, noise_multiplier, steps, orders=(1.25, 1.5, 1.75, 2.0, 2.25, 2.5, 3.0, 3.5, 4.0, 4.5, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20), delta=1e05)[source]¶
Compute and print results of DPSGD analysis.
 Parameters
q (float) – The sampling rate.
noise_multiplier (float) – The ratio of the standard deviation of the Gaussian noise to the l2sensitivity of the function to which it is added.
steps (int) – The number of steps.
orders (Tuple[float, ...]) – An array (or a scalar) of RDP orders.
delta (float) – The target delta.
 Returns
eps
 Raises
ValueError – If target_delta are messed up.
 Return type
float
 objax.privacy.dpsgd.analyze_dp_sample_without_replacement(q, noise_multiplier, steps, orders=(1.25, 1.5, 1.75, 2.0, 2.25, 2.5, 3.0, 3.5, 4.0, 4.5, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20), delta=1e05)[source]¶
Compute and print results of DPSGD analysis for sample without replacement.
 Parameters
q (float) – The sampling rate.
noise_multiplier (float) – The ratio of the standard deviation of the Gaussian noise to the l2sensitivity of the function to which it is added.
steps (int) – The number of steps.
orders (Tuple[float, ...]) – An array (or a scalar) of RDP orders.
delta (float) – The target delta.
 Returns
eps
 Raises
ValueError – If target_delta are messed up.
 Return type
float
 objax.privacy.dpsgd.analyze_renyi(q, noise_multiplier, steps, orders)[source]¶
Compute RDP of the Sampled Gaussian Mechanism.
 Parameters
q (float) – The sampling rate.
noise_multiplier (float) – The ratio of the standard deviation of the Gaussian noise to the l2sensitivity of the function to which it is added.
steps (int) – The number of steps.
orders (Union[float, Tuple[float, ...]]) – An array (or a scalar) of RDP orders.
 Returns
The RDPs at all orders. Can be np.inf.
 objax.privacy.dpsgd.analyze_renyi_sample_without_replacement(q, noise_multiplier, steps, orders)[source]¶
Compute RDP of Gaussian Mechanism using sampling without replacement.
This function applies to the following schemes: 1. Sampling w/o replacement: Sample a uniformly random subset of size m = q*n. 2. ``Replace one data point’’ version of differential privacy, i.e., n is
considered public information.
Reference: Theorem 27 of https://arxiv.org/pdf/1808.00087.pdf (A strengthened version applies subsampledGaussian mechanism)  Wang, Balle, Kasiviswanathan. “Subsampled Renyi Differential Privacy and Analytical Moments Accountant.” AISTATS’2019.
 Parameters
q (float) – The sampling proportion = m / n. Assume m is an integer <= n.
noise_multiplier (float) – The ratio of the standard deviation of the Gaussian noise to the l2sensitivity of the function to which it is added.
steps (int) – The number of steps.
orders (Union[float, Tuple[float, ...]]) – An array (or a scalar) of RDP orders.
 Returns
The RDPs at all orders, can be np.inf.
 Return type
float
 objax.privacy.dpsgd.convert_renyidp_to_dp(orders, rdp, target_eps=None, target_delta=None)[source]¶
Compute delta (or eps) for given eps (or delta) from RDP values.
 Parameters
orders (Tuple[float, ...]) – An array (or a scalar) of RDP orders.
rdp (Tuple[float, ...]) – An array of RDP values. Must be of the same length as the orders list.
target_eps (Optional[float]) – If not None, the epsilon for which we compute the corresponding delta.
target_delta (Optional[float]) – If not None, the delta for which we compute the corresponding epsilon. Exactly one of target_eps and target_delta must be None.
 Returns
eps, delta, opt_order.
 Raises
ValueError – If target_eps and target_delta are messed up.
 Return type
Tuple[float, float, float]