exarl.agents.agent_vault._prioritized_replay

Module Contents

Classes

PrioritizedReplayBuffer

Class implements Prioritized Experience Replay (PER)

class exarl.agents.agent_vault._prioritized_replay.PrioritizedReplayBuffer(maxlen)

Class implements Prioritized Experience Replay (PER)

PER constructor

Parameters

maxlen (int) – buffer length

add(self, experience)

Add experiences to buffer

Parameters

experience (list) – state, action, reward, next_state, done

Returns

full_buffer (done) – True if buffer is full

get_probabilities(self, priority_scale)

Get probabilities for experiences

Parameters

priority_scale (float64) – range [0, 1]

Returns

sample_probabilities (numpy array) – probabilities assigned to experiences based on weighting factor (scale)

get_importance(self, probabilities)

Compute importance

Parameters

probabilities (numpy array) – experience probabilities

Returns

importance_normalized (numpy array) – normalized importance

sample(self, batch_size, priority_scale=1.0)

Sample experiences

Parameters

• batch_size (int) – size of batch

• priority_scale (float, optional) – range = [0, 1]. Defaults to 1.0.

Returns

samples (list) – sampled based on probabilities importance (numpy array): Importance of samples sample_indices (array): Indices of samples

set_priorities(self, indices, errors, offset=0.1)

Set priorities to experiences

Parameters

• indices (array) – sample indices

• errors (array) – corresponding losses

• offset (float, optional) – Small offset. Defaults to 0.1.

get_buffer_length(self)

Get buffer length

Returns

(int) – buffer length