raftnode package

Submodules

raftnode.cli module

raftnode.config module

raftnode.config.chunks(l, n)[source]
raftnode.config.random_timeout()[source]

return random timeout number

raftnode.election module

class raftnode.election.Election(transport: raftnode.transport.Transport, store: raftnode.store.Store, queue: queue.Queue)[source]

Bases: object

ask_for_vote()[source]

ask the other nodes in the cluster to vote so that this node can become the leader

decide_vote(term: int, commit_id: int, staged: dict) → bool[source]

on receiving vote request from the candidate node, decide whether to vote for or against that node

Parameters

  • term (int) – term of the candidate node

  • commit_id (int) – latest commit_id that the that the candidate node holds

  • staged (dict) – any cached/staged data by the candidate node

Returns

True if the voter can vote in favour of the candidate node False otherwise

Return type

bool

handle_delete(payload: dict)[source]
handle_get(payload: dict) → dict[source]

Retrieve data from the database

Parameters

payload (dict) – it contains key using which it’s corresponding value will be retrieved from the database

handle_put(payload: dict) → bool[source]

Function to insert data into the database

Parameters

payload (dict) – data as received from the client

Returns

True if the data is inserted properly False otherwise

Return type

bool

heartbeat_handler(message: dict) → tuple[source]

using this function, the follower node performs checks to validate the heartbeat data received from the leader

Parameters

message (dict) – heartbeat data as sent by the leader node

Returns

term and latest commit_id of this (follower) node

Return type

tuple

increment_vote()[source]
init_timeout()[source]

initialize the timeout loop to check for missed heartbeats from the leader and start the election

reset_timeout()[source]

reset the election timeout after receiving heartbeat from the leader

send_heartbeat(peer: str)[source]

send the heartbeat the peer and analyze it’s response

Parameters

peer (str) – address of the follower node

send_vote_request(voter: str, term: int)[source]

send vote request message to the voter node this message contains the current term of this node, the latest commit id and any cached data

Parameters

  • voter (str) – address of the voter node in ip:port format

  • term (int) – current term of this node

start_election()[source]

wait for the timeout, and start the leader election

start_heartbeat()[source]

If this node is elected as the leader, start sending heartbeats to the follower nodes

timeout_loop()[source]

if this node is not the leader, wait for the leader to send the heartbeat. If heartbeat is not received by the follower, within some unit time then start the election. This loop will run endlessly

update_follower_commit(follower: str)[source]

update the followers log until it’s log is in sync with the leader’s log

Parameters

follower (str) – address of the follower node in ip:port format

raftnode.log module

class raftnode.log.Logging(log_level, **kwargs)[source]

Bases: object

get_logger()[source]

raftnode.raftnode module

Main module.

class raftnode.raftnode.RaftNode(my_ip: str, peers: list, timeout: int, **kwargs)[source]

Bases: raftnode.transport.Transport

run()[source]

start the server, add peers and election timer

start_adding_peers(peers)[source]

if peers are specified at the runtime, add them to the list of peers

start_timeout()[source]

start the election timer

start_transport()[source]

start the socket server for this node

raftnode.store module

class raftnode.store.Store(store_type: str = 'memory', data_dir: str = 'data')[source]

Bases: object

action_handler(message: dict)[source]

handle the commit and log actions sent by the leader node

Parameters

message (dict) – log/commit data as received from the leader

commit(namespace: str, delete: bool = False, **kwargs)[source]

commit the message to the database after getting atleast majority + 1 confirmations from the follower nodes

delete(term: int, payload: dict, transport, majority: int)[source]
get(payload: dict)[source]

retrieve data from the database based on the key in the payload

Parameters

payload (dict) – dictionary consisting the key using which the data needs to be retrieved from the database

put(term: int, payload: dict, transport, majority: int) → bool[source]

Insert data into the database. If this is the leader node, first broadcast the message to all the followers and wait for atleast majority + 1 confirmations. Once majority + 1 followers confirm, send out a commit message. This will instruct the followers to commit the data to their databases

Parameters

  • term (int) – term of this node

  • payload (dict) – data to be inserted into log or database

  • transport (Transport) – instance of the Transport class

  • majority (int) – how many nodes constitute the majority

send_data(message: dict, transport, confirmations: Optional[list] = None)[source]

send the log or commit data to the follower nodes and record their responses in the confirmations list

Parameters

  • message (dict) – data toe be sent to the follower nodes

  • transport (Transport) – instance of the transport class

  • confirmations (list) – list of the confirmations (initialized to False)

raftnode.transport module

class raftnode.transport.Transport(my_ip: str, timeout: int, queue: queue.Queue)[source]

Bases: object

add_peer(message: dict)[source]

This functions adds any new peers to their list of peers

Parameters

message (dict) – message received from the new peer

decode_json(msg)[source]

convert bytes object to json

Parameters

msg (bytes) – bytes object as received from the client or other node

Returns

bytes message converted to json

Return type

dict

echo(peer)[source]

This function will send the ping message to the peer at the address addr

Parameters

peer – address of the peer in ip:port format

encode_json(msg: dict) → bytes[source]

convert json to bytes object

Parameters

msg (dict) – message in dict format

Returns

json message converted to bytes

Return type

bytes

heartbeat(peer: str, message: Optional[dict] = None) → dict[source]

If this node is the leader, it will send a heartbeat message to the follower at address peer

Parameters

  • peer (str) – address of the follower in ip:port format

  • message (dict) – heartbeat message; it consists current term and address of this node (leader node)

Returns

heartbeat message response as received from the follower

Return type

dict

ping(timeout: float)[source]

send a ping message to all the peers in the peers list

Parameters

timeout (float) – time interval after which all peers will get a ping from this node

reconnect(addr: str)[source]

This function tries to connect this node to the peer at address addr 20 times. If the peer is not connected in 20 tries, it returns False

Parameters

addr (str) – address of the other peer

redirect_to_leader(message: dict)[source]

If this node is not the leader, this function will redirect the request along with the message to the leader of the cluster

Parameters

  • message – message to send to the client

  • type – dict

req_add_peer(addr: str)[source]

When this node starts, it send out a message to all the peers to add itself in their peers list

Parameters

addr (str) – address of this node in the format ip:port

send_data(peer=None, message: Optional[dict] = None)[source]

sends heartbeat data to the peer and returns response to this node

Parameters

  • peer (str) – address of the peer in ip:port format

  • message (dict) – heartbeat data message; it contains term and latest commit_id

Returns

vote response as received from the voter node

Return type

dict

serve()[source]
Parameters

election (Election) – instance of the Election class

This function starts a socket server and listen endlessly to the clients. It also checks the message types and delegates the message handling responsibility accordingly.

Message types supported are:

  • add_peer:

    this message type means that a new peer has popped up in the cluster and should be added to the peers list of this server

  • heartbeat:

    the leader, once elected, sends heartbeats to the follower nodes, notifying them that it is alive and running

  • vote_request:

    when a leader is down or there is no leader in the network, any one node randomly becomes a candidate and sends out vote request to other nodes. The other nodes can either vote in favour or against the candidate node

  • ping:

    every node pings every other node from it’s peers list to check if the peer is alive or dead. If it’s dead, this node will remove the peer from it’s peers list

  • put:

    message with type put is received from the client connected to this cluster. If this node is the leader, it will put the data into the database. If it’s not the leader, it will redirect the request to the leader node and send the leader’s response back to the client

  • get:

    message with type get is received from the client connected to this cluster. If this node is the leader, it will retrieve the data from the database and give it back to the client. If it’s not the leader, it will redirect the request to the leader node and send the leader’s response back to the client

  • data:

    this type of message is sent by the leader to the follower nodes along with the heartbeat. It contains the current term and the latest commit_id

vote_request(peer: str, message: Optional[dict] = None)[source]

sends vote request to the peer and return vote response to this node

Parameters

  • peer (str) – address of the peer in ip:port format

  • message (dict) – vote message; this will be sent to the other nodes

Returns

vote response as received from the voter node

Return type

dict

Module contents

Top-level package for raftnode.