Reliable Message Behavior - Revision history

Zai Lynch: fixing double redirect

2008-07-12T04:10:26Z

fixing double redirect

← Older revision		Revision as of 21:10, 11 July 2008
Line 1:		Line 1:
	#REDIRECT [[~~Reliable Message Delivery Over~~ HTTP~~#Workflow~~]]		#REDIRECT [[Certified HTTP]]

Which Linden: Merged content over to Reliable Message Delivery Over HTTP

2007-09-06T22:54:18Z

Merged content over to Reliable Message Delivery Over HTTP

Show changes

Which Linden: /* Assumptions */ Reorganized so it's clearer. Thanks, Strife, for the links.

2007-07-13T19:18:34Z

Assumptions: Reorganized so it's clearer. Thanks, Strife, for the links.

← Older revision		Revision as of 12:18, 13 July 2007
Line 3:		Line 3:
	= Assumptions =		= Assumptions =

	* Reliable ~~hosts will not be down forever. Either a clone will be brought up on different hardware, or the machine itself will reappear within a day or so.~~		* Both parties exchanging messages are Reliable Hosts (defined below)
	* Each reliable host can perform {{Wikipedia\|ACID\|w=n}} operations on the data it contains.
	* Generating a globally unique message ID is inexpensive		* Generating a globally unique message ID is inexpensive
	* We can store an ~~arbitrary~~ amount of 'small' data (such as [[UUID]]s, urls, and date/timestamps) for a "long" time.		* We can store an large amount of 'small' data (such as [[UUID]]s, urls, and date/timestamps) for a "long" time.
	* Any transaction or data handled by this system will become useless well before a "long" time has elapsed. We haven't decided yet whether ~~this~~ means hours, days, or weeks. Any data that needs to be longer-lived will be communicated to another system, so any data in the reliable messaging system that is older than a "long" time can be deleted safely.		* Any transaction or data handled by this system will become useless well before a "long" time has elapsed. We haven't decided yet whether a "long" time means hours, days, or weeks, though the [[chttp]] page mentions 15 days as a placeholder value. Any data that needs to be longer-lived will be communicated to another system, so any data in the reliable messaging system that is older than a "long" time can be deleted safely.

			; Reliable Host : A reliable host has the following properties:
			* The host has a durable data store of finite size, which can store data in such a way that it is guaranteed to be recoverable even in the face of a certain number of hardware failures.
			* The host will not be down forever. Either a clone will be brought up on different hardware, or the machine itself will reappear within a day or so.
			* The host can perform {{Wikipedia\|ACID\|w=n}} operations on the data it contains.

	= Workflows =		= Workflows =

Which Linden: /* Workflow Option 2 */ 202 is noncommittal, 204 means "no content"

2007-07-13T19:02:09Z

Workflow Option 2: 202 is noncommittal, 204 means "no content"

← Older revision		Revision as of 12:02, 13 July 2007
Line 82:		Line 82:

	# Stores the incoming request in a durable store "inbox", if it doesn't already contain a message with the same ID.		# Stores the incoming request in a durable store "inbox", if it doesn't already contain a message with the same ID.
	# Responds with an ack (an empty http response with ~~202~~ status) to the requesting server.		# Responds with an ack (an empty http response with 204 status) to the requesting server.
	# A potentially asynchronous process performs the following steps:		# A potentially asynchronous process performs the following steps:
	## Opens a transaction on the database, locking the inbox request		## Opens a transaction on the database, locking the inbox request
Line 106:		Line 106:

	The Receiver performs the following additional actions:		The Receiver performs the following additional actions:
			* When sending the ack, use status code 202 instead of 204.
	* Before tombstoning the incoming request, takes the return value from the handler method and stores it as the body of an outgoing message destined for the Response URL provided in the request.		* Before tombstoning the incoming request, takes the return value from the handler method and stores it as the body of an outgoing message destined for the Response URL provided in the request.
	* In delivering the response message, the Receiver sends a reliable message (following steps above), with retries, and tombstoning the outgoing message after receiving an ack.		* In delivering the response message, the Receiver sends a reliable message (following steps above), with retries, and tombstoning the outgoing message after receiving an ack.

Which Linden: /* Receiving a Message */ It's not no response, there's clearly some text there.

2007-07-13T19:00:49Z

Receiving a Message: It's not no response, there's clearly some text there.

← Older revision		Revision as of 12:00, 13 July 2007
Line 45:		Line 45:
	## Opens a transaction on the database, locking the inbox request		## Opens a transaction on the database, locking the inbox request
	## Calls the handler method on the receiving node:		## Calls the handler method on the receiving node:
	### <code>def handle_put(body, txn):<br/>return "~~No response~~"</code>		### <code>def handle_put(body, txn):<br/>return "My Response"</code>
	### The handler method can use the open transaction to perform actions in the database that are atomic with the receipt of the message. Any non-idempotent operation must be done atomically in this way.		### The handler method can use the open transaction to perform actions in the database that are atomic with the receipt of the message. Any non-idempotent operation must be done atomically in this way.
	## Stores the return value of the handle method as an outgoing response in the outbox, without closing the transaction		## Stores the return value of the handle method as an outgoing response in the outbox, without closing the transaction.
	## Removes the incoming request from the inbox		## Removes the incoming request from the inbox
	## Closes the transaction		## Closes the transaction

Which Linden: /* Workflows */ Added discussion, fiddled a bit with text.

2007-07-13T18:58:30Z

Workflows: Added discussion, fiddled a bit with text.

← Older revision		Revision as of 11:58, 13 July 2007
Line 82:		Line 82:

	# Stores the incoming request in a durable store "inbox", if it doesn't already contain a message with the same ID.		# Stores the incoming request in a durable store "inbox", if it doesn't already contain a message with the same ID.
	# Responds with an ack (an empty http response with ~~a 200 code~~) to the requesting server.		# Responds with an ack (an empty http response with 202 status) to the requesting server.
	# A potentially asynchronous process performs the following steps:		# A potentially asynchronous process performs the following steps:
	## Opens a transaction on the database, locking the inbox request		## Opens a transaction on the database, locking the inbox request
Line 108:		Line 108:
	* Before tombstoning the incoming request, takes the return value from the handler method and stores it as the body of an outgoing message destined for the Response URL provided in the request.		* Before tombstoning the incoming request, takes the return value from the handler method and stores it as the body of an outgoing message destined for the Response URL provided in the request.
	* In delivering the response message, the Receiver sends a reliable message (following steps above), with retries, and tombstoning the outgoing message after receiving an ack.		* In delivering the response message, the Receiver sends a reliable message (following steps above), with retries, and tombstoning the outgoing message after receiving an ack.

			== Discussion ==

			The major difference between the two options relates to the number of http requests made, and how long each is held open. Option 1 involves fewer http connections for short operations (the best case), but each one is held open for longer (the entire duration of the transaction on the recipient). Option 2 always involves 2 http connections, but each one is only open for as long as it takes to transmit and persist the bytes in the message.

			Because you cannot expect a response from an Option 2 message, the full range of http verbs like GET don't make as much sense. It's therefore a little less RESTful than Option 1.

			There is no inherent conflict in implementing both Option 1 and Option 2. A sophisticated client could provide a response url if it expects the message handling to take a long time. Upon receipt of the request, the server could decide whether to deliver a return message in the response, or to immediately return a 202.

Which Linden: Corrected option 1

2007-07-13T18:47:30Z

Corrected option 1

← Older revision		Revision as of 11:47, 13 July 2007
Line 25:		Line 25:
	# A potentially asynchronous process performs the following steps:		# A potentially asynchronous process performs the following steps:
	## Retrieves the request from the outbox.		## Retrieves the request from the outbox.
	## Performs the HTTP request specified by the outbox request and waits for ~~an ack~~.		## Performs the HTTP request specified by the outbox request and waits for a response.
	## If ~~an ack~~ is not forthcoming, for whatever reason, the process retries after a certain period.		## If a response is not forthcoming, for whatever reason, the process retries after a certain period.
	## If the server sends an error code that indicates that the reliable message will never complete (e.g. 501), or a long timeout expires indicating that an absurd amount of time has elapsed, the method throws an exception.		## If the server sends an error code that indicates that the reliable message will never complete (e.g. 501), or a long timeout expires indicating that an absurd amount of time has elapsed, the method throws an exception.
	# '~~Tombstone~~' the message in the outbox, which essentially marks the message as having been ~~acked~~, so that if the application resumes again, it doesn't resend.		# Opens a transaction on the durable store
			# Stores the incoming response in a durable inbox.
			# 'Tombstones' the message in the outbox, which essentially marks the message as having been received, so that if the application resumes again, it doesn't resend.
			# Closes the transaction on the durable store
			# If the response contains a header indicating a confirmation url on the recipient, performs an HTTP DELETE on the resource to ack the incoming message.

	~~We don't have any~~ explicit semantics for the response body.		There are no explicit semantics for the response body, like HTTP itself. The content will vary depending on the application.

	=== Receiving a Message ===		=== Receiving a Message ===
Line 47:		Line 51:
	## Closes the transaction		## Closes the transaction
	# Discovers a new item in the outbox, responds to the incoming http request with the response from the outbox.		# Discovers a new item in the outbox, responds to the incoming http request with the response from the outbox.

			There is no "tombstoning" of the message in the outbox.

	=== Performing a Job ===		=== Performing a Job ===

Which Linden: /* Performing a Job */

2007-07-12T23:11:42Z

Performing a Job

← Older revision		Revision as of 16:11, 12 July 2007
Line 87:		Line 87:
	=== Performing a Job ===		=== Performing a Job ===

	A job differs from a message in that it is expected to take a "long" time, and return a result at the end. In Option 2, the job requires a separate http connection to report on its completion.		A job differs from a message in that it is expected to take a "long" time, and return a result at the end, essentially an RPC. In Option 2, the job requires a separate http connection to report on its completion.

	To perform a job, the API looks like:		To perform a job, the API looks like:

Strife Onizuka: /* Assumptions */

2007-07-11T07:44:39Z

Assumptions

← Older revision		Revision as of 00:44, 11 July 2007
Line 4:		Line 4:

	* Reliable hosts will not be down forever. Either a clone will be brought up on different hardware, or the machine itself will reappear within a day or so.		* Reliable hosts will not be down forever. Either a clone will be brought up on different hardware, or the machine itself will reappear within a day or so.
	* Each reliable host can perform ACID operations on the data it contains.		* Each reliable host can perform {{Wikipedia\|ACID\|w=n}} operations on the data it contains.
	* Generating a globally unique message ID is inexpensive		* Generating a globally unique message ID is inexpensive
	* We can store an arbitrary amount of 'small' data (such as ~~UUIDs~~, urls, and date/timestamps) for a "long" time.		* We can store an arbitrary amount of 'small' data (such as [[UUID]]s, urls, and date/timestamps) for a "long" time.
	* Any transaction or data handled by this system will become useless well before a "long" time has elapsed. We haven't decided yet whether this means hours, days, or weeks. Any data that needs to be longer-lived will be communicated to another system, so any data in the reliable messaging system that is older than a "long" time can be deleted safely.		* Any transaction or data handled by this system will become useless well before a "long" time has elapsed. We haven't decided yet whether this means hours, days, or weeks. Any data that needs to be longer-lived will be communicated to another system, so any data in the reliable messaging system that is older than a "long" time can be deleted safely.

Which Linden: /* Assumptions */

2007-07-11T06:36:35Z

Assumptions

← Older revision		Revision as of 23:36, 10 July 2007
Line 7:		Line 7:
	* Generating a globally unique message ID is inexpensive		* Generating a globally unique message ID is inexpensive
	* We can store an arbitrary amount of 'small' data (such as UUIDs, urls, and date/timestamps) for a "long" time.		* We can store an arbitrary amount of 'small' data (such as UUIDs, urls, and date/timestamps) for a "long" time.
	* Any transaction or data handled by this system will become useless well before a "long" time has elapsed. We haven't decided yet whether this means hours, days, or weeks. Any data that needs to be longer-lived will be communicated to another system.		* Any transaction or data handled by this system will become useless well before a "long" time has elapsed. We haven't decided yet whether this means hours, days, or weeks. Any data that needs to be longer-lived will be communicated to another system, so any data in the reliable messaging system that is older than a "long" time can be deleted safely.

	= Workflows =		= Workflows =