# Instructions to use NATS

## STRICT RULE: No Go Legacy Code

**ABSOLUTELY NO LEGACY CODE FROM GO ALLOWED!!!!**

Lua NATS client does NOT work like Go NATS:

- No goroutines needed (Lua coroutines block and return)
- No channels needed (direct function calls return values)
- No mutexes needed (single-threaded execution)
- No async I/O wrappers needed (blocking socket reads return directly)

## Error Handling Pattern

**All public functions return a single value - response or error object:**

```lua
local api = require "message/nats/nats-api"
local E = api

-- Error object structure:
-- {error = true, code = "error_code", message = "full error message"}

-- Error object creation helper:
E.err(message, code) -- returns {error = true, code = code, message = message}

-- Function calls:
local response = js:streamInfo(streamName)
if response.error then
    print("Error: " .. response.message)
    return
end
-- Use response...

-- Error constants (from nats-api.lua):
E.errStreamNotFound = "jetstream: stream not found"
E.codeStreamNotFound = "stream_not_found"
-- ... (and more)
```

**Key patterns:**

1. Single return value - no `nil, err` tuples
2. Check `response.error` (boolean) to detect errors
3. Access `response.message` for error text
4. Access `response.code` for error code
5. Use `E.err(E.errStreamNotFound, E.codeStreamNotFound)` to create error objects

**No fallbacks or defaults:**

```lua
-- BAD (Go-style):
local response, err = js:streamInfo(name)
if err then return nil, err end

-- GOOD (Lua-style):
local response = js:streamInfo(name)
if response.error then return response end
```

## STRICT RULE: No Loops in processIncoming()

**CRITICAL: processIncoming() MUST NOT have a loop!**

The blocking socket read + respMap pattern handles everything automatically:

1. `sock:receive()` **blocks** until data arrives (no busy-wait loops)
2. Parser calls `processMsg()` callback for each message
3. `processMsg()` checks if subject starts with `_INBOX.`, extracts token, updates `respMap[token].done = true`
4. `waitForResponse()` helper loops calling `processIncoming()` until `respMap[token].done == true`

**Where THE ONLY loop goes:** In `waitForResponse()` helper function - it calls `processIncoming()` repeatedly until the specific response arrives or timeout. Both `RequestWithContext()` and `RequestMsgWithContext()` call this helper.

**Why this works:**

- Lua coroutines block and return - no goroutines needed
- Single-threaded - no mutexes needed
- Blocking reads return directly when data arrives - no async wrappers needed
- respMap isolates concurrent requests by unique token

## Subscribe Confirmation Pattern

**subscribe() uses PING/PONG for server confirmation:**

When you call `nc:subscribe("foo", callback)`, the function MUST block until the server confirms the subscription is active. Otherwise, if you publish immediately, the message might be lost because the server hasn't processed the SUB command yet.

**How it works:**

1. `subscribe()` sends `SUB foo _EMPTY_ 5\r\nPING\r\n` in one `socketWrite()` call
2. Loops calling `processIncoming()` until `pongCount` increments
3. `processPong()` increments `pongCount` when PONG arrives
4. Server processes SUB first, then responds with PONG
5. When PONG arrives, `subscribe()` returns

**Code pattern:**

```lua
-- In subscribe():
self:socketWrite(proto .. api.PING_SEND)  -- Send SUB + PING together
local previousPongCount = self.pongCount
repeat
    self:processIncoming(true, false)      -- Process messages
    if self.error then
        self.subscriptions[sid] = nil
        return
    end
until self.pongCount ~= previousPongCount  -- Wait for PONG
return sub
```

**Why pongCount instead of checking lastOp:**

- `processIncoming()` processes ALL pending messages in one call
- `lastOp` is the LAST message processed, not the PONG we're waiting for
- Other messages (MSG, INFO, etc.) might arrive before PONG
- `pongCount` is a counter that increments for each PONG - reliable tracking

**Why Go's kickFlusher won't work in Lua:**

Go uses:

- Background goroutine `flusher()` that continuously flushes buffer
- `kickFlusher()` signals the goroutine to flush NOW
- Asynchronous - doesn't wait for confirmation

Lua (single-threaded, no goroutines):

- Can't use background goroutines
- Need synchronous confirmation
- PING/PONG pattern provides confirmation

**Only subscribe() needs PONG confirmation:**

- `subscribe()` - Timing matters! You publish immediately after, need confirmation subscription is active
- `unsubscribe()` - Local cleanup sufficient (we mark `closed=true` immediately, `processMsg()` checks this)
- `close()` - Hard socket close, no confirmation needed
- `publish()` - Fire and forget, no confirmation needed

## Lua NATS Client Architecture

### Callback System

**Callbacks ARE needed for:**

- **Regular subscriptions** - When you `subscribe("foo.bar", callback)`, you want to process messages as they arrive over time. This is the pub/sub pattern where multiple messages come in asynchronously.

**Callbacks are NOT needed for:**

- **Request/Response** - Response messages arrive on `_INBOX.token` subjects. `processMsg()` checks the subject directly, extracts the token, and updates `respMap[token].done = true`. The `_INBOX.*` subscription has `nil` callback - all work is done in `processMsg()`.

### Message Ordering

**Other messages CAN arrive between request and response:**

1. **Heartbeat messages** - NATS server sends `NATS/1.0 100 Idle Heartbeat` periodically
2. **Flow control messages** - `NATS/1.0 100 FlowControl Request` for JetStream consumer push
3. **Messages on other subscriptions** - If you have other active subscriptions, their messages arrive
4. **INFO updates** - Server may send updated INFO during connection

**How respMap handles this:**

- Each request gets unique token in `respMap[token] = {done, payload, error}`
- Wildcard subscription `_INBOX.*` receives ALL responses (with `nil` callback)
- `processMsg()` checks if subject starts with `_INBOX.`, extracts token, sets `respMap[token].done = true`
- `RequestWithContext()` loops calling `processIncoming()` until its response is ready
- Other messages are processed normally (heartbeats logged, flow control handled, subscriptions called)

## Installing NATS Mac OS

NATS configuration file and installation instructions can be found in /nc-devops/conf/nats.conf. Conf file needs to be moved into /private/etc/nats.conf. Restart the NATS server after the configuration file is in place.

You can install NATS with brew with following command.

```bash
brew install nats-server
brew tap nats-io/nats-tools
brew install nats-io/nats-tools/nats # for nats-cli, binary in /opt/homebrew/bin/nats
```

```conf
#		<key>ProgramArguments</key>
#		<array>
#			<string>/opt/homebrew/opt/nats-server/bin/nats-server</string>
#			<string>-c</string>
#			<string>/private/etc/nats/nats.conf</string>
#		</array>
```

Previously shown line needs to be copied into the file `homebrew.mxcl.nats-server.plist` after `<array></array>`.

File locations:

- in M1: code /opt/homebrew/Cellar/nats-server/2.10.16/homebrew.mxcl.nats-server.plist
- in x64: code /usr/local/Cellar/nats-server/2.10.16/homebrew.mxcl.nats-server.plist
- it copies file on every brew services restart to /Users/<yourusername>/Library/LaunchAgents/homebrew.mxcl.nats-server.plist

After copying, do following steps.

Create folder "/private/etc/nats":

```bash
sudo mkdir -p /private/etc/nats
```

Set ownership to your user for the folder:

```bash
sudo chown -R $(whoami) /private/etc/nats/
```

Create folder for JetStream.

```bash
mkdir -p /private/etc/nats/jetstream
```

Create nats-server.log file for logging server information:

```bash
touch /private/etc/nats/nats-server.log
```

Copy proper config file from nc-devops/conf/nats/nats.conf to /private/etc/nats/:

```bash
cp /Volumes/nc/nc-devops/conf/nats/nats.conf /private/etc/nats/nats.conf
```

After completing previous steps restart nats-server:

```bash
brew services restart nats-server
```

If everything goes finem, a following command should print no errors:

```bash
sudo nats-server -c /private/etc/nats.conf
```

You can also see if server has properly boot up with a following command:

```bash
tail -f /private/etc/nats/nats-server.log
```

## Testing that NATS server works

Following commands can be run to test NATS server.

Check that connection is healthy:

```bash
nats server check connection
```

You can try messaging by opening two terminal windows.

In Window 1 you subscribe to "test" with following command:

```bash
nats sub test
```

In Window 2 you send message to "test" with following command:

```bash
nats pub test 'Hello, NATS'
```

If everything works you will receive message 'Hello, NATS'.

## NATS JetStream object store

To use an object store, ensure that JetStream is enabled. You can verify it with the following command:

```bash
nats account info
```

If authorization is on, NATS account info needs credentials. Here is an example command:

```bash
nats account info --server "nats://username:password@localhost:4222"
```

The output should be something like this:

```conf
Connection Information:

               Client ID: 6
               Client IP: 127.0.0.1
                     RTT: 64.996µs
       Headers Supported: true
         Maximum Payload: 1.0 MiB
           Connected URL: nats://127.0.0.1:4222
       Connected Address: 127.0.0.1:4222
     Connected Server ID: ND2XVDA4Q363JOIFKJTPZW3ZKZCANH7NJI4EJMFSSPTRXDBFG4M4C34K

JetStream Account Information:

           Memory: 0 B of Unlimited
          Storage: 0 B of Unlimited
          Streams: 0 of Unlimited
        Consumers: 0 of Unlimited
```

Use the following commands to use the JetStream object-store.

Delete a bucket called "work-center-test":

```bash
nats --trace object del work-center-test
```

Create a bucket called "work-center-test":

```bash
nats --trace object add work-center-test
```

Create a file called "testfile.json" and add simple contents:

```bash
echo '{ "message": "Hello world!" }' > testfile.json
```

Put a file called "testfile" to a bucket "work-center-test":

```bash
nats --trace object put work-center-test testfile.json
```

Put a file called "testfile" to the bucket and give it the name "myname":

```bash
nats --trace object put --name myname work-center-test testfile.json
```

List objects in "work-center-test":

```bash
nats --trace object ls work-center-test
```

Get file "testfile" from "work-center-test":

```bash
nats --trace object get work-center-test testfile.json
```

Get file "testfile" from "work-center-test" to a specific output path:

```bash
nats --trace object get work-center-test --output /temp/testfile.json testfile.json
```

Remove an object from the bucket "work-center-test":

```bash
nats --trace object rm work-center-test testfile.json
```

Get information about bucket "work-center-test":

```bash
nats --trace object info work-center-test
```

Watch changes for "work-center-test":

```bash
nats --trace object watch work-center-test
```

Seal "work-center-test":

```bash
nats --trace object seal work-center-test
```

Delete "work-center-test":

```bash
nats --trace object rm work-center-test
```

## Using NATS with JavaScript and nats.ws library

### Simple setup

Following code connects to NATS instance.

```javascript
const nats = await connect({
	servers: 'yourserver',
	user: 'youruser',
	pass: 'yourpassword'
})
```

Following code subscribes to 'testnats'-subject.

```javascript
// Subscribe to the "testnats" subject
const stringCodec = StringCodec()
const sub = nats.subscribe('testnats')
for await (const msg of sub) {
	console.log(`Received a message: ${stringCodec.decode(msg.data)}`)
	natsMessage.value += `Received a message: ${stringCodec.decode(msg.data)} - `
}
```

Test with command

```command
nats pub testnats "Hello, NATS test message" -s nats://127.0.0.1:4222
```

The following line should appear in Grid test: "wss://demo.nats.io:8443 - Received a message: Hello, NATS test message -"

### Setup with wildcards

NATS supports two types of wildcards. These can be used to match subobjects.

#### Wildcard \*

Wildcard '\*' matches only one level of subobject for example "object.subobject1"

In code presented in simple setup change line with nats.subscribe to the following:

```javascript
const sub = nats.subscribe('testnats.*')
```

Try with the following message. You can replace ".test" with any string:

```bash
nats pub testnats.test "Hello this is a test" -s nats://127.0.0.1:4222
```

### Following example won't work without also subscribing to 'testnats'

```bash
nats pub testnats "Hello this is a test" -s nats://127.0.0.1:4222
```

#### Wildcard <

'<' matches multiple levels of subjects for example "object.subobject1.subobject2"

To test it change nats.subscribe to the following:

```javascript
const sub = nats.subscribe('testnats.<')
```

Try with the following message. You can replace ".test1" and ".test2" with any string:

```bash
nats pub testnats.test1.test2 "Hello this is test" -s nats://127.0.0.1:4222
```

### Subscribing to multiple subjects

Here is a working example how to subscribe multiple subjects on the same NATS-instance.
This code should receive messages sent to both "testnats" and "testnats.\*".

```javascript
// Subscribe to the "testnats" subject
const stringCodec = StringCodec()
const sub1 = nats.subscribe('testnats')
const sub2 = nats.subscribe('testnats.*')

async function printMsgs(subscription) {
	let subj = subscription.getSubject()
	console.log(`Listening for ${subj}`)
	const c = 13 - subj.length
	const pad = ''.padEnd(c)
	for await (const message of subscription) {
		console.log(`[${subj}]${pad} #${subscription.getProcessed()} - ${message.subject} ${message.data ? ' ' + stringCodec.decode(message.data) : ''}`)
		natsMessage.value += `Received a message on ${message.subject}: ${stringCodec.decode(message.data)} - `
	}
}
// Ensure that the messages are being processed
printMsgs(sub1)
printMsgs(sub2)
```

**Do not do it the following way**. In this example, only messages from "testnats" are shown. It will not show messages from "testnats.\*" because the code is stuck on the first async call.

```javascript
// Subscribe to the "testnats" subject
const sc = StringCodec()

// Subscribe to the "testnats" subject
const sub1 = nats.subscribe('testnats')
;(async () => {
	for await (const msg of sub1) {
		console.log(`Received a message on testnats: ${sc.decode(msg.data)}`)
		natsMessage.value += `Received a message on testnats test1: ${sc.decode(msg.data)} `
	}
})()

// Subscribe to the "testnats.*" subject
const sub2 = nats.subscribe('testnats.*')
;(async () => {
	for await (const msg of sub2) {
		console.log(`Received a message on testnats.*: ${sc.decode(msg.data)}`)
		natsMessage.value += `Received a message on testnats.* test2: ${sc.decode(msg.data)} `
	}
})()
// this promise indicates the client closed
const done = nats.closed()
// do something with the connection...
// close the connection
await nats.close()
```

## Using NATS JetStream Object Storage with JavaScript and nats.ws

Following instructions give simple examples how to use ObjectStorage with JavaScript: <https://natsbyexample.com/examples/os/intro/deno>
Here are some examples how to use JetStream with nats.ws library.

### Getting JetStream object

First you need to get JetStream object.

```javascript
const jetStream = nats.jetstream()
```

### Creating JSON object and encoding into uint8Array

In order to store something into bucket it needs to be in uint8Array format.
Following code creates a JSON object, converts it to string and then encodes it to uint8Array:

```javascript
const jsonObject = {
	name: 'I am a test json object',
	description: 'This is a test description',
	otherData: 'example data'
}

const jsonString = JSON.stringify(jsonObject)
const encoder = new TextEncoder()
const uint8Array = encoder.encode(jsonString)
```

### Putting object into Object Storage

As we have uint8Array encoded, we can use following code to put it into a bucket in Object Storage.

```javascript
let info = await objectStore.putBlob({ name: 'testObject.json', description: 'large data' }, uint8Array)
console.log(`added entry ${info.name} (${info.size} bytes)- '${info.description}'`)
```

### Getting object from Object Storage

Following code gets 'testObject.json' from Object Store. It prints object length and decodes data into string.
It then parses string into JSON object and prints it.

```javascript
let data = await objectStore.getBlob('testObject.json')
console.log(`data has ${data?.length} bytes`)

const decoder = new TextDecoder()
const decodedString = decoder.decode(data)

const retrievedJsonObject = JSON.parse(decodedString)
console.log('Parsed JSON object:', retrievedJsonObject)
```

## Using Wireshark to monitor NATS traffic

- bi --cask wireshark (brew install)
- open Wireshark
  - select Loopback: lo0
  - add filter 'port 4222' to see only nats traffic

![alt text](image/wireshark-filter.png)

- start from nats.md: nats --trace object del work-center-test
- see the traffic in Wireshark
  - select some row
  - right click and select follow TCP stream
  - Wireshark opens a new 'follow TCP stream' -window with the traffic

![alt text](image/wireshark-follow-tcp.png)
