Release Notes Pub Share

We'll start tracking the release notes here, as far as language and functionality changes.

5.0.8

Improvements

• syntax and editor improvements, see below
• fiddle now underlines declarations and subtitles
• swagger support
• swagger APIs grouped by specs

Syntax

Better support for calling rules from expressions, much like a function - see query.something below. The rule should return payload and this is what the resulting value of the expression is.

$val x = query.something()

$when query.something => (payload = {total:1, data:[{Joe:{address:"55 york st"}}]})

The sub-trace is attached to the main trace. Also, the sub-trace is not considered when looking for resulting payload.

---

New operator |> for composing expressions. This notation is suggested and will be finalized next release. The payload is carried from expression to expression, evaluated left to right. Use paranthesis to change order of composition.

$val x = query.something() |> transform.it() |> count.it())

$mock ex.ample
=> (payload = 
  query.something() |> 
  payload.data filter x => x not empty |> {
    total:sizeOf(payload), 
    data:payload[0] map (x => x.value + {name:x.key})
  }
)

This will result in something like {
  total:1,
  data: [
  {
     address:"55 york st",
     name: "Joe"
  }
  ]
}

---

Allow when to declare rules inside a class definition:

$anno (inventory = "diesel")
$class PermRole (
  @key name,
  _permissions : JSON
  ) {
  when <static> inv.listAll => (payload = query.perms() |> {total:payload.count, data:payload.data})
}

---

Also allow annotations like <static> on member rules.

5.0.x

Big changes:

• syntax
  • block notation
  • domain updates
• operational
  • cluster
    • streams distributed
    • crons distributed
    • message routing
  • distributed stream batch processing operator

Block notation

These two blocks are equivalent in functionality:

This is the old style:

$mock a.b
=> b.c2 (a=8)
=> (asdf= 3)
=> $if (asdf == 3) do.this
|=> asdf.asdf
|=> asdf.asdf
=> $else do.that
|=> isnot.t
|=> isnot.t
=> c.b
=> diesel.catch
|=> diesel.return

This is the new style:

$mock a.b {
  b.c1 (a=8)  
  (asdf= 3)
  $if (asdf == 3) {
    => asdf.asdf
    => asdf.asdf
  }
  $else {
    => isnot.t
    => isnot.t 
  }
  => c.b
  diesel.catch {
    => diesel.return
  }
}

Note that the syntax markers may act weirdly with the new style...

Also note that you cannot generally combine the old and new syntax - if you want to use block notation, start from the rule definition and be consistent.

Don't use blocks that don't belong to a message like diesel.catch or an $if/$else - they may cause weird behavior.

Always check the structure of the resulting/compiled flow using the "wiki" flag!

Cluster

Support for kube cluster embedded in the EMS build.

• dynamic cluster
• see cluster state
• see/use specific nodes

rule routing

Prefix a message with routing annotations to have it be distributed in a cluster:

$msg <route.lb> diesel.batch.int.parBatchDIST (msg?, context, items, respStreamRef, batchNo, parBatchNo)

When this event is raised with the ==> operator (fire and forget), the processing will be distributed in the cluster. Avaialble options are: lb, other - for testing, singleton, local.

Crons work in cluster

Crons are of three kinds:

• singleton, which work in the singleton node
• distributed, which work in all nodes
• local, which only fires on the local node

When creating a cron, send clusterMode which can be one of singleton, local, all

To have a cron that load-balances work across the cluster, use singleton with <rule.lb> rule routing, i.e.

Streams work in cluster

You can create a stream in a node and then send a ref to this stream to a remote node and that node is able to send data to this stream and manage the stream.

stream timeout

You can have a stream close itself with a timeout. This is good in situations where a distributed processing branch doesn't return anymore.

Diesel uses akka style "embrace failure". When the stream is done, you should compare how many items/batches were processed and take appropriate action.

Pass for instance timeoutMillis = 300000, when creating the stream, to use this feature.

Distributed stream batch operator

This is the operator (look for usages):

diesel.streaming.queryParBatched (
  job = "process ${a.alarmId} - ${a.name}",
  query = "",
  stream = streamName,
  noItems = noItems,
  batchSize = BATCHSIZESEQ,
  parBatchSize = BATCHSIZEPAR,
  queryPageMsg = "a.queryDevices",
  batchProcessorMsg = "please.processParBatch",
  individualReplies = false,
  context = {                 // this is accumulator data passed to each processor
    env: env,
    def: def
  }
  sync=false,        // for local sync debugging pass this true
  distributed=true   // note this flag for distribution of parbatch processing
  )

It will:

• query using a.queryDevices which is paginated in seqBatchSize
• splitting processing in parallel batches with parBatchSize
• process them using please.processParBatch
• send the results of processing to the stream

Internally will create a structure of streams, handlers and flows.

You need to log in to post a comment!