tkuhrt (Wed, 18 Apr 2018 16:45:02 GMT):
Discuss implementation details of Hyperledger Caliper

tkuhrt (Wed, 18 Apr 2018 16:45:02 GMT):
Used for contributors of Hyperledger Caliper to discuss implementation details

klenik (Wed, 18 Apr 2018 18:56:21 GMT):
Has joined the channel.

klenik (Wed, 18 Apr 2018 19:22:20 GMT):
So as the first topic of the channel (hopefully others will also join soon), I would suggest to use the Common Connection Profile feature of the Fabric Node SDK (1.1.0). This would replace a LOT of code that currently assembles the object graphs needed to submit transactions, join channel, etc. Opinions?

wangdong (Thu, 19 Apr 2018 03:57:24 GMT):
Has joined the channel.

wangdong (Thu, 19 Apr 2018 03:58:07 GMT):
yes

wangdong (Thu, 19 Apr 2018 03:58:12 GMT):
agree

wangdong (Thu, 19 Apr 2018 03:58:39 GMT):
And I wonder if we can add some debug info?

wangdong (Thu, 19 Apr 2018 03:59:05 GMT):
As the framework grows, it will be harder and harder to debug the issue

nkl199 (Thu, 19 Apr 2018 08:21:35 GMT):
Has joined the channel.

nkl199 (Thu, 19 Apr 2018 08:29:31 GMT):
+1 for common connection profile. The Composer plugin, which is targeting Fabric, relies on the existence of that. The plugin also uses docker exec commands to create the Fabric configuration (create/join channels etc) which was done to remove any "housekeeping" within the target platform creation side, rather than using e2e code to perform the same. Might be worth considering if the pure Fabric tests could do similar, as it would remove a large amount of code. This would be important moving forward, where there are likely to multiple Fabric versions with breaking API changes to contend with.

klenik (Thu, 19 Apr 2018 13:08:59 GMT):
wow, I've never thought of delegating the "init" phase to the actual deployed Fabric binaries, nice

klenik (Thu, 19 Apr 2018 13:11:11 GMT):
One remark: if I parsed your code correctly, it can execute the commands only in locally running docker containers (with ID of `hostname`)

klenik (Thu, 19 Apr 2018 13:12:39 GMT):
to execute the commands on remote containers, you also have to specify `-H ` (and of course properly set up the remote docker daemon, but that's not the clients problem)

klenik (Thu, 19 Apr 2018 13:21:20 GMT):
and also use the public address of the orderer in the command

nkl199 (Thu, 19 Apr 2018 14:21:51 GMT):
:thinking: ... yup! Not tried executing the commands on remote containers ... i feel an experiment coming up :D

klenik (Thu, 19 Apr 2018 14:41:01 GMT):
yeah, there are some scenarios to consider :) - Fabric components are running in remote docker containers: setting up remote docker daemon, configuring secure communication - Fabric components are running in remote VMs directly: using remote SSH commands, enabling SSH on remote VM, using SSH key(s) on client side maybe the SDK is the more secure way to go right now, but we definitely need to consider the above scenarios, especially for testing purposes during CI :)

JayJong (Mon, 23 Apr 2018 09:01:58 GMT):
Has joined the channel.

Haojun (Tue, 24 Apr 2018 08:12:37 GMT):
Has joined the channel.

Haojun (Tue, 24 Apr 2018 08:30:46 GMT):
+1 for common connection profile. I've recreate an issue on github to record this idea.

Haojun (Tue, 24 Apr 2018 08:34:35 GMT):
@klenik I think cello is woking on the blockchain management/operation, although I haven't studied it yet

Haojun (Tue, 24 Apr 2018 08:48:41 GMT):
Great to have this new channel to discuss Caliper here. Unfortunately, due to some personal issues, I can't put much time in this project recently. So sorry for the possible late reply

klenik (Tue, 24 Apr 2018 10:00:41 GMT):
@Haojun I've looked into cello a bit, it's a promising provisioning tool. But it's still not clear to me what kind of tests we will need: 1) There are some components that can be unit tested: rate controllers, Tx result processing (report generation), steps taken by the workflow based on the config file 2) The interesting/challenging area is the integration testing. The easy option for basic e2e benchmark scenarios is a local docker network. But the zookeeper client scenarios don't really fit into this "local approach" 3) Finally, the boogeyman of testing, performance testing. There was some chatter in the PSWG about a Linux Foundation cluster for the large scale testing of Hyperledger projects. Do you know anything about this?

klenik (Wed, 25 Apr 2018 22:44:18 GMT):
@Haojun @nkl199 have you ever run into the issue #43? I noticed the problem a while ago, but I could only now formulate a hypothesis :thinking:

nkl199 (Thu, 26 Apr 2018 10:08:27 GMT):
@klenik - not to my knowledge, but I will try to investigate your scenario to see if I can replicate it

Haojun (Sat, 28 Apr 2018 03:33:11 GMT):
@klenik It is caused by a bug in runDuration, and I've submitted a [PR](https://github.com/hyperledger/caliper/pull/50) to fix it

Haojun (Sat, 28 Apr 2018 03:33:11 GMT):
@klenik It was caused by a bug of runDuration(), and I've submitted a [PR#50](https://github.com/hyperledger/caliper/pull/50) to fix it

mwagner (Tue, 01 May 2018 13:28:33 GMT):
Has joined the channel.

nkl199 (Tue, 01 May 2018 15:37:40 GMT):
Should we now be considering publishing the src to npm? This would be pretty cool for anyone wanting to pull it as a dependancy and run tests instead of cloning the entire repository ...

Ryan2 (Wed, 02 May 2018 12:29:50 GMT):
Has joined the channel.

Haojun (Thu, 03 May 2018 02:42:43 GMT):
@nkl199 There are some important PRs for iroha as well as eslint compatibility. So I'd like to finish them before publishing the code. Reasonable?

nkl199 (Thu, 03 May 2018 08:33:16 GMT):
@Haojun yh - for sure. As you mention, I think there are several items that need to be sequenced. It would be great to get all the lint checking in as well as any key updates before the travis build is pulled and enabled. The publishing to npm, whilst nice, can be added to the Travis build at a later stage once the basic regression checks are enabled. I've noticed that there are benchmarks for sawtooth and iroha, so I will add those in as Travis build items too ... then it 'should' be a case of adding a licence checker and extending unit tests.

nkl199 (Thu, 03 May 2018 09:05:30 GMT):
I'm looking at the current benchmark tests, with a mind to run each available benchmark in the Travis build, I would like to split each benchmark 'type' (fabric/composer/iroha/sawtooth) into separate folders. This will: 1) Make it very clear to a user where each benchmark example resides 2) Let us use the existing test.js file to run each benchmark in an automated framework So the question is ... are there any objections to this? I am aware that the linting edits will also need to be applied to the newly located files... I've seen that there is a PR with linting fixes for drm/simple .. so I will wait for that PR to be merged before moving on with this

klenik (Thu, 03 May 2018 10:08:25 GMT):
I'll extend my PR with Fabric linting fixes in an hour. @nkl199 if you restructure the benchmark folders, could you make a separate go and node subfolder for Fabric? :)

nkl199 (Thu, 03 May 2018 10:08:51 GMT):
@klenik sure :thumbsup:

tungdt_socoboy (Sat, 05 May 2018 11:40:30 GMT):
Has joined the channel.

Haojun (Mon, 07 May 2018 09:02:16 GMT):
@nkl199 What's the meaning of 'split each benchmark type into separate folders'? Since a benchmark such as simple can run either under fabric or sawtooth or iroha......

wangdong (Mon, 07 May 2018 09:09:01 GMT):
@nkl199 +1 for the separating type. It will be more clearer.

wangdong (Mon, 07 May 2018 09:17:19 GMT):
And I got one issue. I wonder when you update, would you please also give your patch here so that more people can invole and maybe some more discuss?

klenik (Mon, 07 May 2018 13:25:41 GMT):
Good point that the benchmarks have some platform independent parts. Something like the following hierarchy could handle this (very PoC): ` simple/ ├── platform independent files (e.g. callback module) ├── platform1 (e.g. fabric)/ │ ├── platform specific, but version independent files (e.g. network config file) │ ├── v1.0 (or any sub ID) │ │ └── version specific files (e.g. chaincode without CID and ABAC) │ └── v1.1 │ └── version specific files (e.g. chaincode with CID and ABAC or Nodejs chaincode) ├── platform2/ │ └── ... `

klenik (Mon, 07 May 2018 13:25:41 GMT):
Good point that the benchmarks have some platform independent parts. Something like the following hierarchy could handle this (very PoC): ``` simple/ ├── platform independent files (e.g. callback module) ├── platform1 (e.g. fabric)/ │ ├── platform specific, but version independent files (e.g. network config file) │ ├── v1.0 (or any sub ID) │ │ └── version specific files (e.g. chaincode without CID and ABAC) │ └── v1.1 │ └── version specific files (e.g. chaincode with CID and ABAC or Nodejs chaincode) ├── platform2/ │ └── ... ```

klenik (Mon, 07 May 2018 13:25:41 GMT):
Good point that the benchmarks have some platform independent parts. Something like the following hierarchy could handle this (very PoC): ```simple/ ├── platform independent files (e.g. callback module) ├── platform1 (e.g. fabric)/ │ ├── platform specific, but version independent files (e.g. network config file) │ ├── v1.0 (or any sub ID) │ │ └── version specific files (e.g. chaincode without CID and ABAC) │ └── v1.1 │ └── version specific files (e.g. chaincode with CID and ABAC or Nodejs chaincode) ├── platform2/ │ └── ... ```

Haojun (Tue, 08 May 2018 00:57:53 GMT):
OK I see, that looks clearer. BTW, we can also remove test reports to a report folder.

Haojun (Tue, 08 May 2018 01:03:46 GMT):
Hi, can anyone help to check [running composer test with multiple clients](https://github.com/hyperledger/caliper/pull/59)? I was always stuck at 'starting network', may be due to network issue behind the proxy.

Haojun (Tue, 08 May 2018 01:03:46 GMT):
Hi, can anyone help check [running composer test with multiple clients](https://github.com/hyperledger/caliper/pull/59)? I was always stuck at 'starting network', may be due to network issue behind the proxy.

nkl199 (Tue, 08 May 2018 07:50:53 GMT):
I'd help but i submitted it :joy:

nkl199 (Tue, 08 May 2018 09:16:24 GMT):
with regards to reviews - I'm more than happy to assist with any pending PR reviews :thumbsup:

klenik (Thu, 10 May 2018 13:38:31 GMT):
Based on Issue 39, it is not trivial to run Caliper in debug mode (even when the inspector conflict will be fixed), since usually the IDEs will only attach to the main process, and not to the forked children. It would be useful to provide an entry point directly to the local client implementation that can be used during development to run and debug a single client only, since most of the complex code run inside the children processes anyway. Opinions?

gut (Wed, 16 May 2018 13:26:33 GMT):
Has joined the channel.

arthurmsouza (Thu, 17 May 2018 13:20:55 GMT):
Has joined the channel.

gut (Fri, 18 May 2018 10:38:25 GMT):
Has left the channel.

robwahl (Sat, 19 May 2018 16:08:09 GMT):
Has joined the channel.

Haojun (Tue, 22 May 2018 08:12:46 GMT):
Hi @nkl199 @klenik and others, please take a look at this [PR#71](https://github.com/hyperledger/caliper/pull/71#). The PR is for a smallbank benchmark with uses the concept of *batch* from Sawtooth. I'd like to discuss how to handle it with Fabric, since as I known Fabric does not support batch now.

nkl199 (Tue, 22 May 2018 09:43:12 GMT):
The

nkl199 (Tue, 22 May 2018 09:43:12 GMT):
Composer cheats a little on this - we have API methods for addAll() and updateAll(), which loop on a single add operation within fabric

klenik (Tue, 22 May 2018 10:46:21 GMT):
I'm not familiar with the Sawtooth batch concept, so correct me if I'm wrong. Since Fabric doesn't support batches, I think there are 3 options: 1) Push batching to chaincode level: this could ensure atomic batches, but complicates the chaincode (but doesn't affect Caliper). 2) Implement batching on client level: atomic batches are problematic ( @nkl199 is this handled by Composer?), the client simulates the batch with multiple transaction invokes (but doesn't affect Caliper). 3) Caliper provides a syntactic sugar for Fabric batches, which is essentially the same as 2), but inside Caliper. Since 3) also requires that the client submits multiple transaction parameters, you can't really make batches transparent to the client. So why not implement it all on client callback side? I think a benchmark artifact (client module or chaincode) should clearly reflect, that batch simulation is needed for the use case, unless supported by the platform. So I support options 1) and 2). Opinions?

nkl199 (Tue, 22 May 2018 12:25:02 GMT):
I would also be in favour of above options 1 or 2, with a strong preference to option 1. I don't think that Caliper should be responsible for adding missing API methods of a target blockchain platform. Not least because I would hope that someone using Caliper will be using their own chaincode that operates 'out of the box' on a target blockchain platform. @klenik to answer your question, Composer will use the Fabric API to perform multiple `shim.putState()` calls on a passed collection

Haojun (Wed, 23 May 2018 01:10:40 GMT):
Option 2/3 is easier to implement. The only question is should we force the invoking chain, which means the later transaction should not be submitted until it's previous txn has been committed on the ledger? Regarding option 1, the chaincode would be complicated in order to handle a series of operations on the same account . And with option 2, would a batch of txns be counted as multiple txns or only one transaction? The counting method should be the same for different blockchain systems.

Haojun (Wed, 23 May 2018 01:10:40 GMT):
Option 2/3 is easier to implement (I'm not sure what's the difference of 2 and 3?). The only question is should we force the invoking chain, which means the later transaction should not be submitted until it's previous txn has been committed on the ledger? Regarding option 1, the chaincode would be complicated in order to handle a series of operations on the same account . And with option 2, would a batch of txns be counted as multiple txns or only one transaction? The counting method should be the same for different blockchain systems.

Haojun (Wed, 23 May 2018 01:10:40 GMT):
Option 2/3 is easier to implement (I'm not sure what's the difference of 2 and 3?). The only question is should we force the invoking chain for txns in a batch, which means the later transaction should not be submitted until it's previous txn has been committed on the ledger? Regarding option 1, the chaincode would be complicated in order to handle a series of operations on the same account . And with option 2, would a batch of txns be counted as multiple txns or only one transaction? The counting method should be the same for different blockchain systems.

Haojun (Wed, 23 May 2018 01:10:40 GMT):
Option 2/3 is easier to implement (I'm not sure what's the difference of 2 and 3?). The only question is should we force the invoking chain for txns in a batch, which means the later transaction should not be submitted until it's previous txn has been committed on the ledger? Regarding option 1, the chaincode would be complicated in order to handle a series of operations on the same state . And with option 2, would a batch of txns be counted as multiple txns or only one transaction? The counting method should be the same for different blockchain systems.

Haojun (Wed, 23 May 2018 01:20:18 GMT):
@nkl199 If I understand correctly, since endorsor does not actually change the ledger but only simulate running the cc, you can not get a state which has just been putted within a single execution of chaincode, right? For example, in a single execution, the operation as putState(account)->getState(account)->change the account->putState(account) is illegal?

Haojun (Wed, 23 May 2018 01:20:18 GMT):
@nkl199 If I understand correctly, since endorsor does not actually change the ledger but only simulate running the cc, you can not get a state which has just been putted within a single execution of chaincode, right? For example, during a single execution, the operation as putState(account)->getState(account)->change the balance of the account->putState(account) is illegal?

Haojun (Wed, 23 May 2018 01:20:18 GMT):
@nkl199 If I understand correctly, since endorsor does not actually change the ledger but only simulate running the cc, you can not get a state which has just been putted within a single execution of chaincode, right? For example, during a single execution, the operation as 'putState(account)->getState(account)->change the balance of the account->putState(account)' is illegal?

rkrish82 (Wed, 23 May 2018 04:12:36 GMT):
Has joined the channel.

rkrish82 (Wed, 23 May 2018 04:28:42 GMT):
@Haojun In Fabric there is a significant amount of delay between endorsement and committing transaction, that is causing failures in transactions. Higher the tps, higher the error count.

rkrish82 (Wed, 23 May 2018 04:31:00 GMT):
Sawtooth handles batch as atomic operations means all transactions committed if all are valid and none of them committed if one o them is wrong.

klenik (Wed, 23 May 2018 08:55:19 GMT):
@Haojun Regarding you second question: read your own write is not advised (I read this somewhere in the docs about read-write semantics). Although I don't see the point of reading a value you already have. Batch topic: option 2) means that the batch invoke is implemented in the callback module (invokes in a loop), and not in the Caliper code-base. The callback module returns the tx info gathered by the invoke. It could return an array of results, and the Tx counter could be updated accordingly. The logic of multiple invokes depends on the callback module: wait for tx1 result before invoking tx2, or invoke them immediately. However, these choices could interfere with the rate controller. It would be nice to specify the exact workload profile for this benchmark, and see whether it can be integrated without changing the caliper code-base. (implementing a custom rate controller would be acceptable, as the load profile is strongly connected to the benchmark definition)

Haojun (Thu, 24 May 2018 02:23:30 GMT):
@klenik I prefer to implement the batch invoking in Caliper. If you do this on callback module, you need to write different implementation for different blockchain system. For example, it's unnecessary for Sawtooth to invoke multiple times for a batch. Now the invokesmartcontract() has a parameter which is a list of transaction parameters for multiple transactions.

Haojun (Thu, 24 May 2018 02:23:30 GMT):
@klenik I prefer to simulate the batch invoking in Caliper. If you do this on callback module, you need to write different implementation for different blockchain systems. For example, it's unnecessary for Sawtooth to invoke multiple times for a batch. The latest version of invokesmartcontract() has a parameter which is a list of transaction parameters for multiple transactions, and Fabric adaptor use this to submit multiple txns.

Haojun (Thu, 24 May 2018 02:37:01 GMT):
@rkrish82 Correct, the question is should we simulate the atomicity as much as possible without changing Fabric SDK.

rkrish82 (Thu, 24 May 2018 02:43:55 GMT):
@Haojun In Fabric SDK how do we get the block commit status of the transaction?

Haojun (Thu, 24 May 2018 02:48:30 GMT):
SDK can register the block event with a peer, so if a new block is committed, the peer will send a notification to SDK. The application can also register txns to watch with SDK.

rkrish82 (Thu, 24 May 2018 02:58:13 GMT):
Then Application can wait for all create account transactions are committed, later submit other operation transactions

Haojun (Thu, 24 May 2018 03:06:11 GMT):
@rkrish82 I agree with it, it needs to modify the benchmark. But I'm not sure if intel's guy is happy on that.

rkrish82 (Fri, 25 May 2018 08:05:56 GMT):
@hau

rkrish82 (Fri, 25 May 2018 08:06:26 GMT):
@Haojun Even if we fix create account issue some way, but still there seems to be a problem with Farbic in terms of accuracy/consistency of transaction execution. Suppose assume account a1 is created with initial balance 100$ and 2 operations on that account say deposit which adds 50$ to account balance and submitted at 0sec and it get commuted at 0.5sec. In between another withdraw operation of 50$ comes at 0.2 sec and reads current balance as 100$ because first operations is not committed yet. Deposit updates balance as 150 at 0.5sec and withdraw updates balance to 50$ at 0.6sec. Expected balance after execution of 2 operations should be 100$ but we would get 50$ in fabric. Correct me if I am wrong.

Haojun (Fri, 25 May 2018 08:14:04 GMT):
@rkrish82 In such case, the second txn will fail because of the readset/writeset version control of fabric, a MVCC_READ_CONFLICT will be returned.

sidkasat (Thu, 07 Jun 2018 03:41:54 GMT):
Has joined the channel.

klenik (Fri, 08 Jun 2018 09:51:26 GMT):
@Haojun @tkuhrt There was some chatter about a Linux Foundation backed cluster dedicated to the performance evaluation of Hyperledger projects. Do you know anything about it? Where could I get more information?

richzhao (Fri, 08 Jun 2018 14:39:58 GMT):
Has joined the channel.

Haojun (Tue, 12 Jun 2018 09:19:28 GMT):
@klenik sorry, i know nothing about it. Is it discussed at PSWG?

absingh0 (Thu, 14 Jun 2018 04:44:07 GMT):
Has joined the channel.

Katie_Wei (Wed, 20 Jun 2018 18:40:45 GMT):
Has joined the channel.

tkuhrt (Wed, 20 Jun 2018 20:07:07 GMT):
@dhuseby @rjones : Any chance you might know something about the cluster for performance evaluation?

dhuseby (Wed, 20 Jun 2018 20:07:07 GMT):
Has joined the channel.

rjones (Wed, 20 Jun 2018 20:07:07 GMT):
Has joined the channel.

tkuhrt (Wed, 20 Jun 2018 20:07:36 GMT):
The Caliper project update is due for the TSC tomorrow (https://wiki.hyperledger.org/groups/tsc/project-updates/caliper-2018-jun?do=edit). Will someone be able to represent that?

rjones (Thu, 21 Jun 2018 03:22:04 GMT):
@tkuhrt no update on cluster AFAIK

vagnerasilva (Sun, 01 Jul 2018 13:04:29 GMT):
Has joined the channel.

n1zyz (Mon, 02 Jul 2018 12:23:27 GMT):
Has joined the channel.

DivyaAgrawal (Mon, 02 Jul 2018 14:44:59 GMT):
Has joined the channel.

panyu2 (Tue, 03 Jul 2018 07:47:27 GMT):
Has joined the channel.

Haojun (Tue, 03 Jul 2018 07:53:59 GMT):
User User_1 added by Haojun.

Haojun (Tue, 03 Jul 2018 08:00:33 GMT):
Hi guys, I am sorry to tell that due to internal work transferring, I'll no longer in charge of this project in the future. My colleagures @panyu 2 and @jiangyaoguo will take over my job. I'll continue to watch this project and hope to still have chance to contribute to it.

Haojun (Tue, 03 Jul 2018 08:00:33 GMT):
Hi guys, I am sorry to tell that due to internal work transferring, I'll no longer be in charge of this project in the future. My colleagures @panyu 2 and @jiangyaoguo will take over my job. I'll continue to watch this project and hope to still have chance to contribute to it.

klenik (Tue, 03 Jul 2018 08:25:57 GMT):
@Haojun we are sorry to hear that :( and let's hope that your Caliper days are not over :)

klenik (Tue, 03 Jul 2018 08:26:51 GMT):
@jiangyaoguo @panyu 2 could you share your Github name in case we need to mention you in PRs?

Haojun (Tue, 03 Jul 2018 08:46:01 GMT):
panyu4 and jiangyaoguo

klenik (Fri, 06 Jul 2018 09:07:38 GMT):
Now that Fabric 1.2 is released (and a new release will come every quarter year I think), it's high time to resolve issue #37 (common connection profile) and leave most of the work to the SDK. PR #116 affects the deploy part a little, so it would overlap with this refactor. @jiangyaoguo @panyu 2 when could you review PR116, so I can start working on Issue#37? :)

panyu2 (Tue, 10 Jul 2018 05:48:36 GMT):
@klenik We will review PR116 as soon as possible.

hurf (Wed, 11 Jul 2018 07:40:14 GMT):
Has joined the channel.

niteshsolanki (Wed, 11 Jul 2018 08:49:24 GMT):
Has joined the channel.

niteshsolanki (Wed, 11 Jul 2018 09:02:12 GMT):
Hi @hurf

niteshsolanki (Thu, 12 Jul 2018 06:54:03 GMT):
Hi @Haojun please review PR#124

Haojun (Thu, 12 Jul 2018 09:25:06 GMT):
Hi @niteshsolanki , thanks for the contribution, it's an interesting idea :grinning: I've some quick questions and comments, please see github.

niteshsolanki (Thu, 12 Jul 2018 10:06:26 GMT):
@Haojun thanks for following up. updated the comments

klenik (Thu, 12 Jul 2018 15:21:17 GMT):
@Haojun I submitted two (really) small PRs for fixing some small inconveniences. Could you review it please? :)

Haojun (Fri, 13 Jul 2018 01:24:09 GMT):
@klenik done

rjones (Fri, 13 Jul 2018 21:55:02 GMT):
@all : 2FA will soon be required across all GitHub orgs. If you do not have 2FA enabled for your account, you will be automatically removed from the Hyperledger org and will need to be re-added. Please check to ensure you have 2FA enabled. https://help.github.com/articles/securing-your-account-with-two-factor-authentication-2fa/

kodonnel (Mon, 16 Jul 2018 15:15:25 GMT):
Has joined the channel.

kodonnel (Mon, 16 Jul 2018 15:16:42 GMT):
Hi, I've just made a minor PR, would be great if I could get a review. If there is a more appropriate place for me to ask for that please let me know. PR132

nkl199 (Tue, 17 Jul 2018 10:53:37 GMT):
looks good, but I'm unable to merge.

nkl199 (Tue, 17 Jul 2018 10:55:21 GMT):
side note: we *really* should enable/enforce the travis build as part of the PRs now as it is set to run all the required checks on inbound changes, from linting through to e2e perf runs on all frameworks ... this would stop regressions slipping in and more importantly would remove any need for manual checking from us ... this is very important in moving forward

arati_baliga (Wed, 18 Jul 2018 12:12:27 GMT):
Has joined the channel.

kodonnel (Wed, 18 Jul 2018 15:05:51 GMT):
[ ](https://chat.hyperledger.org/channel/caliper-contributors?msg=GozzFeXG3hjFNfxDk) @nkl199 nor I obviously.

nkl199 (Wed, 18 Jul 2018 15:11:23 GMT):
@Haojun - can you publish a list of contributors who have the ability to merge PRs? Then we chase people on that list rather than going direct to you :thumbsup:

rjones (Wed, 18 Jul 2018 15:14:45 GMT):
Send an email to helpdesk@hyperledger.org and ask who can merge

nkl199 (Wed, 18 Jul 2018 15:21:28 GMT):
thanks!

Hz (Thu, 19 Jul 2018 03:09:23 GMT):
Has joined the channel.

Haojun (Mon, 23 Jul 2018 02:55:15 GMT):
@nkl199 You can ask panyu4 or jiangyaoguo @github for merging.

Haojun (Mon, 23 Jul 2018 02:55:33 GMT):
[ ](https://chat.hyperledger.org/channel/caliper-contributors?msg=WrEcjJdHanzQLAxrQ) @nkl199 @jiangyaoguo will work on that recently

Haojun (Mon, 23 Jul 2018 07:56:10 GMT):
Does anybody know how to enable travis for an organisation repo like caliper? Who has the permission to do that?

nkl199 (Mon, 23 Jul 2018 08:23:29 GMT):
@Haojun @jiangyaoguo The travis scripts are already present, the only remaining task is to link the build to the github "required status" items for the repository

Haojun (Mon, 23 Jul 2018 08:34:54 GMT):
OK, but I don't have the permission to set this repo either......

rjones (Mon, 23 Jul 2018 23:54:04 GMT):
@Haojun @nkl199 which repo

rjones (Mon, 23 Jul 2018 23:55:18 GMT):
@Haojun @nkl199 when I look at Caliper repo on github, Travis or Jenkins has never tried to vote on a job

rjones (Mon, 23 Jul 2018 23:55:35 GMT):
@jwagantall ^^^

jwagantall (Mon, 23 Jul 2018 23:55:35 GMT):
Has joined the channel.

Haojun (Tue, 24 Jul 2018 10:01:14 GMT):
@rjones, now we check the compatibility of new PR with integrated blockchains as well as the linting compliance manually, and we hope to use travis to automate it.

nkl199 (Thu, 26 Jul 2018 10:05:12 GMT):
Hi all, just a heads up for the inbound fabric-sdk-node (1.3) release. There have been some big (breaking) changes through the event hub mechanism in base fabric, and this has fed up through to the sdk. Consequently, use of the EventHub is essentially deprecated, and 'we' should look to move to the Channel Event Hub implementation instead (also known as the Event delivery service) at points where the previous event hub was used. I would hope that any of the required changes will be made visible in the e2e tests for the fabric sdk, and we can make similar changes in the caliper code base

klenik (Thu, 26 Jul 2018 17:39:08 GMT):
@nkl199 I think the Common Connection Profile will handle this under the hood somehow, or it will be an easy transition. Do you know something about the backward compatibility for Fabric v1.0? Or do we need to use the deprecated eventhub for v1.0?

klenik (Thu, 26 Jul 2018 17:40:38 GMT):
@stinger112 Can you take a look at the Iroha-related modifications of PR #133? Thank you :)

klenik (Thu, 26 Jul 2018 17:40:38 GMT):
@stinger112 Can you take a look at the Iroha-related modifications of PR#133? Thank you :)

stinger112 (Thu, 26 Jul 2018 17:40:38 GMT):
Has joined the channel.

Haojun (Fri, 27 Jul 2018 01:55:50 GMT):
@rjones to support travis check as PR requirement, do we need to ask for help from the helpdesk? @jiangyaoguo

rjones (Fri, 27 Jul 2018 03:51:59 GMT):
@Haojun yes, you need to email helpdesk@hyperledger.org

zjubfd (Sat, 28 Jul 2018 10:13:09 GMT):
Has joined the channel.

abhishek.s (Wed, 08 Aug 2018 16:40:59 GMT):
Has joined the channel.

nkl199 (Thu, 09 Aug 2018 17:52:50 GMT):
so in essence, someone with the correct access needs to flip the switch on the caliper project here: https://travis-ci.org/profile/hyperledger

nkl199 (Thu, 09 Aug 2018 17:53:23 GMT):
since the travis builds are already enabled, it would be prudent to ensure that the builds are passing before the switch is flipped!

rjones (Thu, 09 Aug 2018 17:56:11 GMT):
https://travis-ci.org/hyperledger/caliper/builds/414178858

nkl199 (Thu, 09 Aug 2018 19:03:26 GMT):
yup, failing as the code coverage is too low in the unit tests .... yikes, yup ... many files have gone into the rate controllers folder that are untested.

nkl199 (Thu, 09 Aug 2018 19:08:39 GMT):
So the choices here revolve around editing package.json nyc section to either: - exclude the rate control section - reduce the coverage requirements to the bare minimum of: Statements : 5.33% ( 13/244 ) Branches : 8.7% ( 8/92 ) Functions : 7.69% ( 3/39 ) Lines : 5.49% ( 13/237 ) The latter is the best option IMO as it will stop more code going in untested

klenik (Thu, 09 Aug 2018 19:12:05 GMT):
@nkl199 next week I'll write explicit tests (not just benchmark configs) for the rate controllers I added. Until then I also support the 2nd option

nkl199 (Thu, 09 Aug 2018 19:12:24 GMT):
:thumbsup:

nkl199 (Thu, 09 Aug 2018 19:12:55 GMT):
there's a hideous amount of test effort in there :cry:

nkl199 (Thu, 09 Aug 2018 19:13:49 GMT):
- i'll submit a PR to drop the coverage

nkl199 (Thu, 09 Aug 2018 19:19:30 GMT):
(travis build currently running in my personal travis builds)

nkl199 (Thu, 09 Aug 2018 19:24:35 GMT):
green and raised PR :thumbsup:

nkl199 (Thu, 09 Aug 2018 19:25:13 GMT):

Screen Shot 2018-08-09 at 15.24.52.png

jiangyaoguo (Thu, 16 Aug 2018 03:05:38 GMT):
@nk1

jiangyaoguo (Thu, 16 Aug 2018 03:06:50 GMT):
@nk199 what about we close coverage check for now? https://github.com/hyperledger/caliper/pull/158

nkl199 (Mon, 20 Aug 2018 12:43:32 GMT):
hi, that is entirely the point - all new PRs should come complete with tests

nkl199 (Tue, 21 Aug 2018 15:30:07 GMT):
as an FYI for inbound fabric-sdk-node changes (which caliper relies upon) the EventHub is removed, which is used in the channel join process. I am currenlty making changes on a local branch to make caliper compatible with the unstable fabric sdk (1.3 snapshots)

karthikmohan91 (Wed, 22 Aug 2018 12:15:02 GMT):
Has joined the channel.

nkl199 (Thu, 30 Aug 2018 10:46:42 GMT):
As an update to the above, i have got Caliper working with non-tls fabric 1.3. Due to the difference in the APIs used to work with different levels, i've split the fabric portion into subfolders, with the Fabric interface at the root level folder: SRC/Fabric -/1.1 -/1.3 -fabric.js And I'm using a 'level' flag when invoking 'main.js' that sets a process variable to decide which code base to use by dynamically changing the imports within fabric.js: ``` if (process.env.LEVEL){ level = '/' + process.env.LEVEL + '/'; } else { level = '/1.1/'; } const util = require('./' + level + 'util.js'); ``` I'm going to investigate/expand this a little more, as it would be nice to be able to work with TLS enabled networks too. Interested on opinions for the code base split method used above

nkl199 (Thu, 30 Aug 2018 10:51:06 GMT):
Also, I am investigating the use of the new `network` api commands in fabric-sdk-node for transaction submission

klenik (Thu, 30 Aug 2018 11:25:50 GMT):
@nkl199 can't you access the version of the installed Fabric SDK through its `package.json` file? That way the user doesn't need to explicitly set the `LEVEL` env variable. It would be good to compose a list of changes between the SDK/Fabric versions, both for added features and API changes

nkl199 (Thu, 30 Aug 2018 12:02:12 GMT):
good point, it could also form part of the config.json file

nkl199 (Thu, 30 Aug 2018 12:30:16 GMT):
(whilst i'm at it, i'm async/awaiting all the fabric driver code)

klenik (Thu, 30 Aug 2018 12:31:16 GMT):
That'a a good idea! (The other parts should be transformed too, the current error messages are horrible :P )

nkl199 (Thu, 30 Aug 2018 12:49:13 GMT):
:joy: agree! In all of the configuration I am also ripping out the tape.test aspects, as that is not a perf test aspect, but a remnant of the code origins from the node-sdk-e2e tests

nkl199 (Thu, 30 Aug 2018 16:13:47 GMT):
... for the version setting, i'm picking up on the `config.info.version` field, and if not present defaulting to 1.1

nkl199 (Fri, 31 Aug 2018 08:18:58 GMT):
User User_2 added by nkl199.

sureshtedla (Fri, 31 Aug 2018 17:34:58 GMT):
Has joined the channel.

nkl199 (Tue, 04 Sep 2018 13:09:55 GMT):
latest update - caliper is now working on fabric 1.3, with the 1.3 client apis, and i have even added the ability to use the new "invoke" commands to issue a transaction as a one liner ... which is nice!

nkl199 (Tue, 04 Sep 2018 13:10:38 GMT):
... i'm planning to look into the tls aspects now, then submit a PR

klenik (Tue, 04 Sep 2018 20:21:52 GMT):
btw, how is the compatibility with 1.2? :D many questions/errors originate from it (although mainly because they use 1.2 SDK with 1.1 Fabric)

nkl199 (Wed, 05 Sep 2018 07:30:09 GMT):
not tried yet to be honest, i can do a sniff test on it tho

klenik (Wed, 05 Sep 2018 08:49:55 GMT):
Before the official release we did some tests with 1.2 using caliper, and it worked fine. Although the new features (e.g., private collections) are missing on the caliper side.

nkl199 (Thu, 06 Sep 2018 14:31:26 GMT):
now have tls enabled for fabric ... so getting closer to let people have a look/play ...

sgaddam (Tue, 11 Sep 2018 10:32:45 GMT):
Has joined the channel.

longac (Tue, 18 Sep 2018 02:48:19 GMT):
Has joined the channel.

compleatang (Fri, 21 Sep 2018 17:45:53 GMT):
Has left the channel.

nkl199 (Wed, 26 Sep 2018 08:42:02 GMT):
There are a fair few PRs pending and it would appear that momentum with Caliper is slowing down. Both @klenik and I would be keen to keep up the development of Caliper, but would need increased authority to do so - at the very least the ability to merge PRs. We should also be looking into having some regular telephone interlocks to chat about what we are working on, thinking about, and general considerations on what could/should be included within Caliper to help usage. Is there anyone here we can talk to with regards to that?

nkl199 (Wed, 26 Sep 2018 08:42:02 GMT):
@rjones There are a fair few PRs pending and it would appear that momentum with Caliper is slowing down. Both @klenik and I would be keen to keep up the development of Caliper, but would need increased authority to do so - at the very least the ability to merge PRs. We should also be looking into having some regular telephone interlocks to chat about what we are working on, thinking about, and general considerations on what could/should be included within Caliper to help usage. Is there anyone here we can talk to with regards to that?

tkuhrt (Wed, 26 Sep 2018 15:45:57 GMT):
@Haojun @FlyingTiger : It appears that you are the current committers on Caliper. Thoughts on adding @klenik and @nkl199 as additional committers?

FlyingTiger (Wed, 26 Sep 2018 15:45:57 GMT):
Has joined the channel.

tkuhrt (Wed, 26 Sep 2018 17:20:17 GMT):
Also, a reminder that the project update for Caliper will be on the schedule for tomorrow's TSC: https://wiki.hyperledger.org/groups/tsc/project-updates/caliper-2018-sep?do=edit

nkl199 (Fri, 28 Sep 2018 07:59:19 GMT):
Hi, I was on the call, but appear to have been alone in that regard :(

rkrish82 (Thu, 04 Oct 2018 04:30:02 GMT):
Hi All, Please someone review the PR - https://github.com/hyperledger/caliper/pull/159 and https://github.com/hyperledger/caliper/pull/167

nkl199 (Thu, 04 Oct 2018 07:49:50 GMT):
the above requires @Haojun or @FlyingTiger

panyu2 (Wed, 10 Oct 2018 00:21:02 GMT):
We are now working on these PRs. @rkrish82

nkl199 (Wed, 10 Oct 2018 08:36:47 GMT):
Thanks @panyu 2 - noticed that there has been a flurry of activity in the repository - good to see :smile:

panyu2 (Wed, 10 Oct 2018 09:48:51 GMT):
:relaxed:, give us more time, we will make Caliper become healthier.

klenik (Wed, 10 Oct 2018 09:50:05 GMT):
Hi everyone! Since there was some chatter about starting regular Caliper community calls, @nkl199 and I have put together a google docs about the various todos/plans regarding Caliper. The following link is in commenting mode for obvious reasons, but if you like the idea, it can be migrated into Hyperledger hands, like the PSWG docs https://docs.google.com/document/d/1QQFmuRW4QXk0HCUIw1NoqweNQBnQ4P3eOhAE10m1vng/edit?usp=sharing

klenik (Wed, 10 Oct 2018 09:52:16 GMT):
@hurf @panyu 2 @tkuhrt I don't know the exact procedure for starting such scheduled calls, but I'd happy to help out with anything you need

panyu2 (Thu, 11 Oct 2018 00:54:16 GMT):
Thanks for your suggestions. Create a template for the issue and PR descriptions, and provide good practices on how to ask questions properly on the proper channels

panyu2 (Thu, 11 Oct 2018 00:57:49 GMT):
Thanks for your suggestions. For this one --Create a template for the issue and PR descriptions, and provide good practices on how to ask questions properly on the proper channels, I think we can generate a document about FAQ to provide good practices on how to avoid the misconfigured issues.

hurf (Thu, 11 Oct 2018 10:11:15 GMT):
@klenik great work

hurf (Thu, 11 Oct 2018 10:12:28 GMT):
There's several issues from my side can be added to the plan

hurf (Thu, 11 Oct 2018 10:13:00 GMT):
 Align with PSWG definitions. Caliper was done before PSWG publishes its whitepaper, it time to check the consistency.  Somebody has added new monitoring module, it’s included in TrustedBench work. Need to find out are there more unmerged features from TrustedBench work, and upstream to Caliper  Other DLT support. But which?  Consider extend Caliper’s scope? Add system/e2e test targeting some most concerned blockchain features, like consensus effectiveness.

hurf (Thu, 11 Oct 2018 10:13:23 GMT):
Some could be short term, some long term

hurf (Thu, 11 Oct 2018 10:13:54 GMT):
I think we can start community calls next week

klenik (Thu, 11 Oct 2018 10:14:11 GMT):
That would be great!

hurf (Thu, 11 Oct 2018 10:14:39 GMT):
I'm testing my zoom settings, hope it could work fine for everybody

klenik (Thu, 11 Oct 2018 10:15:06 GMT):
I send you the editing link in private, so you guys can also manage the doc

hurf (Thu, 11 Oct 2018 10:15:54 GMT):
yes thank, either by chat or huruifeng@huawei.com

klenik (Thu, 11 Oct 2018 10:16:38 GMT):
granted editing access through email

hurf (Thu, 11 Oct 2018 10:25:05 GMT):
got it , thx

tkuhrt (Thu, 11 Oct 2018 17:02:41 GMT):
[ ](https://chat.hyperledger.org/channel/caliper-contributors?msg=XZuAxcKntoXJDsLxn) @klenik @hurf @panyu 2 : If you are all interested in using the Hyperledger Community Zoom channel for hosting meetings, please find a time on the [Hyperledger Community Calendar](https://calendar.google.com/calendar/embed?src=linuxfoundation.org_nf9u64g9k9rvd9f8vp4vur23b0%40group.calendar.google.com&ctz=foo) that preferably does not conflict with another community meeting and work with @rjones to set it up.

tkuhrt (Thu, 11 Oct 2018 17:04:26 GMT):
[ ](https://chat.hyperledger.org/channel/caliper-contributors?msg=djzJssSaekeG45Sjt) @klenik : Can you make sure that this link exists on the [Hyperledger Caliper wiki page](https://wiki.hyperledger.org/projects/caliper) if you haven't already placed it there?

klenik (Thu, 11 Oct 2018 18:23:09 GMT):
@tkuhrt added the link to the wiki page. Can you add the link to the channel topic for ease of access?

klenik (Thu, 11 Oct 2018 18:27:45 GMT):
@hurf @panyu 2 I'm flexible regarding the call times, so choose a time that works for you. I think most of the contributors here are scattered in the UTC+[1-8] time zone range, but we'll see who else is interested in the calls.

klenik (Thu, 11 Oct 2018 18:49:49 GMT):
Also, the GitHub invitation for the Caliper Committer team arrived and accepted, thank you all for managing this process :)

tkuhrt (Thu, 11 Oct 2018 19:10:36 GMT):

tkuhrt (Thu, 11 Oct 2018 19:11:32 GMT):
Used for contributors of Hyperledger Caliper to discuss implementation details. See [planning document](https://docs.google.com/document/d/1QQFmuRW4QXk0HCUIw1NoqweNQBnQ4P3eOhAE10m1vng/edit?usp=sharing)

tkuhrt (Thu, 11 Oct 2018 19:11:57 GMT):
Discuss implementation details of Hyperledger Caliper. See [planning document](https://docs.google.com/document/d/1QQFmuRW4QXk0HCUIw1NoqweNQBnQ4P3eOhAE10m1vng/edit?usp=sharing)

tkuhrt (Thu, 11 Oct 2018 19:13:02 GMT):
[ ](https://chat.hyperledger.org/channel/caliper-contributors?msg=HzTvjTafBd2pQxzSN) Congratulations on becoming a maintainer, @klenik

panyu2 (Mon, 15 Oct 2018 03:03:26 GMT):
@klenik @nkl199 @hurf @FlyingTiger and someone concerns about attending the meeting: Does the time 10.17 14:00 is ok for you ?

panyu2 (Mon, 15 Oct 2018 03:07:45 GMT):
10.17 14:00-15:00 is the UTC+8 Time

klenik (Mon, 15 Oct 2018 03:35:24 GMT):
@panyu 2, 14:00 UTC+8 sounds good to me

hurf (Mon, 15 Oct 2018 06:42:08 GMT):
We can also discuss at our first meeting to see whether the timing is good for people from other places(if we have any).

hurf (Mon, 15 Oct 2018 06:43:11 GMT):
Our first meeting will be at Oct 17, @tkuhrt can you help to schedule the meeting using community zoom?

tkuhrt (Mon, 15 Oct 2018 07:07:27 GMT):
@hurf : I sent you an invite. Can you confirm that I got the time zone correct? Also, what sort of schedule would you like to have these meetings? Weekly, bi-weekly?

hurf (Mon, 15 Oct 2018 07:10:03 GMT):
Correct, thanks Tracy

hurf (Mon, 15 Oct 2018 07:10:37 GMT):
I think for now weekly could be better.

hurf (Mon, 15 Oct 2018 07:10:49 GMT):
We need a little pressure:-)

klenik (Mon, 15 Oct 2018 07:17:43 GMT):
@hurf What will be the agenda?

tkuhrt (Mon, 15 Oct 2018 08:04:08 GMT):
Great. I will make it weekly

hurf (Mon, 15 Oct 2018 08:17:08 GMT):
1.round call 2.discuss meeting time 3.review current states 4.agree on merge rules and health metrics 5.discuss development plan 6.technical review of specific PR(optional, if any)

hurf (Mon, 15 Oct 2018 08:18:11 GMT):
Also I suggest to make health metrics review a fixed agenda item for following meetings.

rkrish82 (Mon, 15 Oct 2018 10:16:37 GMT):
@FlyingTiger Please review the PR - https://github.com/hyperledger/caliper/pull/190

panyu2 (Wed, 17 Oct 2018 07:43:53 GMT):
Meeting Minutes(2018-10-17) 1. Review current state; 2. Discuss regular meeting time: UTC 7am(8am winter) and UTC 1pm(2pm winter), shifting every week. 3. Discuss requirements for PRs to be accepted: 1) Number of reviewer ACKs (with 2 maintainers confirmation); 2) Meet CI requirements but reviewers should also inspect the correctness/meaningfulness of tests. 4. Please Nick complete the template for issue and PR descriptions and provide good practices on how to ask questions properly on the proper channels.(Priority: High) 5. Please Nick lead the work that create a Caliper homepage instead of separate Readme.md files. feihu and panyu help on implementation. (Priority: Medium) 6. Please Victor make a list of known experts for HLP project. (Priority: Medium) 7. Please Attila refactor Promise chains to async/await wherever possible.(Priority: High) 8. Please Panyu change current logging to a unified logging framework. (Priority: High) 9. Please Feihu Jiang change current configuration framework to a unified configuration framework. (Priority: High)

nkl199 (Wed, 17 Oct 2018 09:36:52 GMT):
templates up in a PR now :thumbsup:

klenik (Wed, 17 Oct 2018 09:37:00 GMT):
7. => Created the Issues [192](https://github.com/hyperledger/caliper/issues/192) and [193](https://github.com/hyperledger/caliper/issues/193) for the async/await and `tape` refactoring, and tagged them with the new `good first issue` label for extra motivation. If no one will resolve them by the time I'm finished with the Fabric common connection profile upgrade, then I'll work on them myself. 8. =>[This utility function](https://github.com/hyperledger/fabric-sdk-node/blob/95b02d9b2ce508d8a3684792b6b040ae04a4067c/fabric-client/lib/utils.js#L119:16) from the Node SDK could be reused for logging (without introducing a mandatory dependency to the SDK). 9. => [This class](https://github.com/hyperledger/fabric-sdk-node/blob/release-1.3/fabric-client/lib/Config.js) from the Node SDK could be reused for the configuration framework, it's pretty flexible and Fabric-like, keeping the similar "look-and-feel" between the different projects (again, without introducing a mandatory dependency to the SDK)

panyu2 (Thu, 18 Oct 2018 02:25:02 GMT):
I just publish our meeting minutes on Caliper's wiki. Every week after the meeting , we can update it.

nkl199 (Thu, 18 Oct 2018 07:43:17 GMT):
In order to get the caliper homepage going, we will need a new branch alongside `master` to host the files. Traditionally the brach is named `gh-pages` though it can be anything now, such as `docs`. The project settings can then toggled to publish the site from that branch, and it will be accessible via the url `https://hyperledger.github.io/caliper`. Creation of the branch and editing the project settings requires admin rights, and I think Ry is required for this?

panyu2 (Thu, 18 Oct 2018 08:10:21 GMT):
You mean the maintainer's rights is not enough for the branch creation and editing the project settings? If so, we can ask Ry.

nkl199 (Thu, 18 Oct 2018 10:36:52 GMT):
Ry was required to enable the Travis CI process, whilst a maintainer can (should) be able to create a new branch, I am uncertain about the ability to modify project settings

rjones (Thu, 18 Oct 2018 16:29:48 GMT):
@nkl199 send a request to helpdesk@hyperledger.org. I think maintainers can create branches

rjones (Thu, 18 Oct 2018 16:32:43 GMT):
gh pages is enabled for master branch right now. once you create the new branch, send a helpdesk ticket to change the branch they’re publsihed from @nkl199

hotbydefault (Fri, 19 Oct 2018 03:59:11 GMT):
Has joined the channel.

nkl199 (Fri, 19 Oct 2018 08:33:29 GMT):
thanks!

klenik (Fri, 19 Oct 2018 11:00:18 GMT):
@nkl199 @panyu 2 I created the `gh-pages` branch, so the rest is up to the helpdesk and your email :)

nkl199 (Fri, 19 Oct 2018 11:00:32 GMT):
:thumbsup:

panyu2 (Tue, 23 Oct 2018 01:21:45 GMT):
Hi, I'm sorry I've made a mistake in the minutes. The regular meeting time: UTC 7am(8am winter) and UTC 1pm(2pm winter), will shift every two weeks not every week. So tomorrow(10.24 Wednesday), we will hold the meeting at UTC 7am.

klenik (Tue, 23 Oct 2018 13:25:10 GMT):
So the same time as last week then? @hurf What's the agenda for tomorrow?

panyu2 (Wed, 24 Oct 2018 01:00:36 GMT):
Yes, same time as last week.

panyu2 (Wed, 24 Oct 2018 01:01:28 GMT):
Agenda:

panyu2 (Wed, 24 Oct 2018 01:03:37 GMT):
1. progress of last week's work;

panyu2 (Wed, 24 Oct 2018 01:09:47 GMT):
1. the progress of last week's work; 2. continue to discuss the doc attila shared and discuss develop plan; 3. agree on health metrics. I just suggest. Do you have any proposal?

FlyingTiger (Wed, 24 Oct 2018 07:19:53 GMT):
As

FlyingTiger (Wed, 24 Oct 2018 07:26:59 GMT):
As the develop document, like Fabric: code > env > console args > config file > defaults, and the Fabric Node SDK put this together through nconf. But I didn't find where nconf is used in Fabric Node SDK, Could you give me some points please? @klenik @nkl199

FlyingTiger (Wed, 24 Oct 2018 07:27:03 GMT):
Thanks very much!

klenik (Wed, 24 Oct 2018 07:35:41 GMT):
@FlyingTiger This is the `Config` class that wraps `nconf` and sets the priorities among the setting sources: https://github.com/hyperledger/fabric-sdk-node/blob/release-1.3/fabric-client/lib/Config.js The default configuration file for the SDK is here: https://github.com/hyperledger/fabric-sdk-node/blob/release-1.3/fabric-client/config/default.json Caliper could provide a similar one with the available/configurable settings you identified, then users can also set for example the `request-timeout` setting as an environment variable, like `REQUEST_TIMEOUT=10000`

klenik (Wed, 24 Oct 2018 07:36:29 GMT):
@nkl199 Any pointers on where the SDK uses these settings and how can you handle prefixes like `CALIPER_REQUEST_TIMEOUT=10000` ?

FlyingTiger (Wed, 24 Oct 2018 07:41:25 GMT):
Thanks, I'll look into this. @klenik

klenik (Wed, 24 Oct 2018 07:59:02 GMT):
NB: passing the user provided settings (environment variables, command line arguments, config file) to remote clients will be interesting :)

nkl199 (Wed, 24 Oct 2018 08:15:27 GMT):
@klenik - client side or server side timeout?

klenik (Wed, 24 Oct 2018 08:16:09 GMT):
Doesn't matter, just an example of how we can retrieve the value of a setting

nkl199 (Wed, 24 Oct 2018 08:18:53 GMT):
ah, like `Client.setConfigSetting('request-timeout', 60000);` within the Client ... which goes dow to `BaseClient`

nkl199 (Wed, 24 Oct 2018 08:18:53 GMT):
ah, like `Client.setConfigSetting('request-timeout', 60000);` within the Client ... which goes dow to `BaseClient`?

nkl199 (Wed, 24 Oct 2018 08:19:06 GMT):
(there is a simlar getConfigSetting)

klenik (Wed, 24 Oct 2018 08:21:53 GMT):
Yep, plus I think there is a global Config instance in the utils, and get/setConfig. @FlyingTiger So it has quite a simple API if you reuse the mentioned `Config` class

klenik (Wed, 24 Oct 2018 08:52:44 GMT):
Put together a spreadsheets to (manually) monitor the metrics. Link with commenting rights: https://docs.google.com/spreadsheets/d/1jdE8ZJNfLG4NIzEoMFNq6lywLxXWCzNbSEDqP-UJD6Q/edit?usp=sharing @hurf I sent you the link with editing rights via email

panyu2 (Wed, 24 Oct 2018 08:57:20 GMT):
@klenik , I just create the same spreadsheets. Well, I'll put your link on the wiki.

klenik (Wed, 24 Oct 2018 09:01:02 GMT):
Should have assigned someone on the task to avoid this :D Oh well, still better than 0 sheets :)

panyu2 (Wed, 24 Oct 2018 09:04:30 GMT):
yes.:slight_smile:

panyu2 (Wed, 24 Oct 2018 09:17:19 GMT):
I find the meeting time Wednesday UTC 1pm has already assigned other meetings. So maybe we should change the Wednesday UTC 1pm to Tuesday UTC 2pm.

panyu2 (Wed, 24 Oct 2018 09:22:40 GMT):
@tkuhrt , in order to cover more contributors' time, we would like to shift the time between UTC 6am on Wednesday and UTC 2pm on Tuesday every two weeks. would you please set it for us? Thank you

klenik (Wed, 24 Oct 2018 09:27:47 GMT):
@panyu 2 Just to confirm, UTC 2PM is right after the PSWG call, right?

panyu2 (Wed, 24 Oct 2018 09:28:01 GMT):
yes

rjones (Wed, 24 Oct 2018 10:25:45 GMT):
@panyu 2 please send me an email with exactly what you want changed: rjones@linuxfoundation.org

panyu2 (Wed, 24 Oct 2018 11:50:06 GMT):
@rjones I just sent you the email with the exact changed time.

rjones (Wed, 24 Oct 2018 15:47:56 GMT):
@panyu2 I think I moved the meeting to the correct time - please let me know

panyu2 (Thu, 25 Oct 2018 01:18:50 GMT):
@rjones Thank you so much. The time on 2018.10.30 is correct. I'm sorry I've forgotten to adjust the time in winter. So I've replied the mail and drawn a from to explain the time clearly.

houqinghui (Fri, 26 Oct 2018 00:32:07 GMT):
Has joined the channel.

panyu2 (Tue, 30 Oct 2018 00:30:41 GMT):
Hi，to whom concerns the regular meeting, the meeting will take place at UTC 2pm on 10.30, just the time after PSWG.

klenik (Mon, 05 Nov 2018 17:12:45 GMT):
@nkl199 @panyu2 I merged the docs PR, but it looks like that the website is still deployed from the master branch. Did the helpdesk confirmed that they changed the publish-branch to gh-pages?

klenik (Mon, 05 Nov 2018 17:13:59 GMT):
@FlyingTiger @panyu2 left some minor comments on the logging and config framework integration PRs

nkl199 (Mon, 05 Nov 2018 17:41:57 GMT):
@klenik - an email now needs to be sent to have the change made - I'll do that one :thumbsup:

klenik (Mon, 05 Nov 2018 17:44:10 GMT):
Ah, I thought it was already sent when the branch was created. Then everything is (will be) okay 🙂

panyu2 (Tue, 06 Nov 2018 11:18:42 GMT):
@klenik Thanks for you comments. I'm working on them.

panyu2 (Tue, 06 Nov 2018 11:23:29 GMT):
@nkl199 , if there is anything needs my help, please let me know

panyu2 (Tue, 06 Nov 2018 11:23:29 GMT):
@klenik , I've merged it.

klenik (Tue, 06 Nov 2018 11:25:57 GMT):
@panyu2 the config pr is merged, so you can rebase and use it for the logging level/target config. Let me know when the pr is updated and I'll take a look at it

panyu2 (Tue, 06 Nov 2018 11:27:34 GMT):
Hi，to whom concerns the regular meeting, the meeting will take place at UTC 3pm on 11.6, just the time after PSWG. The link of the meeting can be found on the Caliper's wiki https://wiki.hyperledger.org/projects/caliper

panyu2 (Tue, 06 Nov 2018 11:31:38 GMT):
@klenik got it. It may facilitate the logging config. I'll do some modifications according to the config pr.

nkl199 (Tue, 06 Nov 2018 11:33:17 GMT):
the helpdesk ticket for enabling the gh-pages has been raisde

nkl199 (Tue, 06 Nov 2018 11:33:17 GMT):
the helpdesk ticket for enabling the gh-pages has been raised

nkl199 (Tue, 06 Nov 2018 11:33:19 GMT):
ed

nkl199 (Tue, 06 Nov 2018 14:02:08 GMT):
apologies - I am unable to make the maintainers call today, though my update is: - pending activation of website following email to helpdesk - need to rebase 1.3 changes to enable merge

klenik (Tue, 06 Nov 2018 14:09:45 GMT):
@nkl199 any update on the how-to about adding new pages to the website?

nkl199 (Tue, 06 Nov 2018 14:10:22 GMT):
-> will make a CR on that ASAP to the gh-pages branch

nkl199 (Tue, 06 Nov 2018 17:45:17 GMT):
https://hyperledger.github.io/caliper/

nkl199 (Tue, 06 Nov 2018 17:45:20 GMT):
:heart_eyes:

nkl199 (Tue, 06 Nov 2018 17:48:08 GMT):
now raising a request to have the website featured in the "description" of the caliper project, so people can link to it directly

klenik (Tue, 06 Nov 2018 17:49:58 GMT):
Awesome 😁 and I'll put it on the wiki page if I don't forget it

tkuhrt (Tue, 06 Nov 2018 17:50:08 GMT):
@nkl199 : I can edit the description here: https://github.com/hyperledger/caliper

tkuhrt (Tue, 06 Nov 2018 17:50:20 GMT):
Do you just want it to reference https://hyperledger.github.io/caliper/

nkl199 (Tue, 06 Nov 2018 17:50:24 GMT):
wohoo! Please!

nkl199 (Tue, 06 Nov 2018 17:50:38 GMT):
there should be an option to link the website

tkuhrt (Tue, 06 Nov 2018 17:50:43 GMT):
Check it out!

tkuhrt (Tue, 06 Nov 2018 17:50:52 GMT):
That was simple

nkl199 (Tue, 06 Nov 2018 17:51:21 GMT):
yay! perfect!!

nkl199 (Tue, 06 Nov 2018 17:51:27 GMT):
Thanks :D

klenik (Wed, 07 Nov 2018 18:00:13 GMT):
@nkl199 @hurf I merged the additional documentation guidelines/how-to from Nick, added the site URL to the wiki page, and also sent news about it to the mailing list and rocketchat channel. If you know any more forums, spread the news :) Btw, do we have a visitor counter for the site? I couldn't find it under the project insights (but there are some interesting metrics). Maybe once the visiting starts, it will appear.

klenik (Wed, 07 Nov 2018 18:00:13 GMT):
@nkl199 @hurf I merged the additional documentation guidelines/how-to from Nick, added the site URL to the wiki page, and also sent news about it to the mailing list and rocketchat channel. If you know any more forums, spread the news :) Btw, do we have a visitor counter for the site? I could find it under the project insights (but there are some interesting metrics). Maybe once the visiting starts, it will appear.

tkuhrt (Wed, 07 Nov 2018 18:09:34 GMT):
We might be able to provide a Google Analytics ID. @rjones do you know what process might be?

rjones (Wed, 07 Nov 2018 18:45:59 GMT):
@klenik @tkuhrt please send me an email rjones@linuxfoundation.org and ask me for what you need

rjones (Wed, 07 Nov 2018 18:46:13 GMT):
I will get you an ID

tkuhrt (Wed, 07 Nov 2018 19:24:56 GMT):
@klenik : I have the Google Analytics ID that I can share with the right person. I will DM you this

tkuhrt (Wed, 07 Nov 2018 19:25:06 GMT):
unless I should share it with someone else

klenik (Thu, 08 Nov 2018 11:47:24 GMT):
@tkuhrt @rjones I integrated the GA tags with the site (in my fork), but according to [this page](https://support.google.com/analytics/answer/7318509) we have to disclose that the site tracks some things. What is the policy for that (text of the disclosure, location)?

klenik (Thu, 08 Nov 2018 11:47:24 GMT):
@tkuhrt @rjones I integrated the GA tags with site (in my fork), but according to [this page](https://support.google.com/analytics/answer/7318509) we have to disclose that the site tracks some things. What is the policy for that (text of the disclosure, location)?

nkl199 (Thu, 08 Nov 2018 12:50:23 GMT):
https://termsfeed.com/blog/privacy-policy-google-analytics/

nkl199 (Thu, 08 Nov 2018 12:50:28 GMT):
that's a helpful link

nkl199 (Thu, 08 Nov 2018 12:51:07 GMT):
looks like we could have a seperate "Privacy Policy" page in the left menu, along with the "open source" items

nkl199 (Thu, 08 Nov 2018 12:51:07 GMT):
looks like we could have a separate "Privacy Policy" page in the left menu, along with the "open source" items

klenik (Thu, 08 Nov 2018 14:25:04 GMT):
Yep, I'll do that (as practice), but I'd still like some help with the official content of the page (the cookie pop-ups, however, are beyond my skills) :)

tkuhrt (Thu, 08 Nov 2018 16:38:59 GMT):
From [this site](https://policies.google.com/technologies/partner-sites), I see: > Many websites and apps use Google Analytics to understand how visitors engage with their sites or apps. If you don’t want Analytics to be used in your browser, you can [install the Google Analytics browser add-on](https://tools.google.com/dlpage/gaoptout). Learn more about [Google Analytics and privacy](https://support.google.com/analytics/answer/6004245). The [hyperledger.org privacy policy](https://www.linuxfoundation.org/privacy/) has: > Purposes and Legitimate Interests > > TLF uses the personal information we collect for our legitimate business interests, which include the following purposes: > > _Analytics_. To gather metrics to better understand how users access and use our Sites and Services and participate in our Projects; to evaluate and improve the Sites, including personalization, to develop new services; and to understand metrics regarding the community health of our Projects. So maybe a combination of the two would be useful. But I am not a lawyer.

klenik (Thu, 08 Nov 2018 18:14:09 GMT):
Okay, then I'll concatenate these things somehow :thumbsup:

klenik (Fri, 09 Nov 2018 07:08:25 GMT):
Sample: https://aklenik.github.io/caliper/docs/Privacy_Policy.html Not much, but better than nothing. If you also think so, I'll raise the PR

klenik (Fri, 09 Nov 2018 07:08:25 GMT):
@tkuhrt Sample: https://aklenik.github.io/caliper/docs/Privacy_Policy.html Not much, but better than nothing. If you also think so, I'll raise the PR

tkuhrt (Fri, 09 Nov 2018 15:56:04 GMT):
@klenik : Let me run this past the folks at The Linux Foundation and see what they think. It definitely covers the use of Google Analytics.

tkuhrt (Fri, 09 Nov 2018 16:06:54 GMT):
@klenik : Here is what I got back. I would suggest they just link to the LF's privacy policy at https://www.linuxfoundation.org/privacy/ (probably easiest via a "Privacy Policy" link in the footer of https://hyperledger.github.io/caliper/). The LF's main privacy policy already includes appropriate provisions regarding analytics (including specific reference to Google Analytics) so I don't think the projects will need anything custom as long as they point there.

klenik (Fri, 09 Nov 2018 22:04:52 GMT):
@tkuhrt Did exactly that. Well, almost, I put it in the header so I surely don't mess up the layout anywhere else. @panyu2 Could you please merge the PR?

panyu2 (Sun, 11 Nov 2018 02:42:56 GMT):
@klenik I've merged it.

panyu2 (Tue, 13 Nov 2018 08:53:10 GMT):
Hi，to whom concerns the regular meeting, the meeting will take place at UTC 6am on 11.14. The link of the meeting can be found on the Caliper's wiki https://wiki.hyperledger.org/projects/caliper

klenik (Wed, 14 Nov 2018 23:19:24 GMT):
@panyu2 @nkl199 So based on list of issues I got from @panyu2, I compiled the following list suggested actions/reasons: 64: Composer-related. Asked for details ==> wait 1-2 weeks 78: Core-related. Looks stale. There were some fixes to client exits since then. ==> close as inactive/can't reproduce error 92: Offering to contribute ==> close since not an issue :) 93: Fabric-related. Orderer service not available yet, possible because longer network start-up time ==> no reaction for 8 days, wait a few more days 96: Composer-related. Possible timeout while deploying chaincode. OP suggested closing the issue. ==> close the issue as can't reproduce, suggest increasing timeout in peer settings 101: Composer-related. Unidentified user/activation required. Been stale for a while. ==> Ask for details (error logs, etc), wait 105: Fabric-related. Same as 93, linked suggested fix ==> wait a few days 107: Fabric-related. Slow TPS for multiple orderer. Possible because Caliper doesn't support multiple orderer yet. ==> suggest to wait for multiple orderer support, and close the issue 114: The solution is ready, either as a contribution, or taking the proposed fix from the issue ==> ping RobertDiebels 119: Too general, should be split into multiple refactoring issues. Parts of it are already fixed ==> close this one 130-131: High timeout value in callback module ==> the callback module is just an example, not core-Caliper, so close these, with appropriate comment 140: Nick has a solution ready for this 141: not Caliper-related based on the logs and code (blockchain-explorer), so closed it 143: Timeout issue, exhibiting the expected behavior, so closed it 150: really stale, without any error details, hasn't provided additional information ==> should close it 152: OP has been stale for a long time ==> wait a week for response 153: Fabric-related. Requested details ==> wait for a response 154: General questions, closed it. 163: stale and answered 165: proposed solution through a closed issue ==> wait for reply 168: Fabric-related. Requested details details 171: General question and answered. Closed it. 173: Fabric-related. Asked for progress 183: Platform support request. Definitely a won't fix for now. 195: Fabric-related. Asked for progress 199: Fabric-related. Requested details 201: Kafka related. Pointed to an other Kafka issue. 204-205: Fabric-related. My guess is resource issues, since it works for smaller network. Possibly duplicates 214: Not Caliper-related, so should close it Feel free to (dis)agree. I already closed some issues that were trivially not issues

klenik (Wed, 14 Nov 2018 23:20:22 GMT):
Reply with the list of issues that should be closed in your opinion

FlyingTiger (Tue, 20 Nov 2018 08:03:41 GMT):
Hi guys, I found a problem

FlyingTiger (Tue, 20 Nov 2018 08:03:57 GMT):
|---------|-----------------------------------|-------------|-------------|----------|----------|------------|-------------| | Docker | couchdb.peer0.org1.example.com | 73.2MB | 67.6MB | 27.07% | 14.31% | 899.0KB | 2.1MB | |---------|-----------------------------------|-------------|-------------|----------|----------|------------|-------------| | Docker | ca.org2.example.com | 5.1MB | 5.1MB | 0.08% | 0.01% | 0B | 0B | |---------|-----------------------------------|-------------|-------------|----------|----------|------------|-------------| | Docker | couchdb.peer1.org2.example.com | 73.0MB | 67.7MB | 28.28% | 16.10% | 1.4MB | 3.0MB | |---------|-----------------------------------|-------------|-------------|----------|----------|------------|-------------| | Docker | ca.org1.example.com | 4.9MB | 4.9MB | 0.06% | 0.00% | 0B | 0B | |---------|-----------------------------------|-------------|-------------|----------|----------|------------|-------------| | Docker | couchdb.peer1.org1.example.com | 113.4MB | 85.8MB | 62.54% | 18.25% | 2.4MB | 4.3MB | |---------|-----------------------------------|-------------|-------------|----------|----------|------------|-------------| | Docker | orderer.example.com | 45.2MB | 27.6MB | 24.64% | 5.66% | 4.5MB | 16.1MB | |---------|-----------------------------------|-------------|-------------|----------|----------|------------|-------------| | Docker | couchdb.peer0.org2.example.com | 86.3MB | 77.2MB | 38.89% | 17.20% | 1.8MB | 3.5MB

FlyingTiger (Tue, 20 Nov 2018 08:06:59 GMT):
When I posted PR, the travis-ci ran, and I found the usage of CPU of couchdb.peer*.org*.example.com dockers is too high

FlyingTiger (Tue, 20 Nov 2018 08:07:20 GMT):
Does anybody know what's wrong with it?

FlyingTiger (Tue, 20 Nov 2018 08:10:09 GMT):
It maybe cause the timeout problem. The result is Submitted: 1000 Succ: 93 Fail:0 Unfinished:907, etc.

FlyingTiger (Tue, 20 Nov 2018 08:18:19 GMT):
@nkl199 @klenik Could you please give me some information about this?

nkl199 (Tue, 20 Nov 2018 09:29:43 GMT):
I know that couch is an absolute CPU/memory hog, especially if using non-indexed queries

klenik (Tue, 20 Nov 2018 09:40:46 GMT):
Especially if you're running multiple instance on the same physical host

klenik (Tue, 20 Nov 2018 09:40:46 GMT):
Especially if you're running multiple instances on the same physical host

FlyingTiger (Tue, 20 Nov 2018 11:41:38 GMT):
OK, I got it. Thank you

panyu2 (Wed, 21 Nov 2018 01:13:40 GMT):
Hi，to whom concerns the regular meeting, the meeting will take place at UTC 6am on 11.21. The link of the meeting can be found on the Caliper's wiki https://wiki.hyperledger.org/projects/caliper

gregdhill (Mon, 26 Nov 2018 12:36:12 GMT):
Has joined the channel.

panyu2 (Tue, 27 Nov 2018 04:32:46 GMT):
Hi，to whom concerns the regular meeting, the meeting will take place at UTC 3pm on 11.27. The link of the meeting can be found on the Caliper's wiki https://wiki.hyperledger.org/projects/caliper

klenik (Tue, 27 Nov 2018 14:03:19 GMT):
@panyu2 @nkl199 @hurf Sorry folks, looks like I can't make it to today's call (conflict with the Budapest Hyperledger Meetup). Update: 1) Still need the YAML config support to integrate the new Fabric adapter. @houqinghui is working on it. 2) The adapter also supports Fabric v1.0 with minimal additional code (~30 LOC) while using the connection profile, so it'll be an all-in-one adapter for Fabric hopefully. 3) The documentation is still WIP. 4) If everything goes according to plan, I'll use the new adapter to demonstrate a high-throughput Fabric CC at the meetup :)

klenik (Tue, 27 Nov 2018 14:32:24 GMT):
Google Analytics for the health metrics table: 115 users since the start of the website :thumbsup: (now that the old docs have been removed, maybe this number will increase even more)

gregdhill (Tue, 27 Nov 2018 17:13:56 GMT):
Hey, I'm one of the developers from Monax (Hyperledger Burrow) and I'm currently exploring Caliper integrations. I'm quite new to NodeJS, so it would be great to have a brief chat with someone about the system this week! :)

klenik (Tue, 27 Nov 2018 20:57:53 GMT):
@gregdhill Hi Greg. Feel free to drop me a message and we can schedule something :)

klenik (Thu, 29 Nov 2018 11:19:35 GMT):
@panyu2 @nkl199 @houqinghui @FlyingTiger I left some comments on the FAQ section in the dev docs, every opinion/suggestion is welcome :)

nkl199 (Thu, 29 Nov 2018 11:19:46 GMT):
:thumbsup:

panyu2 (Fri, 30 Nov 2018 01:09:25 GMT):
:thumbsup: How to use caplier to test the self-deployed network is also a frequent question

panyu2 (Mon, 03 Dec 2018 07:43:18 GMT):
Hi，to whom concerns the regular meeting, the meeting will take place at UTC 3pm on 12.4. The link of the meeting can be found on the Caliper's wiki https://wiki.hyperledger.org/projects/caliper

panyu2 (Tue, 11 Dec 2018 00:04:18 GMT):
Hi，to whom concerns the regular meeting, the Caliper meeting is postponed this week due to the Global Forum.

Abhinavgrg074 (Fri, 14 Dec 2018 06:13:58 GMT):
Has joined the channel.

gregdhill (Sat, 15 Dec 2018 13:43:31 GMT):
Hey all, I've made a couple of PRs for Burrow related things...

gregdhill (Sat, 15 Dec 2018 13:43:38 GMT):
https://github.com/hyperledger/caliper/pull/258

gregdhill (Sat, 15 Dec 2018 13:43:48 GMT):
https://github.com/hyperledger/caliper/pull/257

klenik (Sun, 16 Dec 2018 11:19:21 GMT):
@gregdhill Awesome! I'll take a look at it in the evenings if the hotel wifi will allow it :sweat_smile:

panyu2 (Wed, 19 Dec 2018 00:55:34 GMT):
Hi，to whom concerns the regular meeting, the meeting will take place at UTC 6am on 12.9. The link of the meeting can be found on the Caliper's wiki https://wiki.hyperledger.org/projects/caliper

klenik (Wed, 19 Dec 2018 05:43:36 GMT):
Sorry, can't make it this morning, I'm at a blockchain winter school. Will send a report during the day.

panyu2 (Wed, 19 Dec 2018 05:58:29 GMT):
Got it.

panyu2 (Fri, 21 Dec 2018 03:25:04 GMT):
Hi，to whom concerns the regular meeting, since there are conflicts between the next two week's meeting time and vocations (Christmas holiday and New year‘s holiday)， these two meetings will be postponed. The latest meeting will be scheduled for UTC 6 am on January 9.

nkl199 (Fri, 21 Dec 2018 09:18:03 GMT):
hi, just some thoughts whilst I have them ... keen to explore the options here. We have been considering streamlining the Caliper repository, by moving out the benchmarks into a separate repository and publishing those in isolation, and I believe that is still a good direction. We would be decoupling the benchmarks with the source code .... and this is a good thing. A further consideration is the contract and networks folders, which are tightly coupled to the samples themselves as they also contain aspects related to the samples. My thoughts are that we should: - move the networks folder into the benchmarks (I think this works well as we can encapsulate the entire "benchmark" as a chaincode deployed on a specific DLT structure) - dynamically update the network config file with `chaincodes` and `context` present in the driving sample config file so that we can decouple the dependancies. This appears to be mainly present in the fabric and iroha networks - remove `contracts` from the `src` folder and nest them inside the `benchmarks` folder

nkl199 (Fri, 21 Dec 2018 09:18:03 GMT):
hi, just some thoughts whilst I have them ... keen to explore the options here. We have been considering streamlining the Caliper repository, by moving out the benchmarks into a separate repository and publishing those in isolation, and I believe that is still a good direction. We would be decoupling the benchmarks with the source code .... and this is a good thing. A further consideration is the contract and networks folders, which are tightly coupled to the samples themselves as they also contain aspects related to the samples. My thoughts are that we should: - move the networks folder into the benchmarks (I think this works well as we can encapsulate the entire "benchmark" as a chaincode deployed and tested on a specific DLT structure) - dynamically update the network config file with `chaincodes` and `context` present in the driving sample config file so that we can decouple the dependancies. This appears to be mainly present in the fabric and iroha networks - remove `contracts` from the `src` folder and nest them inside the `benchmarks` folder

nkl199 (Fri, 21 Dec 2018 09:18:03 GMT):
hi, just some thoughts whilst I have them ... keen to explore the options here. We have been considering streamlining the Caliper repository, by moving out the benchmarks into a separate repository and publishing those in isolation, and I believe that is still a good direction. We would be decoupling the benchmarks with the source code .... and this is a good thing. A further consideration is the contract and networks folders, which are tightly coupled to the samples themselves as they also contain aspects related to the samples. My thoughts are that we should: - move the networks folder into the benchmarks (I think this works well as we can encapsulate the entire "benchmark" as a chaincode deployed and tested on a specific DLT structure) - dynamically update the network config file with `chaincodes` and `context` present in the driving sample config file so that we can decouple the dependancies. This appears to be mainly present in the fabric and iroha networks, but we shouldn't have to maintain information in 2 locations for a single tests - remove `contracts` from the `src` folder and nest them inside the `benchmarks` folder

nkl199 (Fri, 21 Dec 2018 09:18:03 GMT):
hi, just some thoughts whilst I have them ... keen to explore the options here. We have been considering streamlining the Caliper repository, by moving out the benchmarks into a separate repository and publishing those in isolation, and I believe that is still a good direction. We would be decoupling the benchmarks with the source code .... and this is a good thing. A further consideration is the contract and networks folders, which are tightly coupled to the samples themselves as they also contain aspects related to the samples. My thoughts are that we should: - move the networks folder into the benchmarks (I think this works well as we can encapsulate the entire "benchmark" as a chaincode deployed and tested on a specific DLT structure) - dynamically update the network config file with `chaincodes` and `context` present in the driving sample config file so that we can decouple the dependancies. This appears to be mainly present in the fabric and iroha networks, but we shouldn't have to maintain information in 2 locations for a single tests - remove `contracts` from the `src` folder and nest them inside the `benchmarks` folder - modify Caliper to run with the new structure Once the 'benchmark specific' items are consolidated, we could them move them into a separate repository, leaving only the caliper src, which we can maintain and publish in isolation

nkl199 (Fri, 21 Dec 2018 09:23:39 GMT):
:point_up_tone1: let me know if you have any thoughts on the above :thumbsup:

gregdhill (Fri, 21 Dec 2018 11:48:19 GMT):
Hi @nkl199, I'm still quite new to Caliper but keen to contribute in any way I can! These seem like great ideas and I certainly agree with greater modularity and more code reuse where possible. I'm not sure about isolating the benchmarks in another repository though which I feel would bring too much overhead to users who simply want to get up and running, but I guess there is an argument for DLT providers who have their own benchmarks and just want to run the toolkit.

nkl199 (Fri, 21 Dec 2018 11:50:25 GMT):
yeah, it's a tough choice really, if the benchmarks were separated there would be a need to make their use as seamless as possible ... it is achievable, since it is what was done within Composer (the playground pulls sample networks from npm upon demand)

klenik (Fri, 21 Dec 2018 15:55:14 GMT):
Well, once caliper is published as an npm package, it can be easily installed into a cloned sample repo. So +1 from me to the separation idea. As for the networks. I tried to find some common structure, but in the end, I opted for the duplication. We could parameterize the compose files and the rest, but then it'll be hard to follow what's happening for a non-power user. So since the networks are examples and not critical core-code, I voted for duplication for the sake of simplicity.

panyu2 (Sat, 22 Dec 2018 01:28:34 GMT):
``` 1. I think we need to retain some simple examples even if Caliper is published as an npm package. If they are separated, it would make user difficulty to use it. ``` 2. About the networks, agree with @klenik . ``` ``` Since Caliper is a tool, all the changes we make must first consider whether it is convenient for users. ``` ```

panyu2 (Sat, 22 Dec 2018 01:28:34 GMT):
1. I think we need to retain some simple examples even if Caliper is published as an npm package. If they are separated, it would make user difficulty to use it. 2. About the networks, agree with @klenik . Since Caliper is a tool, all the changes we make must first consider whether it is convenient for users.

panyu2 (Sat, 22 Dec 2018 01:28:34 GMT):
Hi，to whom concerns the regular meeting, the meeting will take place at UTC 6am on 1.9. The link of the meeting can be found on the Caliper's wiki https://wiki.hyperledger.org/projects/caliper

nkl199 (Sat, 22 Dec 2018 08:08:32 GMT):
sure, though we could add an npm script into the package to pull down the samples into a known directory relative to caliper itself

panyu2 (Tue, 08 Jan 2019 08:47:21 GMT):
Hi，to whom concerns the regular meeting, the meeting will take place at UTC 8am on 1.9. The link of the meeting can be found on the Caliper's wiki https://wiki.hyperledger.org/projects/caliper

nkl199 (Wed, 09 Jan 2019 08:09:48 GMT):
.... anyone else joining the meeting?

panyu2 (Thu, 10 Jan 2019 08:56:23 GMT):
@nkl199 ，I'm sorry. I was on the business trip yesterday, so I missed the meeting.

panyu2 (Tue, 15 Jan 2019 06:12:45 GMT):
Hi，to whom concerns the regular meeting, the meeting will take place at UTC 8am on 1.16. The link of the meeting can be found on the Caliper's wiki https://wiki.hyperledger.org/projects/caliper

wangqq (Tue, 15 Jan 2019 06:15:06 GMT):
Has joined the channel.

klenik (Wed, 16 Jan 2019 07:23:19 GMT):
Can't make the call today (again), sorry. But good news is, that I've almost finished the documentation for the new adapter, so I'm raising the PRs today (finally)

panyu2 (Wed, 16 Jan 2019 07:34:16 GMT):
:thumbsup:

panyu2 (Wed, 16 Jan 2019 07:39:44 GMT):
@klenik @nkl199 about the FAQ, do you have some suggestions?

panyu2 (Wed, 16 Jan 2019 07:39:44 GMT):
@klenik @nkl199 about the FAQ, do you have any suggestions?

klenik (Wed, 16 Jan 2019 07:47:02 GMT):
it looks good in general, found some typos, so maybe a look-through is still needed (I suggest Grammarly, it's awesome)

gregdhill (Wed, 16 Jan 2019 11:07:30 GMT):
Hello again everyone :) and happy new year

gregdhill (Wed, 16 Jan 2019 11:07:44 GMT):
Got a new Burrow related PR https://github.com/hyperledger/caliper/pull/282

gregdhill (Wed, 16 Jan 2019 11:08:14 GMT):
also fixes some of the typos in the adapter documentation

Bayeed (Fri, 18 Jan 2019 20:53:35 GMT):
Has joined the channel.

Bayeed (Fri, 18 Jan 2019 20:54:12 GMT):
Hi! I have a simple node.js app which uses fabcar network. My intention is to test it with Caliper but having problems with lack of resources/documentations/tutorials on this. Can anyone help me out with some directions or suggestions? Thank you in advance.

amiec (Sat, 19 Jan 2019 15:32:42 GMT):
Has joined the channel.

amiec (Sat, 19 Jan 2019 15:40:04 GMT):
Hi all, I have a question about Caliper and the metric "stale block rate". "Stale blocks" refer to blocks that are valid and published despite the fact that an earlier block has already been published to the network. This obviously only occurs in Proof-of-Work-style consensus (such as PoET - which is why I'm bringing it up), and it occurs when two or more blocks are published very close in time to each other, such that a node publishes a block before seeing the earlier valid block. Stale block rate is an important metric for both performance and security of a PoW-style system. The higher the number of stale blocks, the more vulnerable the network is to forking attacks and the more performance degradation is experience by the validators (throughput goes down). The folks who wrote BLOCKBENCH have a way to measure stale block rate (at least, these metrics are published in this paper https://arxiv.org/pdf/1804.00399.pdf). I want to use Caliper with Sawtooth and get a handle on stale block rate in the network. My questions are: 1) has any work been done to this effect? 2) if not, would it be a good thing to contribute to this project? 3) if so, does anybody have any ideas for me in terms of how to start approaching that project? I've started to dissect Caliper code but it's certainly a lot to swallow! Please and thank you very much.

DmitriPlakhov (Mon, 21 Jan 2019 12:39:34 GMT):
Has joined the channel.

panyu2 (Tue, 22 Jan 2019 07:12:54 GMT):
@Bayeed Hope this website can help you. https://hyperledger.github.io/caliper/docs/1_Getting_Started.html

panyu2 (Tue, 22 Jan 2019 07:17:56 GMT):
@jiangfeihu

panyu2 (Tue, 22 Jan 2019 07:17:56 GMT):
@FlyingTiger what do you think about @amiec 's question?

panyu2 (Tue, 22 Jan 2019 07:21:03 GMT):
Hi，to whom concerns the regular meeting, the meeting will take place at UTC 3pm on 1.22. The link of the meeting can be found on the Caliper's wiki https://wiki.hyperledger.org/projects/caliper

rkrish82 (Tue, 22 Jan 2019 10:52:31 GMT):
I have raise the pr to support fabric 1.4 release changes. - https://github.com/hyperledger/caliper/pull/290

rkrish82 (Tue, 22 Jan 2019 10:52:39 GMT):
Please some one review it

rkrish82 (Tue, 22 Jan 2019 10:52:55 GMT):
@panyu2 @nkl199

FlyingTiger (Tue, 22 Jan 2019 12:03:20 GMT):
@amiec To your questions, first one, Caliper has no work been done to this effect. Second and third one, welcome to contribute, you could open an issue firstly, and describe why, what you want to do, and the goal etc, then we could talk on that issue and give some advice.

panyu2 (Tue, 22 Jan 2019 15:03:50 GMT):
anyone else join the meeting?

FlyingTiger (Wed, 23 Jan 2019 03:44:58 GMT):
@amiec Could you please describe how to calculate the metric "stale block rate" for sawtooth? So that we could judge could Caliper add this metric, and how?

amiec (Fri, 25 Jan 2019 17:49:44 GMT):
[ ](https://chat.hyperledger.org/channel/caliper-contributors?msg=F9qWGeQdBkdMLr9vN) @FlyingTiger Good idea, I will get back to you when I know more of these details (I think studying the BLOCKBENCH code may help me understand how that metric is technically acquired), and then will open an issue. Thanks for your response!

FlyingTiger (Sat, 26 Jan 2019 08:34:29 GMT):
@amiec OK, my pleasure

panyu2 (Mon, 28 Jan 2019 00:58:25 GMT):
Hi，to whom concerns the regular meeting, the meeting will take place at UTC 3pm on 1.29. The link of the meeting can be found on the Caliper's wiki https://wiki.hyperledger.org/projects/caliper

rkrish82 (Tue, 29 Jan 2019 15:04:28 GMT):
@pany2

rkrish82 (Tue, 29 Jan 2019 15:05:37 GMT):
@panyu2 @FlyingTiger Did you guys get a chance to review the changes?

rkrish82 (Tue, 29 Jan 2019 15:05:47 GMT):
PR #290

rkrish82 (Tue, 29 Jan 2019 15:06:02 GMT):
https://github.com/hyperledger/caliper/pull/290

panyu2 (Tue, 29 Jan 2019 15:22:47 GMT):
@rkrish82 Sorry for late review. We are looking into these new support prs.

rkrish82 (Tue, 29 Jan 2019 15:23:54 GMT):
Thanks

panyu2 (Sun, 03 Feb 2019 07:31:09 GMT):
Hi，to whom concerns the regular meeting, since the Chinese Spring Festival is coming, the next meeting on 2.6 will be postponed.

rjones (Mon, 04 Feb 2019 18:06:08 GMT):
Has left the channel.

klenik (Mon, 04 Feb 2019 21:00:49 GMT):
@panyu2 @FlyingTiger @nkl199 If someone has some free time, the new Fabric adapter and documentation is ready: https://github.com/hyperledger/caliper/pull/296 https://github.com/hyperledger/caliper/pull/297 It's "just" two JS files, and a bunch of examples, so never mind the number of added file :) If you have any questions to make the review easier, just ask. In the meatime, I'll take a look at the other pending PRs.

panyu2 (Mon, 11 Feb 2019 00:52:16 GMT):
@klenik we are just back from Spring Festival Holiday. We'll review them as soon as possible.

panyu2 (Mon, 11 Feb 2019 00:54:49 GMT):
@rkrish82 I had left some comments under your pr one week ago. Since new prs were merged, there are some conflicts. Would you please resolve them?

rkrish82 (Mon, 11 Feb 2019 03:12:49 GMT):
@panyu2 sure, i will address them

panyu2 (Tue, 12 Feb 2019 00:47:24 GMT):
Hi，to whom concerns the regular meeting, the meeting will take place at UTC 8 am on 2.13. The link of the meeting can be found on the Caliper's wiki https://wiki.hyperledger.org/display/caliper

panyu2 (Tue, 12 Feb 2019 00:47:54 GMT):
@rkrish82 , I saw your updates. I'll do some tests.

panyu2 (Thu, 14 Feb 2019 07:47:00 GMT):
@rkrish82 I've left one reply under your pr. There might have something to change.

panyu2 (Thu, 14 Feb 2019 07:51:09 GMT):
Hi, the bootcamp will hold at Mar 7~8,2019. Welcome everyone interested with Caliper to join Caliper's session and tutorial.

panyu2 (Tue, 19 Feb 2019 03:45:09 GMT):
Hi，to whom concerns the regular meeting, the meeting will take place at UTC 3 pm on 2.19. The link of the meeting can be found on the Caliper's wiki https://wiki.hyperledger.org/display/caliper

hurf (Wed, 20 Feb 2019 10:53:19 GMT):
Hi all, as suggested by Attila, we found the current rotating Caliper call schedule is kind confusing and not easy for people to keep track with. So we're going to simplify the schedule to a fixed one, as every Wednesday, 9am UTC time.

panyu2 (Tue, 26 Feb 2019 09:53:05 GMT):
Hi，to whom concerns the regular meeting, the meeting will take place at UTC 9 am on 2.27. The link of the meeting can be found on the Caliper's wiki https://wiki.hyperledger.org/display/caliper

klenik (Tue, 26 Feb 2019 13:20:30 GMT):
@panyu2 @hurf @FlyingTiger Started adding some potential bootcamp tasks to the end of the dev docs. Intended topics: improve logging support, improve config support, improve module life-cycle management/protocol, modularize rate controller implementations, modularize user callback implementations (the loggig task is almost finished in the doc)

klenik (Tue, 26 Feb 2019 14:03:23 GMT):
@rjones Could you update the meeting calendar for Caliper please? Now we have a fix schedule for every Wednesday at 9AM UTC. Thanks!

rjones (Tue, 26 Feb 2019 14:03:25 GMT):
Has joined the channel.

rjones (Tue, 26 Feb 2019 19:27:08 GMT):
@klenik please send a request to community-architects@hyperledger.org and lay out directly what you need, thanks

klenik (Tue, 26 Feb 2019 19:28:02 GMT):
@rjones will do!

rjones (Tue, 26 Feb 2019 19:28:32 GMT):
Thanks! There are four of us on that alias, and sending it in email ensures someone will see it :)

rjones (Tue, 26 Feb 2019 19:28:32 GMT):
Thanks! There are four of us on that alias, and sending it in email ensures someone will see it :O)

rjones (Tue, 26 Feb 2019 19:29:40 GMT):
I know we did a bunch of adjustments in the last week, as well, when @panyu2 asked for it

rjones (Tue, 26 Feb 2019 19:32:09 GMT):
when I say _we_ I mean _I_

klenik (Tue, 26 Feb 2019 19:32:41 GMT):
Yes, but since then we came to the conclusion that the flexible (and complicated) schedule didn't bring new people to the calls, however, it complicated the life of the core developers 😁

rjones (Tue, 26 Feb 2019 19:32:42 GMT):
right now, it's 0900-1000 UTC

rjones (Tue, 26 Feb 2019 19:33:29 GMT):
there's only one meeting on the calendar that I see

klenik (Tue, 26 Feb 2019 19:36:05 GMT):
Ah, you're right! Today my calendar notified me about the old schedule B meeting, that's why I started this thread. Should've checked first, sorry 😅

rjones (Tue, 26 Feb 2019 19:36:52 GMT):
I did send cancellation notices to the list, I thought. Perhaps I missed one though

panyu2 (Tue, 05 Mar 2019 01:23:13 GMT):
Hi，to whom concerns the regular meeting, since the team will on the way to attend the bootcamp event on 3.7~3.8 at HongKong, tomorrow's meeting will be canceled. Welcome everyone who will attend the bootcamp event to join Caliper's activities at Table 3.

panyu2 (Wed, 13 Mar 2019 05:51:34 GMT):
Hi，to whom concerns the regular meeting, since there is a conflict on the conference today, today's meeting will be canceled.

MHBauer (Tue, 19 Mar 2019 01:24:06 GMT):
Has joined the channel.

panyu2 (Tue, 19 Mar 2019 08:41:56 GMT):
Hi，to whom concerns the regular meeting, the meeting will take place at UTC 9 am on 3.20. The link of the meeting can be found on the Caliper's wiki https://wiki.hyperledger.org/display/caliper

klenik (Wed, 20 Mar 2019 08:56:56 GMT):
I'll be a few minutes late, sorry

nkl199 (Thu, 21 Mar 2019 13:11:44 GMT):
some more information on the potential use of worker threads for local clients: https://blog.logrocket.com/node-js-multithreading-what-are-worker-threads-and-why-do-they-matter-48ab102f8b10

nkl199 (Thu, 21 Mar 2019 13:12:00 GMT):
they are an experimental feature of node 10.5

nkl199 (Thu, 21 Mar 2019 13:12:00 GMT):
they are an experimental feature of node 10.5 - some more information here, relating to Node11 https://codeforgeek.com/2019/01/getting-started-node-worker-thread/

nkl199 (Mon, 25 Mar 2019 17:00:34 GMT):
Hi all, just to let you know that I have started on the refactor to enable npm publishing ... I would recommend holding off any big PRs until it is complete/approved/delivered :joy:

rjones (Mon, 25 Mar 2019 17:11:45 GMT):
Has left the channel.

hurf (Tue, 26 Mar 2019 11:50:25 GMT):
Copy that

panyu2 (Wed, 27 Mar 2019 05:53:36 GMT):
Hi，to whom concerns the regular meeting, the meeting will take place at UTC 9 am on 3.27. The link of the meeting can be found on the Caliper's wiki https://wiki.hyperledger.org/display/caliper

klenik (Wed, 27 Mar 2019 08:51:08 GMT):
Might be a few minutes late

rohanjulka19 (Mon, 01 Apr 2019 18:14:09 GMT):
Has joined the channel.

ZichengWang (Mon, 01 Apr 2019 20:44:30 GMT):
Has joined the channel.

amiec (Tue, 02 Apr 2019 23:54:41 GMT):
Hi all - I've been using Caliper with Sawtooth and have generated some initial experimental runs. A given experiment involves a series of rounds in which the txn delivery rate increases from ~5 tps to 45 tps stepping by 10 tps. Each round is duration-based and lasts about a minute. I'm seeing some strange measurements where the throughput is reported as 0 for a given round with some frequency. I'm in the process of trying to figure out what's going on here, and I have a few theories that are likely to do with my backend system, but in order to think clearly I need to know EXACTLY how Caliper measures and reports "throughput". I would greatly appreciate any thoughts on the following questions: How is throughput calculated using the Sawtooth adaptor? For example, are transactions considered "put through" once they have been committed to the blockchain in a block? Additionally, how does this notion of throughput span across various rounds in the same experiment? For example, the backend system doesn't know anything about the rounds... so it seems possible that block commits that occur very close to the border of rounds (duration-based in particular) could have the power to distort the reported throughput, causing some rounds to report high throughput while neighboring rounds report little or no throughput? How does Caliper delimit throughput measurements with regard to rounds? My difficulties arise from the fact that PoET consensus has a probabilistic aspect to it, and while blocks may be published *on average* at a rate of target_wait_time, there may be anomalous wait times that cause blocks to be published in a more sporadic way. I would greatly appreciate any wisdom or advice that this community can share with me that will help me better design my experiments and trust the output. I hope to contribute significantly to the Caliper project, but I definitely need some help! Thank you!!

klenik (Wed, 03 Apr 2019 07:11:50 GMT):
Can't make it to today's call, I have a conflicting meeting :/

nkl199 (Wed, 03 Apr 2019 08:10:49 GMT):
sorry - was 10 mins late today ... tho suspect there might be a hiccup this week due to daylight savings kicking in and making me 50mins early instead :joy:

nkl199 (Wed, 03 Apr 2019 08:25:32 GMT):
@amiec - thanks for reaching out. I will confess to have mainly been involved in Fabric/Composer benchmarks and not Sawtooth .. though I am cutting my teeth with that one now. From an initial dive into the code, the Sawtooth adaptor is similar to FabricComposer in that it waits for block commit events until marking a transaction as successful. My knowledge on Sawtooth itself is a little weak, but I know in Fabric the blocks are written based on both size and timeout limits. Each round is sequenced with a time interval (5 second gap between rounds iirc), and rounds only proceed upon completion (pass/fail) of all transactions in the current round. Time interval benchmarks submit transactions for a given time, but will still wait for all of the submitted transactions to pass/fail before moving on. Each round has it's own throughput measurement, tho the console displayed statistics do confuse this because as the benchmark runs it shows details of the complete benchmark and not individual benchmark rounds (ie the current status of success/fail/unfinnished txns)

nkl199 (Wed, 03 Apr 2019 11:09:31 GMT):
Hi all ( @panyu2 @klenik @hurf ). As promised, I have been working on the refactor to split the Caliper code base into modules that can be published to NPM. This is now available for inspection/comment on my personal branch https://github.com/nklincoln/caliper-1/tree/refactor-for-publishing Working adaptors are: -fabric -fabric-ccp -composer One of the new dependancies in the "main" Caliper repository is Lerna, which helps with dependancy management between all the packages. If you clone my branch, to build it you will need to run: - `npm run repoclean` - `npm run bootstrap` In the root caliper folder. To run a benchmark, there is a ReadMe file in /caliper/pacakges/caliper-application that essentially runs the `run.js` script with some args. Remember that only 3 of the adaptors are currently working ... Iroha seems to ignore the expected Caliper operation, I on the maintainers calls this morning I heard thaa the Sawtooth adapter has been modified

nkl199 (Wed, 03 Apr 2019 11:11:50 GMT):
My suspicion is that there will soon be another pacakge called caliper-cli, which is essentially just the run.js script ... once users have that, then they can run a benchmark by passing a few workspace folder items only.... rather neat! We would still need the caliper-application package to include in our regression tests

nkl199 (Wed, 03 Apr 2019 11:33:51 GMT):
I will continue working on remaining adapters and update the branch as I do so

nkl199 (Wed, 03 Apr 2019 15:16:08 GMT):
... anyone here have experience using caliper with either: -burrow -sawtooth ?

amiec (Wed, 03 Apr 2019 15:53:53 GMT):
@nkl199 thanks so much for the prompt response! A followup question if you don't mind: my workloads are sometimes going to exceed the processing capacity of my validator and/or consensus algorithm. For a while, this means transactions queuing up, until eventually they begin to be dropped entirely (when the queue is full). Based on what you know about how Caliper calculates its metrics and how it handles rounds, what effect might this have on my results? Because you mention that it will submit transactions for the given duration, but then still wait for all the submitted transactions to pass/fail before moving on... If a transaction is completely dropped (never makes it into a queue), does Caliper handle this correctly and call it a "failure" ? I'm trying to figure out how this might lead to round measurements that are 0...

nkl199 (Wed, 03 Apr 2019 16:03:04 GMT):
Hmmm, good question! I would recommend using the `FixedFeedbackRate` controller that allows you to target a fixed TPS, but will throttle back if a txn threshold is exceeded. Caliper will consider a dropped txn as a failure (so long as the SUT rejects the transaction via timeout ... fabric does this for sure). Any failures will skew results, and sometimes make the possible tps

amiec (Wed, 03 Apr 2019 18:46:51 GMT):
FixedFeedbackRate! That's a great idea and will try some experiments with that. I'm not sure what Sawtooth does regarding dropped transactions... I'm not sure if a Sawtooth SUT formally rejects it in a way that Caliper would be notified about... It may be the case that it is simply ignored completely in the case of a full queue (in which case it sounds like that would screw up Caliper). I'll try to ask the Sawtooth folks.

amiec (Wed, 03 Apr 2019 19:31:51 GMT):
Ok so the Sawtooth developers said that "you will get a "queue full" error, as an HTTP 429 error status" when a transaction is submitted to a validator that already has a full queue. Does anybody know if/how Caliper reacts to a message of this type?

panyu2 (Thu, 04 Apr 2019 01:27:13 GMT):
@nkl199 , Burrow is written by @gregdhill ; @houqinghui is currently working on irhoa's adapter; as for the sawtooth, you can ask @FlyingTiger the adapter issues.

panyu2 (Thu, 04 Apr 2019 01:34:23 GMT):
@nkl199 , this PR (https://github.com/hyperledger/caliper/commit/382c58c0fe15cf4c23a580747a788c881f4cbbce#diff-e991dbbc99a0c765d2e19e0a88cbd007) specified current supporting 1.0.5 version Transaction Processor of sawtooth. just a small change.

nkl199 (Thu, 04 Apr 2019 11:31:14 GMT):
Thanks - I've rolled in the above change and the PR400.... I have also got the sawtooth adaptor working now ...

nkl199 (Thu, 04 Apr 2019 11:47:28 GMT):
OK, so my current investigations on `main` are that neither iroha or burrow have working benchmarks ... can anyone else confirm?

nkl199 (Thu, 04 Apr 2019 13:27:46 GMT):
Hi, Travis build has now been fixed - it fails on failing benchmarks and passes when it should only. It has been swapped across to run benchmarks using the new packages :thumbsup: . Pending confirmation on the state of burrow/iroha benchmarks before raising the official PR ... then comes the small issue of updating the docs :joy:

klenik (Thu, 04 Apr 2019 14:50:30 GMT):
@nkl199 I'll look into your fork tonight/tomorrow. About the adapter interfaces: we really need to unify them on the init/invoke/query level, otherwise writing quasi-crossplatform benchmarks will be a nightmare.

amiec (Fri, 05 Apr 2019 15:30:32 GMT):
[ ](https://chat.hyperledger.org/channel/caliper-contributors?msg=YCZxFTQjN9hTed98x) @nkl199 Hi @nkl199 , two quick followup questions if you can: 1) I started performing some experiments with FixedFeedbackRate as my ratecontroller, and I used values for unfinished_per_client of first 100, and then of 10. My delivery rate was 40 TPS, which I expect to be a little bit high for my system but not crazy. In both cases, I saw plenty of throughput measurements of 0.0... Despite running the simulation for between 1000 and 2000 seconds (should have been PLENTY of time to get many blocks published, throughput should definitely not be 0, unless there is some sort of thrashing going on, or perhaps errors in the measurement system) My question is then: do you have any recommended value for unfinished_per_client?? I was additionally confused by the fact that I saw many queue_full errors popping up during these fixed-feedback simulations (many more than 10!), and I would have thought that the fixed_feedback controller would have throttled back in that case... it makes me wonder if it is working correctly for Sawtooth?? 2) Your last sentence "Any failures will skew results, and sometimes make the possible tps" seems incomplete, and I'm wondering if you can elaborate on how transaction failures might skew results? In what way might they be skewed?

nkl199 (Mon, 08 Apr 2019 08:31:19 GMT):
Hi @amiec - yeah, that sentence was cut off (not sure where the rest of it went!) - failing transactions complete faster than passing transactions (in general), so the resulting TPS will be higher than it should be

nkl199 (Mon, 08 Apr 2019 08:31:56 GMT):
I'd recommend @FlyingTiger for sawtooth adapter advice

FlyingTiger (Mon, 08 Apr 2019 12:26:05 GMT):
@amiec Could you please provide the whole log file for your test? The log file default locate on caliper/log directory. Caliper Sawtooth adapter now do not use specifically an HTTP 429 response.

amiec (Mon, 08 Apr 2019 23:34:29 GMT):
Thank you @nkl199 and @FlyingTiger ! FlyingTiger I've sent the logs via a DM. I have some more questions and I'll just start with one here to keep from overwhelming: If the adapter doesn't use the 429 response, what is the "lifecycle" of a transaction that is submitted to a full queue and therefore dropped? For example: Caliper generates and submits the transaction... it has some internal record of having done so. The transaction is dropped and a 429 response comes back to Caliper but is ignored. Caliper then.... considers this transaction "unfinished"? for how long? are there timeouts? How long until the transaction is considered a "failure"? Is it ever? With regard to the fixed-feedback-rate controller... it needs to have a notion of how many "unfinished" there are at any given time, so what is that notion? Thank you!

panyu2 (Wed, 10 Apr 2019 02:41:42 GMT):
Hi，to whom concerns the regular meeting, the meeting will take place at UTC 9 am on 4.10. The link of the meeting can be found on the Caliper's wiki https://wiki.hyperledger.org/display/caliper

klenik (Wed, 10 Apr 2019 07:25:57 GMT):
Apologies, can't make today's call :/

rjones (Sat, 13 Apr 2019 00:08:41 GMT):
Has joined the channel.

rjones (Sat, 13 Apr 2019 00:08:51 GMT):
Could I ask Caliper maintainers to please join https://lists.hyperledger.org/g/maintainers ? I'd appreciate it.

rjones (Mon, 15 Apr 2019 19:40:49 GMT):
Could I persuade the owner of the planning document in the channel header to migrate it to the wiki? https://docs.google.com/document/d/1QQFmuRW4QXk0HCUIw1NoqweNQBnQ4P3eOhAE10m1vng/edit#

rjones (Mon, 15 Apr 2019 19:41:30 GMT):
the tutorial and planning document are already linked from https://wiki.hyperledger.org/display/caliper

panyu2 (Tue, 16 Apr 2019 01:42:30 GMT):
@rjones , the planning document which is same as the 'Development Working Document' has already been linked to the wiki.

rjones (Tue, 16 Apr 2019 01:42:58 GMT):
@panyu2 what I mean, is the content on the wiki

panyu2 (Tue, 16 Apr 2019 01:47:31 GMT):
Got it. We are making a new plan, and after it is done we will release it to the wiki and current document will be replaced. @rjones

rjones (Tue, 16 Apr 2019 01:47:55 GMT):
OK thank you very much

panyu2 (Tue, 16 Apr 2019 02:15:55 GMT):
@rjones joined the group.

panyu2 (Tue, 16 Apr 2019 02:15:57 GMT):
Hi，to whom concerns the regular meeting, the meeting will take place at UTC 9 am on 4.17. The link of the meeting can be found on the Caliper's wiki https://wiki.hyperledger.org/display/caliper

klenik (Wed, 17 Apr 2019 08:02:33 GMT):
@panyu2 @nkl199 @hurf @FlyingTiger The call starts now, right? Or did I mess up the time again?

houqinghui (Wed, 17 Apr 2019 08:04:51 GMT):
@klenik the meeting will take place at UTC 9 am on 4.17. The link of the meeting can be found on the Caliper's wiki https://wiki.hyperledger.org/display/caliper

klenik (Wed, 17 Apr 2019 08:07:28 GMT):
Ah, summer time, love these fluctuating time zones... "see" you in an hour :)

nkl199 (Wed, 17 Apr 2019 11:13:02 GMT):
something to look at using: https://sonarcloud.io

nkl199 (Wed, 17 Apr 2019 11:13:14 GMT):
gives a very good GA gate for the code base

nkl199 (Wed, 17 Apr 2019 11:14:42 GMT):
-> highlights bugs, "code-smells" etc

nkl199 (Wed, 17 Apr 2019 11:14:58 GMT):
we should of course be aiming to carry zero bugs ;-)

nkl199 (Wed, 17 Apr 2019 11:17:18 GMT):
and it is free for open source projects :tada: :money_with_wings:

julient (Tue, 23 Apr 2019 16:29:16 GMT):
Has joined the channel.

panyu2 (Wed, 24 Apr 2019 06:01:16 GMT):
Hi，to whom concerns the regular meeting, the meeting will take place at UTC 9 am on 4.24. The link of the meeting can be found on the Caliper's wiki https://wiki.hyperledger.org/display/caliper

klenik (Wed, 24 Apr 2019 06:15:27 GMT):
@panyu2 et al. Hi, won't be able to attend the call today, project work stuff conflict... Update: I improved the runtime config (nconf) support, writing the docs for it currently, plus planning the "test your own fabric network" tutorial.

panyu2 (Wed, 24 Apr 2019 06:16:14 GMT):
@klenik got it.

panyu2 (Wed, 24 Apr 2019 06:18:32 GMT):
@klenik Do you have any suggestions on Nick's PR 405？

klenik (Wed, 24 Apr 2019 06:20:40 GMT):
Didn't have a chance to look at it yet in detail, I'll try to commit some time to it tonight

houqinghui (Wed, 24 Apr 2019 09:49:50 GMT):
@nkl199 Hi Nick, what were you doing when we were having the meeting today?

houqinghui (Wed, 24 Apr 2019 09:49:50 GMT):
@nkl199 Hi Nick, are you busy today, we were looking forward hearing from you in today's meeting.

nkl199 (Wed, 24 Apr 2019 13:46:44 GMT):
apologies, i've been travelling (currently in the states) and lost track of what day it is :face_palm_tone2:

houqinghui (Thu, 25 Apr 2019 01:02:28 GMT):
Got it. Have a nice trip.

nkl199 (Thu, 25 Apr 2019 13:36:50 GMT):
thanks :D I saw that the new packaging structure has been merged (As has the docs) - I'm going to raise some issues relating to what was spotted during the review process so that they do not get lost

amiec (Thu, 25 Apr 2019 13:50:37 GMT):
Does Caliper work with MacOS?

nkl199 (Thu, 25 Apr 2019 13:57:01 GMT):
yes

nkl199 (Thu, 25 Apr 2019 13:57:01 GMT):
yes - I use a mac :thumbsup:

julient (Mon, 29 Apr 2019 07:15:13 GMT):
Hello everyone! Are you planning / already working on benchmarking Corda? Is there I way I could contribute to this work? thanks for your reply, Julien

julient (Mon, 29 Apr 2019 07:15:13 GMT):
Hello everyone! Are you planning / already working on benchmarking Corda? Is there a way I could contribute to this work? thanks for your reply, Julien

panyu2 (Tue, 30 Apr 2019 00:57:56 GMT):
@julient , we are planning to work on Corda, but this work has not started yet. Any contribution to this work is appreciated. As for supporting Corda, an adapter should be written. Here is the documentation about how to write an adapter https://hyperledger.github.io/caliper/docs/Writing_Adapters.html, you can read this and other existed adapters and learn to write an corda adapter. Feel free to ask any questions about it. :slight_smile:

panyu2 (Tue, 30 Apr 2019 01:11:39 GMT):
Hi，to whom concerns the regular meeting, since there is a conflict between the regular meeting and International Labor Day vocation, the meeting on 5.1 will be canceled .

nkl199 (Thu, 02 May 2019 09:17:55 GMT):
@russanto - Hi, figured that I should answer your question in the general channel here. The packaging refactor is the biggest change that is going to be present going forward, it was necessary in order to enable publishing modules to npm. There is a roadmap being worked on and that will be released ASAP, but there should not be any significant changes any time soon - the biggest future change is moving the caliper clients into Docker containers, though this is not planned just yet

russanto (Thu, 02 May 2019 09:17:56 GMT):
Has joined the channel.

russanto (Mon, 06 May 2019 22:32:28 GMT):
I submitted a PR with the ethereum adapter I wrote

russanto (Mon, 06 May 2019 22:32:28 GMT):
I submitted a PR with the ethereum adapter I wrote. I hope it can be useful.

panyu2 (Tue, 07 May 2019 03:02:04 GMT):
@russanto :thumbsup:, we will review it soon.

panyu2 (Tue, 07 May 2019 03:05:01 GMT):
Hi，to whom concerns the regular meeting, some of the team members will attend a blockchain meeting in China during the regular meeting time, so this week's meeting will be canceled.

nkl199 (Tue, 07 May 2019 16:27:07 GMT):
With regards to the above - I'll hop on the call at the regular time anyhow, just in case anyone else turns up ... if nobody is there after 10mins, I'll hang up :thumbsup:

hurf (Wed, 08 May 2019 05:39:53 GMT):
@russanto thanks a lot for your contribution. It is exact what Caliper needed.

hurf (Wed, 08 May 2019 05:41:39 GMT):
Recently we've been changing folder structures to support easier packaging and installation. Sorry for the inconvenience brought to you.

hurf (Wed, 08 May 2019 05:42:54 GMT):
It is planned to have some change to the API and try to fix it to make the development stable.

hurf (Wed, 08 May 2019 05:44:27 GMT):
The roadmap is under discussion, we had a draft of it. I'm happy you can also join the discussion. @houqinghui can share with you the draft roadmap including milestones and planned features.

nkl199 (Fri, 10 May 2019 13:01:20 GMT):

Clipboard - May 10, 2019 2:01 PM

nkl199 (Fri, 10 May 2019 13:04:11 GMT):
Hi, I've been experimenting with a few things and I have the following to report: - the use of worker threads is currently blocked by gRPC. The underlying C implementation would need to be changed in order for the javascript wrapper to operate correctly. Even if we were to make that change, getting the PR in and then picked up and/or backported is unrealistic. The best option left for us here is to press on with dockerising the clientFactory, and have it spawn client containers of a specific flavour (node/java/go etc) - I've created a caliper-cli package that enables running of a benchmark using the command `caliper benchmark run -w -c -n `, if we go ahead with this then we should look at pulling all the utility scripts into it (`caliper benchmark list` | `caliper zooclient start` etc)

klenik (Fri, 10 May 2019 13:06:55 GMT):
> have it spawn client containers of a specific flavour @nkl199 What do you mean by this? Since we don't have a common glossary, I'm a little lost in the client terminology :sweat_smile:

nkl199 (Fri, 10 May 2019 13:08:56 GMT):
haha sorry - a ClientFactory Docker container could spawn and communicate with new CaliperClient Docker containers ... since we would likely be pulling those from `dockerhub/caliper` we could have different sdk clients as opposed to the current nodeJS limitation

nkl199 (Fri, 10 May 2019 13:08:56 GMT):
haha sorry - a ClientFactory Docker container could spawn and communicate with new CaliperClient Docker containers ... since we would likely be pulling those from `dockerhub/caliper` we could have different language sdk clients as opposed to the current nodeJS limitation

klenik (Fri, 10 May 2019 13:12:45 GMT):
so basically we open up the possibility to implement the adapters in other languages with other SDKs while keeping the perf and metric monitoring/reporting intact? :thinking:

nkl199 (Fri, 10 May 2019 13:20:39 GMT):
yup

klenik (Fri, 10 May 2019 14:15:56 GMT):
Although the rate controller and user callback logic are in the same process for efficiency reasons, so those must be ported to different languages as well if we want multi language support. But first let's stabilize the Node.JS implementation :D

nkl199 (Mon, 13 May 2019 12:34:29 GMT):
... settled at the Consensus/Construct conference, it's nice here (aside from the weather) :thumbsup:

nkl199 (Mon, 13 May 2019 12:34:29 GMT):
... settled at the Consensus/Construct conference, it's nice here (aside from the weather) :thumbsup: This does mean that I will be unable to attend the maintainers call this week tho ... my main updates are above though.

rjones (Mon, 13 May 2019 12:50:31 GMT):
@nkl199 stop by the booth - we have trucker hats!

houqinghui (Tue, 14 May 2019 03:59:47 GMT):
Hi，to whom concerns the regular meeting, the meeting will take place at UTC 9 am on 5.15. The link of the meeting can be found on the Caliper's wiki https://wiki.hyperledger.org/display/caliper

russanto (Wed, 15 May 2019 10:10:53 GMT):
I completely forgot about the meeting, I would participate.

russanto (Wed, 15 May 2019 10:11:52 GMT):
Why smallbank benchmark is the only one to use "transaction_type" key for the workload generation instead of "verb" ?

klenik (Wed, 15 May 2019 10:31:25 GMT):
@russanto I guess Sawtooth expects the transaction spec in a different format. We'll need to come up with a high level builder pattern for creating tx invoke options, so this part of the user modules can also be platform independent (more or less)

houqinghui (Thu, 16 May 2019 01:16:29 GMT):
@russanto Ok, see you next week.

houqinghui (Thu, 16 May 2019 01:29:35 GMT):
@klenik Hi attila, were you busy yesterday? we were looking forward to hearing from you in the meeting.

klenik (Thu, 16 May 2019 11:05:08 GMT):
@houqinghui Yes, sorry, I have a paper deadline tomorrow, so life is quite chaotic right now (so much that I even completely forgot about the call) :/ Anything in particular you'd like to discuss? We could utilize this channel more, it's faster than discussing things only on the weekly calls :)

nkl199 (Thu, 16 May 2019 11:39:45 GMT):
+1 to the above

nkl199 (Thu, 16 May 2019 11:40:01 GMT):
(more discussion on this channel on work in progress etc)

raj_shekhar (Thu, 16 May 2019 11:53:14 GMT):
Has joined the channel.

circlespainter (Sat, 18 May 2019 07:37:24 GMT):
Has joined the channel.

raj_shekhar (Mon, 20 May 2019 05:29:08 GMT):
Hi Team, Is Caliper is compatible with fabric 1.4.x??

houqinghui (Mon, 20 May 2019 07:05:42 GMT):
@raj_shekhar Yes, you can use Caliper to test your own blockchain network if it has been supported. This is the doc site https://hyperledger.github.io/caliper/docs/1_Getting_Started.html, which can guide you how to use Caliper.

raj_shekhar (Mon, 20 May 2019 07:10:04 GMT):
In the documents it is mentioned that "the lastest version that has been verified is v1.1.0"

raj_shekhar (Mon, 20 May 2019 07:10:04 GMT):
In the documents it is mentioned that "the lastest version that has been verified is v1.1.0" @houqinghui

raj_shekhar (Mon, 20 May 2019 07:10:23 GMT):
I want to know if anybody tried for fabric 1.4.x

klenik (Mon, 20 May 2019 08:39:43 GMT):
@raj_shekhar The doc is probably outdated on that account. Check out the Fabric-ccp adapter doc, it's definitely uo2date

klenik (Mon, 20 May 2019 08:39:43 GMT):
@raj_shekhar The doc is probably outdated on that account. Check out the Fabric-ccp adapter doc, it's definitely up2date

raj_shekhar (Mon, 20 May 2019 08:40:48 GMT):
yeah

raj_shekhar (Mon, 20 May 2019 08:41:22 GMT):
just one more thing - what is the difference between , Fabric and Fabric CCP ?

klenik (Mon, 20 May 2019 08:43:45 GMT):
The latter was implemented from scratch to utilize the common connection profile (ccp) feature of the sdk. It has some extra features from a usability point of view.

raj_shekhar (Mon, 20 May 2019 08:44:27 GMT):
thanks @klenik

houqinghui (Tue, 21 May 2019 02:07:28 GMT):
Hi，to whom concerns the regular meeting, the meeting will take place at UTC 9 am on 5.15. The link of the meeting can be found on the Caliper's wiki https://wiki.hyperledger.org/display/caliper

russanto (Tue, 21 May 2019 09:29:36 GMT):
@klenik I'm working on the ethereum adapter to bring some modification and improvements. When finished I'll dedidcate some time to the doc you requested on the PR

klenik (Tue, 21 May 2019 21:01:43 GMT):
Just some agenda item ideas for tomorrow's call (not in priority order): 1) Deprecating the old Fabric adapter (hard to maintain, lot of related issues/fires to put out, only provides the extra tx-mode feature compared to the CCP adapter, and haven't seen active usage of it in the community) 2) Modularizing the core package (make the flow, monitor, client, reporting parts more object-oriented with clean APIs), a must-have for starting versioning 3) Brainstorming about how a useful Caliper UI would look like (more importantly, what it should be able to do), this would facilitate the integration of the internship project that starts soon 4) Being more strict about github issues (misconfig should be forwarded here or to the mailing list, only real bugs and enhancements should be tracked through github issues) 5) Cathcing up about what's everyone working on :) 6) Other things you can think of

klenik (Wed, 22 May 2019 07:39:13 GMT):
@houqinghui @panyu2 Just to be sure, the call starts in 1 hour and 20 minutes, right? :D

houqinghui (Wed, 22 May 2019 07:39:56 GMT):
@klenik Yes:grinning:

nkl199 (Wed, 22 May 2019 14:48:52 GMT):
Hi all, so one of the things I was working on within the Fabric adaptor is the ability to create a channel configuration transaction directly, through using the protobufs. I've achieved this (finally) and now we can create channels without the need for a channel.tx file ... which raises the question ... how about we just don't use them? We have some options: 1) insert the `createChannelTxEnvelope` code into the fabric-ccp adapter in-place of the `_getChannelConfigFromConfiguration` and enable the provision of a tx file, *or* the creation of a channelConfigTxn so that people can use either option. This will retain a code path fork at the point of channel creation, but introduce the need to support two different channel descriptors in the fabric.json blockchain config file. 2) Only use the channelConfigTxn route and replace the other routes Either way, the manner in which the channel is created should be invisible to a user. My opinion is that enabling a channel description in the blockchain config file is ideal, since it means they have no additional steps to perform and can control it within a single (blockchain) config file instead of messing with separate config.tx files ... should they wish to create/interact with more channels. That's just my opinion though Happy to kick off a discussion about the above here :thumbsup:

houqinghui (Thu, 23 May 2019 08:47:13 GMT):
@nkl199 Can you show us these codes since you have achieved this? Creating/interacting with more channels should be supported by Caliper, i don't know wether creating a channelConfigTxn will cost much time.

nkl199 (Thu, 23 May 2019 08:48:08 GMT):
sure - creating the channelConfigTx is super-quick, as it is "simply" knowing the right information to put in the protos ... it's just a shame that none of that is documented

nkl199 (Thu, 23 May 2019 08:49:06 GMT):
we would still need to generate and use the certificates, but that's not so much of an issue (IMO)

klenik (Thu, 23 May 2019 08:50:07 GMT):
Yep, plus channel creation is only in the init phase, so speed doesn't really matter

nkl199 (Thu, 23 May 2019 09:14:39 GMT):
ok - i'll make the change in `fabric` and `fabric-ccp` to create channels based on specification in the blockchain config file and raise a PR

klenik (Thu, 23 May 2019 09:15:13 GMT):
:thumbsup

klenik (Thu, 23 May 2019 09:15:13 GMT):
:thumbsup:

nkl199 (Thu, 23 May 2019 10:40:09 GMT):
:thinking: so ... hit the "interesting" aspect relating to all the different network configurations. Adding this change requires modification of 162 configuration files ... we might want to reduce this number. I agree that we want to show users (and easily test) large networks, but we may also want to limit the number we maintain ...

klenik (Thu, 23 May 2019 10:40:41 GMT):
Note: I think we should keep the file-based approach also, because many users use the Fabric tooling to generate config txs.

nkl199 (Thu, 23 May 2019 10:41:33 GMT):
:thumbsup:

nkl199 (Thu, 23 May 2019 10:41:36 GMT):
easy done

klenik (Thu, 23 May 2019 10:41:38 GMT):
@nkl199 I'm fine with removing the larger network samples: 2 org 1 peer versions for goleveldb (and maybe couchdb) should be enough

klenik (Thu, 23 May 2019 10:42:14 GMT):
bigger networks won't run smoothly on a local machine anyway

nkl199 (Thu, 23 May 2019 10:43:11 GMT):
ok, sounds good to me - an alternative would be both ends of the spectrum (2org1peer and 3org2peer)

klenik (Thu, 23 May 2019 10:43:13 GMT):
(plus when deprecating the fabric adapter half of the configs will disappear anyway)

nkl199 (Thu, 23 May 2019 10:46:08 GMT):
OK, so the plan is: - swap out the `fabric` adaptor to use a generated channel configTx - replace the use of cryptogen tooling with channelTx creation in the `fabric-ccp` adaptor, but retain the option of a *.tx file - reduce the number of sample networks provided - add some docs

klenik (Thu, 23 May 2019 11:35:20 GMT):
sounds like an easy plan :thumbsup: :D

klenik (Thu, 23 May 2019 12:22:30 GMT):
@nkl199 @panyu2 @houqinghui I added the meeting minutes about yesterday's call, tell me if I missed something: https://wiki.hyperledger.org/display/caliper/2019-05-22+Minutes

nkl199 (Thu, 23 May 2019 15:32:46 GMT):
... raised a PR covering the above discussion on channel creation and reduction in test networks

houqinghui (Fri, 24 May 2019 07:19:34 GMT):
@klenik Hi attila, i am confused about `Remove the tx-mode related codes from the main Caliper flow` in the meeting minutes, why do we remove the tx-mode?

houqinghui (Fri, 24 May 2019 07:20:50 GMT):
Can you tell me what Fabric-Solidity means? :grinning:

klenik (Fri, 24 May 2019 08:09:01 GMT):
@houqinghui The tx-mode is part of the older Fabric adapter, which will be deprecated. Plus [this part](https://github.com/hyperledger/caliper/blob/343321a2378e96f00dad05a2a27dd24a0f99424e/packages/caliper-core/lib/caliper-flow.js#L242) is kind of inlines a custom test during the init phase. I have some ideas on how to make this nicer (perform two round, the first does the file write, and ignore the first round from the report, the second performs the actual test from the files), but Caliper needs some additional features first. So the tx-mode will be added to the Fabric-CCP adapter eventually, I'm interested in the performance improvements, but it should be done in a less intrusive way.

klenik (Fri, 24 May 2019 08:11:37 GMT):
@houqinghui as for Fabric-Solidity: [this project](https://github.com/hyperledger/fabric-chaincode-evm) contains a chaincode, that basically embeds Burrow's EVM implementation, and makes it possible to run Solidity contracts in Fabric-style. I've integrated this CC with Caliper in my fork, so basically you can easily deploy and interact with Solidity contracts through the Fabric-CCP adapter. Once I tidy up the adapter a bit, I'll raise a PR about this support. (So Caliper will support 3 EVM compatible platforms soon :P )

klenik (Fri, 24 May 2019 14:43:30 GMT):
Although it's mainly just formality at this point, let's welcome @nkl199 as a new Caliper maintainer :champagne:

nkl199 (Fri, 24 May 2019 14:44:07 GMT):
Thanks!! Good to be on board :D

nkl199 (Tue, 28 May 2019 09:55:21 GMT):
@rjones - is it possible to ensure that Travis builds are enabled on the Caliper repository please? I've noticed that not all PRs are resulting in a build being run ... my repository is configured to run them by default and they are passing ... thanks!

rjones (Tue, 28 May 2019 19:18:52 GMT):
@nkl199 it looks like it is building? https://travis-ci.org/hyperledger/caliper

klenik (Tue, 28 May 2019 19:28:18 GMT):
The weekly Caliper call will start on 05.29. at 9AM UTC. The contact information is on the [Caliper Wiki page](https://wiki.hyperledger.org/display/caliper/Hyperledger+Caliper). Proposed agenda (feel free to extend): 1) Gathering the requirements for publishing the npm packages 2) Gathering the requirements for publishing Docker images 3) Brainstorming about how a privacy-preserving Caliper architecture would look like (components, and high-level runtime steps). Caliper has been removed from [this doc](https://docs.google.com/document/d/14paUx8CC_lWV01TMHNmeu3NGxtxb5rgtipyMr3vrR8w/edit), but the concerns are still valid. 4) Hopefully coming up with a short and focused roadmap (building on the previous roadmap) with the absolutely necessary tasks for the above points. Please leave some reaction/emoji on this message if you can attend the call.

nkl199 (Wed, 29 May 2019 08:22:19 GMT):
:thinking: strange - but yes, yes it does! Thanks for looking into it

klenik (Thu, 30 May 2019 09:39:16 GMT):
Added the meeting minutes of yesterday's call. Hope I didn't leave out anything important https://wiki.hyperledger.org/display/caliper/2019-05-29+Minutes

nkl199 (Fri, 31 May 2019 12:55:30 GMT):
For anyone keen to start experimenting with the CLI package, you can get an advanced preview within this PR: https://github.com/hyperledger/caliper/pull/468, which you can pull down locally. Once you have bootstrapped the project (new dependancies and packages... sorry, it couldn't be avoided) if you run the newly introduced integration test within `caliper-tests-integration` (/scripts/run-integration-tests.js), you will end up with all the caliper npm modules installed on your machine locally :tada: You will then have access to: - `caliper benchmark run -c -n -w ` - `caliper zooservice start` with an optional config file passed via -c - `caliper zooservice stop` with an optional config file passed via -c - `caliper zooclient start` We are edging ever closer to publishing these modules to npm ... so watch this space for the first release!!

klenik (Fri, 31 May 2019 13:10:48 GMT):
@nkl199 :champagne: I'll try to start the review on Sunday. NPM, here we come :ok_hand:

rjones (Sat, 01 Jun 2019 00:10:52 GMT):
https://gist.github.com/ryjones/5500064ce49d137e3c6be9bb90a81111

Jasonyou (Sun, 02 Jun 2019 22:48:51 GMT):
Has joined the channel.

klenik (Tue, 04 Jun 2019 10:20:37 GMT):
The weekly Caliper call will start on 06.05. at 9AM UTC. The contact information is on the [Caliper Wiki](https://wiki.hyperledger.org/display/caliper) page. Proposed agenda (feel free to extend): 1) Discussion about the UI integration (how, where, what, etc. ). @Jasonyou will also join the call. He's the intern who's working on the Caliper Visualization project during the next 3 months (and hopefully after too :) ) so it'll be a close collaboration with the Caliper contributor team :thumbsup: 2) Status updates

Jasonyou (Tue, 04 Jun 2019 15:29:52 GMT):
Hello everyone, this is Jason You ;) Really excited to join the Caliper community. I will focus on the Caliper Visualization project and is willing to learn more about the expectation for this project from all of you!

houqinghui (Wed, 05 Jun 2019 14:03:30 GMT):
Hi nick, I pulled down your repo, and i run the newly introduced integration test within `caliper-tests-integration`, but i donot know the command that can run Caliper.

klenik (Wed, 05 Jun 2019 14:05:12 GMT):
@houqinghui ``` caliper benchmark run -c benchmark/composer/config.yaml -n network/fabric-v1.3/2org1peercouchdb/composer.json -w ../caliper-samples/ ``` I got this from the test script, couldn't find it in a doc/readme though. I guess @nkl199 is waiting for my review to submit a doc PR

klenik (Wed, 05 Jun 2019 14:06:21 GMT):
@houqinghui Don't forget to run `npm run setup_verdaccio` first in the integration folder, so the packages are published locally

houqinghui (Thu, 06 Jun 2019 01:44:12 GMT):
@klenik I run the command `caliper benchmark run -c benchmark/composer/config.yaml -n network/fabric-v1.3/2org1peercouchdb/composer.json -w ../caliper-samples/` in the caliper-samples folder, but couldn't run Caliper, the error is `caliper command not found`. Are you sure the command is right?

klenik (Thu, 06 Jun 2019 06:24:47 GMT):
@houqinghui The old run-benchmark approach still works, but if you want to test the CLI, the following steps are needed (most of it is local npm publishing) ``` # starting from the root folder npm i && npm run repoclean -- --yes && npm run bootstrap # then to the test "package" cd packages/caliper-tests-integration/ # Verdaccio server requires a dummy user if publishing via npm echo '//localhost:4873/:_authToken="foo"' > ${HOME}/.npmrc echo fetch-retries=10 >> ${HOME}/.npmrc export npm_config_registry=http://localhost:4873 # Start npm server and publish latest packages to it npm run setup_verdaccio # and now the caliper command is available caliper benchmark run -c benchmark/simple/config.yaml -n network/fabric-v1.4/2org1peergoleveldb/fabric-ccp-go.yaml -w ../caliper-samples/ ```

nkl199 (Thu, 06 Jun 2019 06:50:58 GMT):
running the integration test(s) will also result in the above being run - i've updated readme's in the main repo, but not modified the docs website .... i can start work on that now

klenik (Thu, 06 Jun 2019 06:54:51 GMT):
True, but the integration tests also tear down verdaccio, these extracted steps leave the caliper command available to you if you want some more tests. It woks nice btw, I'm already collecting my review offline, since the github review page is broken by the sheer number of changes :joy: :champagne:

nkl199 (Thu, 06 Jun 2019 07:07:50 GMT):
haha, cheers - note that the tests will install caliper globally into your system, so that even once verdaccio has been torn down, it will still be there

nkl199 (Thu, 06 Jun 2019 07:08:33 GMT):
... might need something to clean that off?

klenik (Thu, 06 Jun 2019 07:08:41 GMT):
ahh, didn't know that, then the test script is the way to go :)

klenik (Thu, 06 Jun 2019 07:09:09 GMT):
once you have caliper, why would you delete it? :thinking: :joy:

nkl199 (Thu, 06 Jun 2019 07:09:23 GMT):
good point!!

rjones (Fri, 07 Jun 2019 15:44:27 GMT):
If you use GitHub please do this: https://help.github.com/en/articles/configuring-two-factor-authentication

russanto (Sun, 09 Jun 2019 15:14:45 GMT):
In the Implement Ethereum adapter https://github.com/hyperledger/caliper/pull/432 pull request I updated the code to run with the caliper cli. I also provided some doc in a second PR https://github.com/hyperledger/caliper/pull/476

klenik (Tue, 11 Jun 2019 22:28:38 GMT):
The weekly Caliper call will start on 06.12. at 9AM UTC. The contact information is on the [Caliper Wiki](https://wiki.hyperledger.org/display/caliper) page. Proposed agenda (feel free to extend): 1) Discussion about the extent of SUT monitoring that Caliper should be responsible for (motto: don't duplicate existing tools, maybe provide integration with them) 2) Monitor(s) => Caliper-core => Reporter(s) design discussion (we'll have a new reporter, the remote(?) UI) 3) Updates and additional ideas

nkl199 (Wed, 12 Jun 2019 08:11:57 GMT):
4) final additions prior to npm release

russanto (Wed, 12 Jun 2019 08:28:12 GMT):
I won't be able to participate, in the case in which there are doubts on my PRs just text

klenik (Wed, 12 Jun 2019 08:30:24 GMT):
@russanto Sure, no problem. We'll leave comments on the PR directly, so it's more tracable

klenik (Wed, 12 Jun 2019 11:09:31 GMT):
@nkl199 @houqinghui Added today's meeting minutes: https://wiki.hyperledger.org/display/caliper/2019-06-12+Minutes Feel free to extend it.

klenik (Wed, 12 Jun 2019 11:09:50 GMT):
@nkl199 Could you open the Github issues for the pending tasks we discussed?

nkl199 (Wed, 12 Jun 2019 11:17:33 GMT):
sure

klenik (Thu, 13 Jun 2019 09:50:14 GMT):
@nkl199 I'm wondering about the following Caliper NPM usage/use case: 1) Right now, the Caliper CLI package includes the highest versions of the other packages (with Fabric SDK 1.4.0 for example). 2) Once the packages are published, I imagined the following workflow for Caliper projects: ``` cd ~/myCaliperProject/ npm install --save caliper-fabric@1.3.0 # suppose the artifacts are already in this folder caliper benchmark run -w ... -c ... -n ... ```

klenik (Thu, 13 Jun 2019 09:51:28 GMT):
So the users install the specific adapter package, and that will "include" the CLI. Is this feasible with the current dependency structure? Or how did you envision using a published package?

nkl199 (Thu, 13 Jun 2019 09:52:24 GMT):
we would need a new CLI published when there are new "bumps" in the adaptors that cause breaking changes

nkl199 (Thu, 13 Jun 2019 09:53:39 GMT):
for instance, because we have 2 different dependancies in the fabric adaptor, we will need two fabric adaptors (fabric 1.1, and fabric 1.2+) ... we would need 2 CLI modules published to point to these

nkl199 (Thu, 13 Jun 2019 09:54:42 GMT):
this will simplify somewhat when we dockerize the clients, as we can keep the same caliper-core and CLI unless there is an interface change

klenik (Thu, 13 Jun 2019 09:56:37 GMT):
yeah, but if you use the CLI now, it will come with fixed adapter versions and their dependencies (fixed SDK versions). How can I use the 1.3.0 Fabric SDK with the Fabric adapter through the CLI (which is published)? How would a Caliper project init/bootstrap look like?

klenik (Thu, 13 Jun 2019 09:58:27 GMT):
https://github.com/hyperledger/caliper/blob/master/packages/caliper-cli/lib/benchmark/lib/runBenchmark.js#L60 Based on this line, we don't even need to specify the deps statically, it's resolved during runtime

klenik (Thu, 13 Jun 2019 09:59:00 GMT):
Does that mean we can completely decouple the CLI from the adapter packages (except caliper-core)?

nkl199 (Thu, 13 Jun 2019 09:59:19 GMT):
pretty messy to be honest: - fabric adaptor published for each SDK version - CLI adaptor published multiple times with a dependancy on each of the published fabric-adaptors - the CLI tracks with the most recent code base The bootstrap would not change The line mentioned above will pull in the current adaptoir dependancy linked in the released item

nkl199 (Thu, 13 Jun 2019 10:01:27 GMT):
The problem with the above, is that backporting fixes to previous versions of the adaptor is *painful* - the normal process would be to have a separate branch for each supported level ... which is normally done in multiple repositories to prevent branch explosion

klenik (Thu, 13 Jun 2019 10:14:45 GMT):
Fortunately, for now we cover each supported level with one code-base, only the installed SDK versions differ. And if the platforms take semantic versioning seriously (so far they do), then maybe we'll be able to keep one adapter code-base per platform (if we managed it with Fabric, the most complex platform so far, then we can probably do that with the other platforms also, but only for major versions of course, >=1.0.0)

klenik (Thu, 13 Jun 2019 10:19:18 GMT):
Since the CLI code is platform-independent, could the following install scenario work (by completely decoupling the CLI from the adapter packages)? ``` cd ~/myCaliperProject/ npm install --save caliper-cli@1.0.0 caliper-fabric@1.3.0 # suppose the artifacts are already in this folder caliper benchmark run -w ... -c ... -n ... ``` And when the above linked line requires caliper-fabric, it can [load it](https://nodejs.org/api/modules.html#modules_loading_from_node_modules_folders) from the `myCaliperProject/node_modules` folder. Or am I missing something?

nkl199 (Thu, 13 Jun 2019 10:22:49 GMT):
the problem is inside the adaptor package itself https://github.com/hyperledger/caliper/blob/master/packages/caliper-fabric-ccp/package.json#L25

nkl199 (Thu, 13 Jun 2019 10:23:18 GMT):
we need to target the SDK there, and then the CLI references the adaptor that uses a specific dependancy

nkl199 (Thu, 13 Jun 2019 10:24:05 GMT):
I'm wondering if the above will work tho

nkl199 (Thu, 13 Jun 2019 10:24:22 GMT):
.. we could certainly test it with multiple pblishes to verdaccio

nkl199 (Thu, 13 Jun 2019 10:24:22 GMT):
.. we could certainly test it with multiple publishes to verdaccio

klenik (Thu, 13 Jun 2019 10:32:04 GMT):
I think it should be possible to publish the same package with different deps, right? Something like this (very high-level pseudo code): for every adapter: for deps in [dep_set1, dep_set2, ..., dep_setN] : publish caliper-@ with deps publish caliper-cli@sem_ver So the adapter package could be installed either with or without the CLI, whatever the use case is. Since the CLI is "just" a smart wrapper around `caliper-flow/run`

klenik (Thu, 13 Jun 2019 10:40:09 GMT):
At least I hope it's possible...

klenik (Tue, 18 Jun 2019 13:43:56 GMT):
The weekly Caliper call will start on 06.19. at 9AM UTC. The contact information is on the [Caliper Wiki](https://wiki.hyperledger.org/display/caliper) page. Proposed agenda (feel free to extend): 1) Continue NPM-related discussion 2) Steps towards formalizing the dataflow inside Caliper (i.e., between metric sources and exporter targets). See [this issue](https://github.com/hyperledger/caliper/issues/483)

nkl199 (Tue, 25 Jun 2019 18:52:57 GMT):
3) Inbound changes with proposed HFRD additions 4) Increased alignment with Perf/Scale workgroup

nkl199 (Fri, 28 Jun 2019 16:45:58 GMT):
Hi, just to let everyone know that I'm going on vacation from this weekend (first in over 2 years!!) so won't be attending the next two Caliper calls :wave:

klenik (Tue, 02 Jul 2019 05:56:00 GMT):
I'll also be away tomorrow, so unless someone has something to discuss and can orchestrate the call, _tomorrow's call is canceled_

rjones (Mon, 08 Jul 2019 18:08:41 GMT):
Has left the channel.

klenik (Wed, 10 Jul 2019 07:42:07 GMT):
Due to lack of important agenda items, I think we can cancel today's call. Any Q&A and updates can be done in this channel

klenik (Thu, 11 Jul 2019 15:11:45 GMT):
I think I have a nice (recurring) agenda item for next week's call: https://github.com/hyperledger/caliper/issues/488 After some trial and error, the npm publishing workflow of Caliper is getting clearer. Feel free to leave comments on the issue, I'll also drop the link in the #caliper channel

nkl199 (Mon, 15 Jul 2019 16:51:05 GMT):
-> proposal to modify the time of the current meeting to fall at 9am EST (I make that 2pm UTC)

klenik (Mon, 15 Jul 2019 17:08:20 GMT):
[ ](https://chat.hyperledger.org/channel/caliper-contributors?msg=PiumHk5NTg4neWE57) Fine for me :thumbsup: I guess it's to include potential east coast contributors? :P

klenik (Tue, 16 Jul 2019 07:09:00 GMT):
@nkl199 As I see, that timeslot is taken in the HL Community Zoom channel

nkl199 (Tue, 16 Jul 2019 09:10:15 GMT):
3pm UTC looks to be available ... That would be OK for me, any objections on this channel? (shame we can't do an at-here :( )

nkl199 (Tue, 16 Jul 2019 09:12:38 GMT):
I think discussion on the move to dockerized clients would be handy - we will want to create some more repositories for each client on a per-language basis ... and we should consider a separate samples repository too perhaps.

klenik (Tue, 16 Jul 2019 09:15:33 GMT):
Yes, now the npm packages are around the corner, separating the samples package might be a good idea, so it can be cloned without additional bootstrap, then a simple npm install will add the CLI and needed adapter

klenik (Tue, 16 Jul 2019 09:18:49 GMT):
[ ](https://chat.hyperledger.org/channel/caliper-contributors?msg=GtXvG9G9DcTkMD2Mc) Yep, that slot is free and also fine for me :thumbsup:

klenik (Tue, 16 Jul 2019 11:17:14 GMT):
The weekly Caliper call will start on 07.17. at **3PM UTC**. Zoom channel: https://zoom.us/my/hyperledger.community Proposed agenda (feel free to extend): 1. Publishing Caliper to npm. See this discussion: https://github.com/hyperledger/caliper/issues/488 2. Probably a quick demo from me about a new package-based usage of Caliper (see my message in the issue above) 3. Deprecating the old fabric adapter (rename or delete?) and closing the related issues 4. Improving the docs before publishing. See https://github.com/hyperledger/caliper/issues/507 5. Dismissing the "file generation mode" feature (https://github.com/hyperledger/caliper/pull/410) since the following PR significantly boosted the sending rate of Caliper: https://github.com/hyperledger/caliper/pull/509

kalyan.aruru (Mon, 22 Jul 2019 10:39:48 GMT):
Has joined the channel.

kalyan.aruru (Mon, 22 Jul 2019 10:44:36 GMT):
Hi Does anyone have documentation/tutorial for using caliper with fabric

nkl199 (Mon, 22 Jul 2019 11:18:30 GMT):
beyond this: https://hyperledger.github.io/caliper/docs/1_Getting_Started.html ?

nkl199 (Tue, 23 Jul 2019 08:26:00 GMT):
hi all, worth noting the deprecations within Fabric 1.4.2 https://github.com/hyperledger/fabric/releases/tag/v1.4.2

nkl199 (Wed, 24 Jul 2019 07:31:31 GMT):
Hi, I've enabled the wiki on the caliper git repo- it will hopefully enable us to make some quick notifications etc and help us store "related" material for consumption

nkl199 (Wed, 24 Jul 2019 07:33:40 GMT):
Please note that the recurring weekly Caliper Contributor call has been moved to 3pm UTC on a Wednesday, in order to help US contributors attend

nkl199 (Wed, 24 Jul 2019 07:34:50 GMT):
https://zoom.us/my/hyperledger.community

Jasonyou (Tue, 30 Jul 2019 21:28:54 GMT):
Just in case somebody is still looking for solution of ERR from gyc when running `npm run install_cli`: Klenik and I tested the local installation of caliper-cli on Ubuntu 18.04, and here is the solution.

Jasonyou (Tue, 30 Jul 2019 21:29:35 GMT):

Permission-Issue-Install-CLI.docx

klenik (Wed, 31 Jul 2019 14:33:51 GMT):
@nkl199 Something came up, I can't join the call. Could you please coordinate it?

nkl199 (Wed, 31 Jul 2019 14:34:40 GMT):
sure :thumbsup:

nkl199 (Wed, 31 Jul 2019 15:24:19 GMT):
... a rather lonely call :(

nkl199 (Fri, 02 Aug 2019 09:17:13 GMT):
Topics for next maintainers call, please feel free to extend via comments! : - Status of npm publish - Argo - Current status of Prometheus integration

nkl199 (Fri, 02 Aug 2019 09:17:13 GMT):
Topics for next maintainers call, please feel free to extend via comments! : - Status of npm publish - Argo - Current status of Prometheus integration - Potential move to CircleCI for builds

russanto (Sat, 03 Aug 2019 08:55:32 GMT):
Hi guys, how can I set to jump license check, for example, on binary files ?

russanto (Sat, 03 Aug 2019 08:55:57 GMT):
I'm merging latest changes on master in the ethereum pull request

klenik (Sat, 03 Aug 2019 08:58:19 GMT):
@russanto you can add exclusion rules to the package json, like this: https://github.com/hyperledger/caliper/blob/1cb88d52193e4eaa8dad5e566dd899a69d4495cb/packages/caliper-fabric/package.json#L54

russanto (Sun, 04 Aug 2019 08:49:17 GMT):
any ideas on why travis is failing to run tests ? It seems it doesn't find some dependencies. https://travis-ci.org/hyperledger/caliper/jobs/567253002

klenik (Sun, 04 Aug 2019 08:55:34 GMT):
@russanto hmm, this seems like a transient error, always complaining about a different package. Can you do a force push later to trigger the CI again?

russanto (Sun, 04 Aug 2019 16:24:00 GMT):
different error again

nkl199 (Tue, 06 Aug 2019 07:59:53 GMT):
@russanto I'm looking into that, it fails for a different reason each time during the npm install stage from verdaccio. I've been unable to recreate locally, which is leading me to believe that it is a transient error due to a resource limitation in the Travis build

klenik (Wed, 07 Aug 2019 16:01:32 GMT):
Nice call everyone :ok_hand: I think it's the first time we ran out of time, and the topics were awesome :thumbsup:

Jasonyou (Wed, 07 Aug 2019 16:02:05 GMT):
Yeah really great presentation and explain!

nkl199 (Wed, 07 Aug 2019 16:02:25 GMT):
good progress being made I think :D

klenik (Wed, 07 Aug 2019 16:06:47 GMT):
Yeah, the puzzle pieces are starting to form a nice picture :D

Jasonyou (Thu, 08 Aug 2019 23:01:09 GMT):

test-error.png

Jasonyou (Thu, 08 Aug 2019 23:01:10 GMT):
Hello I was trying to run a test in the `./caliper/packages/caliper-samples` with the command `npx caliper benchmark run -w . -c benchmark/simple/config-linear-rate.yaml -n network/fabric-v1.4.1/raft/fabric-ccp-node-tls.yaml`. But it throw some error said `Error: Couldn't create Channel 'mychannel': Failed to connect before the deadline URL:grpcs://localhost:7050`

Jasonyou (Thu, 08 Aug 2019 23:41:43 GMT):
Some questions on GUI & CLI integration: I'm currently trying to integrate the Caliper GUI with the Caliper-cli, and I read through the source code these two days. Here are some thoughts and insights, please let me know if you know a better way to do this or can give me any advices. Thank! Here is my current strategy to build the Caliper real-time visualization GUI: >> [GUI front-end: query the API in every X-seconds] -> [API that query data from DB] -> [DB] <- [Caliper-CLI: helper scripts/functions update real-time test results to DB] [Data Collecting Strategies] 1. I will write a data cleaning and collecting script with JS in the `./caliper/packages/caliper-core/lib/caliper-gui` directory. This script will include functions that receives data from one or more scripts in the `caliper-core`, and then push the cleaned JSON data to the DB (MongoDB). 2. Then I can either build an API to query the data from the DB, or directly query the visualization data from the DB to the GUI front-end visualization. The real-time functionality will be accomplished with Socket.io. 3. I will build the GUI, DB, and API in the `caliper-core/caliper-gui` directory, so they are more manageable when we publish separate npm packages, i.e. `caliper-cli` and `caliper-gui`. [Current issue] I read through the `./caliper-core/lib/` source code, and realized that there are multiple scripts can provide useful test report data. But I still didn’t find a good place to get clear real-time time series data, so I might need some advice about that part. I found some scripts I can modify to collect data and then send to the DB, where the updated data will then be queried by the front-end GUI: - `./caliper/packages/caliper-core/lib/caliper-flow.js`: - The `const report = new Report(monitor)` can provide some system test result. Maybe we can have one more helper function in the `report.js` script to return JSON results, instead of just print it out or save it in a local file. - ‘./caliper/packages/caliper-core/lib/client/caliper-local-client.js` can provide detailed `TxStats` object or `resultStats` in the `async doTest(msg)` function. I can simple add a helper function from my data collection script to update these data to the DB. - But the question here is that I still don’t know the exact JSON data structure of the `resultStats` object, since I encountered some issue in installation. So now I will need to developer of that part to help me decide how to format the JSON output. - Another question is that I’m not sure how long is the resultStats, and whether this is a ordered test statistics array. Cause in the line #275-#277, the code only return the first index result. I will need someone to help me to answer this question too. - `./caliper/packages/caliper-core/lib/report/report.js` has some very helpful functions for data collection, but they are all just static data, which means they are not real-time time series data. But I can add some function directly in this script to collect time-series JSON data object, which can be sent to the DB. Here are just some of I thoughts on the integration of GUI and Caliper-cli/Caliper-core. I need to get some help and advices from people who know the data flow under Caliper-core/Caliper-cli. Thanks very much for the help!

Jasonyou (Thu, 08 Aug 2019 23:56:41 GMT):
PS: I just want to make sure that all types of data-flow (JSON object) we have in the `caliper-core` scripts, so I don't need to re-implement tools by myself ;)

Jasonyou (Thu, 08 Aug 2019 23:56:41 GMT):
PS: I just want to make sure that I found all types of data-flow (JSON object) we have in the `caliper-core` scripts, so I don't need to re-implement tools by myself ;)

FlyingTiger (Fri, 09 Aug 2019 01:50:32 GMT):
@Jasonyou Failed to connect, could you please try to use no tls network file instead, for example, using https://github.com/hyperledger/caliper/blob/master/packages/caliper-samples/network/fabric-v1.4.1/2org1peergoleveldb/fabric-go.yaml?

FlyingTiger (Fri, 09 Aug 2019 01:52:01 GMT):
@Jasonyou If has the problem again, please check the docker containers are runnning by using docker ps.

FlyingTiger (Fri, 09 Aug 2019 02:01:53 GMT):
@Jasonyou About the datasource, you could reference the https://github.com/hyperledger/caliper/tree/master/packages/caliper-core/lib/gui document, and https://github.com/hyperledger/caliper/blob/master/packages/caliper-core/lib/gui/src/demo.js#L178 , https://github.com/hyperledger/caliper/blob/master/packages/caliper-core/lib/test/defaultTest.js#L99 , https://github.com/hyperledger/caliper/blob/master/packages/caliper-core/lib/client/client-orchestrator.js#L189 , the updates are array to save txUpdate results

Jasonyou (Fri, 09 Aug 2019 02:04:35 GMT):
Perfect I will try all of these and see if they works!

Jasonyou (Fri, 09 Aug 2019 02:04:42 GMT):
Thanks a lot @FlyingTiger !

nkl199 (Fri, 09 Aug 2019 08:13:23 GMT):
Hi Jason, you want to place your code in `/caliper/pacakges/caliper-gui` then import any other required components (caliper-core for instance) as a dependancy

nkl199 (Fri, 09 Aug 2019 08:13:23 GMT):
Hi @Jasonyou , you want to place your code in `/caliper/pacakges/caliper-gui` then import any other required components (caliper-core for instance) as a dependancy

nkl199 (Fri, 09 Aug 2019 08:14:36 GMT):
your GUI will end up replicating/replacing the caliper-flow.js file, but in it's own repository, where you can have more control over things - and inject your own test observer etc

nkl199 (Fri, 09 Aug 2019 08:26:03 GMT):
I'd recommend that you have a look in my current prometheus-monitor branch under the `core/lib/test-observers` that are set to replace the `gui/src/demo.js` implementation that @FlyingTiger mentioned above - we are making it possible to use multiple test observers so that we can momintor local/remote running tasks

Jasonyou (Sat, 10 Aug 2019 19:29:06 GMT):
Thanks a lot @nkl199! I forked your caliper branch repo and is building the GUI in the `caliper/packages/caliper-gui` directory. I tried again to install the caliper-cli in the `caliper-test-integration` and succeed with the node v8.16.0 and npm permission issue fixed: "https://docs.npmjs.com/resolving-eacces-permissions-errors-when-installing-packages-globally". But I still cannot run the sample benchmark test even after I tried all the .yaml configuration files in the `caliper-sample` directory @FlyingTiger ... I mainly got two types of errors: ``` info: [adapters/fabric]: Sleeping 5s... 2019-08-10T18:25:55.608Z - error: [Remote.js]: Error: Failed to connect before the deadline URL:grpc://localhost:7051 ``` AND ``` error: [caliper-flow]: Error: Error: Invalid endorsement for marbles@v0 in mychannel from peer0.org1.example.com: 2 UNKNOWN: Stream removed at proposalResponses.forEach (/home/jasonyou/.npm-global/lib/node_modules/caliper-cli/node_modules/caliper-fabric/lib/fabric.js:1316:31) at Array.forEach () at Fabric._instantiateChaincodes (/home/jasonyou/.npm-global/lib/node_modules/caliper-cli/node_modules/caliper-fabric/lib/fabric.js:1312:35) at ```

Jasonyou (Sat, 10 Aug 2019 19:29:06 GMT):
Thanks a lot @nkl199! I forked your caliper branch repo and is building the GUI in the `caliper/packages/caliper-gui` directory. I tried again with your `caliper-1` branch to install the caliper-cli in the `caliper-test-integration` and succeed with the node v8.16.0 and npm permission issue fixed: "https://docs.npmjs.com/resolving-eacces-permissions-errors-when-installing-packages-globally". But I still cannot run the sample benchmark test even after I tried all the .yaml configuration files in the `caliper-sample` directory @FlyingTiger ... I mainly got two types of errors: ``` info: [adapters/fabric]: Sleeping 5s... 2019-08-10T18:25:55.608Z - error: [Remote.js]: Error: Failed to connect before the deadline URL:grpc://localhost:7051 ``` AND ``` error: [caliper-flow]: Error: Error: Invalid endorsement for marbles@v0 in mychannel from peer0.org1.example.com: 2 UNKNOWN: Stream removed at proposalResponses.forEach (/home/jasonyou/.npm-global/lib/node_modules/caliper-cli/node_modules/caliper-fabric/lib/fabric.js:1316:31) at Array.forEach () at Fabric._instantiateChaincodes (/home/jasonyou/.npm-global/lib/node_modules/caliper-cli/node_modules/caliper-fabric/lib/fabric.js:1312:35) at ```

Jasonyou (Sat, 10 Aug 2019 19:29:06 GMT):
Thanks a lot @nkl199! I forked your caliper branch repo and is building the GUI in the `caliper/packages/caliper-gui` directory. I tried again with your `caliper-1` branch to install the caliper-cli in the `caliper-test-integration` and succeed with the node v8.16.0 and npm permission issue fixed: "https://docs.npmjs.com/resolving-eacces-permissions-errors-when-installing-packages-globally". But I still cannot run the sample benchmark test even after I tried all the .yaml configuration files in the `caliper-sample` directory as suggested by @FlyingTiger ... I mainly got two types of errors: ``` info: [adapters/fabric]: Sleeping 5s... 2019-08-10T18:25:55.608Z - error: [Remote.js]: Error: Failed to connect before the deadline URL:grpc://localhost:7051 ``` AND ``` error: [caliper-flow]: Error: Error: Invalid endorsement for marbles@v0 in mychannel from peer0.org1.example.com: 2 UNKNOWN: Stream removed at proposalResponses.forEach (/home/jasonyou/.npm-global/lib/node_modules/caliper-cli/node_modules/caliper-fabric/lib/fabric.js:1316:31) at Array.forEach () at Fabric._instantiateChaincodes (/home/jasonyou/.npm-global/lib/node_modules/caliper-cli/node_modules/caliper-fabric/lib/fabric.js:1312:35) at ```

Jasonyou (Sat, 10 Aug 2019 19:30:00 GMT):
The first error looks like the Fabric docker server cannot be connected from Caliper sample benchmark test. So I want to see how to DEBUG this part (which script to check)

Jasonyou (Sat, 10 Aug 2019 19:30:32 GMT):
The second error is more like a adaptor error somewhere in the `fabric.js` script

Jasonyou (Sat, 10 Aug 2019 19:30:32 GMT):
The second error is more like an adaptor error somewhere in the `fabric.js` script

Jasonyou (Sat, 10 Aug 2019 19:30:32 GMT):
The second error is more like an adaptor error somewhere in the `fabric.js` or the `caliper-flow` script

Jasonyou (Sat, 10 Aug 2019 19:30:32 GMT):
The second error is more like an adaptor error somewhere in the `fabric.js` or the `caliper-flow.js` script

Jasonyou (Sat, 10 Aug 2019 19:34:16 GMT):
Also @nkl199, for importing dependencies in caliper-core, do you mean import those needed scripts one by one, or the whole caliper-core package? (I only found you imported 'Logger' from `caliper-utils` in the scripts in the `prometheus` directory. I'm not very clear about in what order should this GUI be integrated, i.e. is the GUI turned on by the user manually, or it will automatically triggered by some code in the caliper-core/caliper-cli and pop out?

Jasonyou (Sat, 10 Aug 2019 19:34:16 GMT):
Also @nkl199, for importing dependencies in caliper-core, do you mean importing those needed scripts one by one, or the whole caliper-core package? (I only found you imported 'Logger' from `caliper-utils` in the scripts in the `prometheus` directory. I'm not very clear about in what order should this GUI be integrated, i.e. is the GUI turned on by the user manually, or it will automatically triggered by some code in the caliper-core/caliper-cli and pop out?

Jasonyou (Sat, 10 Aug 2019 19:39:01 GMT):
For the 'failed to connect' error, I also googled online and tried so methods, such as reinstall globally the latest versions of "fabric-ca-client, fabric-client, and grpc packages" with npm

Jasonyou (Sat, 10 Aug 2019 19:39:01 GMT):
For the 'failed to connect' error, I also googled online and tried some methods, such as reinstall globally the latest versions of "fabric-ca-client, fabric-client, and grpc packages" with npm

Jasonyou (Sat, 10 Aug 2019 19:39:01 GMT):
For the 'failed to connect' error, I also googled online and tried some methods, such as reinstall globally the latest versions of `fabric-ca-client`, `fabric-client`, and `grpc packages` with npm

nkl199 (Mon, 12 Aug 2019 08:33:49 GMT):
@Jasonyou The GUI should be a single "parent" item that the user starts up and is then able to run Caliper benchmarks. So the minimal user flow would be: - start up GUI - import benchmark file (tests, benchmark description, network topology) - run the benchmark I'd recommend that you look at Composer Playground to see what I mean there https://github.com/hyperledger/composer/tree/master/packages is the code repo and https://composer-playground.mybluemix.net/ is a docker image of the Playground hosted in IBM cloud You can import what you need from caliper core using `{X, Y, Z} = require('caliper-core');` and you will end up creating your own code that replaces `caliper-flow' in the core package, as you need to feed it items provided via the UI and not command line argument

Jasonyou (Mon, 12 Aug 2019 14:44:29 GMT):
Perfect! That's exactly my concern and confuse! Thanks a lot!

klenik (Mon, 12 Aug 2019 22:16:25 GMT):
I'll be on vacation till next Tuesday, so I won't make it to the call this week.

soumyanayak (Tue, 13 Aug 2019 06:56:36 GMT):
Has joined the channel.

Jasonyou (Tue, 13 Aug 2019 22:07:19 GMT):

peer0-connection-issue.png

Jasonyou (Tue, 13 Aug 2019 22:07:20 GMT):
Hi guys I read through the source code today, and found something wired happens to the docker server for `peer0.org1.example.com`. I'm not sure if this is just my misunderstood of how docker works or is an issue. When I run the benchmark test with caliper, the `Error: Failed to connect before the deadline URL:grpc://localhost:7051` always occur when the Fabric adaptor tries to query the Blockchain channel in the line #1434 of code `./caliper/packages/caliper-fabric/lib/fabric.js` Fabric class, _joinChannels function. So I keep checking the running docker process/container in another Terminal console, and realized that the `peer0.org1.example.com` only ran for 1 second, and then disappeared... Can someone help me to check if this is the reason of test error? Thanks a lot!

Jasonyou (Tue, 13 Aug 2019 22:07:20 GMT):
Hi guys I read through the source code today, and found something wired "bug" happens to the docker server for `peer0.org1.example.com`. I'm not sure if this is just my misunderstood of how docker works or is an issue. When I run the benchmark test with caliper, the `Error: Failed to connect before the deadline URL:grpc://localhost:7051` always occur when the Fabric adaptor tries to query the Blockchain channel in the line #1434 of code `./caliper/packages/caliper-fabric/lib/fabric.js` Fabric class, _joinChannels function. So I keep checking the running docker process/container in another Terminal console, and realized that the `peer0.org1.example.com` only ran for 1 second, and then disappeared... Can someone help me to check if this is the reason of test error? Thanks a lot!

FlyingTiger (Wed, 14 Aug 2019 01:15:12 GMT):
@Jasonyou Using docker ps --all to list the stopped containers, and using docker logs to fetch the logs of the stopped container, then analyse the logs and try to find the reason why it stopped.

Jasonyou (Wed, 14 Aug 2019 01:17:40 GMT):
Okay! I will try this analysis method. Thanks!

FlyingTiger (Wed, 14 Aug 2019 01:32:34 GMT):
You

FlyingTiger (Wed, 14 Aug 2019 01:35:45 GMT):
Of course, You could post the logs on this channel, so someone could help you to check the reason why the container stopped.

FlyingTiger (Wed, 14 Aug 2019 01:37:26 GMT):
And yes, the peer0.org1.example.com only ran for 1 second, and then disappeared is the reason of the test error.

Jasonyou (Wed, 14 Aug 2019 02:14:39 GMT):
I have a question for the config file generation: do you expect the GUI to generate/update/export/import both blockchain network configuration files and test benchmark configuration files, or we just need the test benchmark files for Caliper core?

Jasonyou (Wed, 14 Aug 2019 02:39:14 GMT):

Jasonyou - Tue Aug 13 2019 22:39:00 GMT-0400 (Eastern Daylight Time).txt

Jasonyou (Wed, 14 Aug 2019 02:40:33 GMT):
I actually got a different error when I ran the ``caliper benchmark run --caliper-workspace . --caliper-benchconfig benchmark/simple/config-linear-rate.yaml --caliper-networkconfig network/fabric-v1.4.1/2org1peergoleveldb/fabric-node.yaml ` with config files in the *2org1peergoleveldb* directory

Jasonyou (Wed, 14 Aug 2019 02:41:25 GMT):

Invalid-endorsor.png

Jasonyou (Wed, 14 Aug 2019 02:53:00 GMT):
The complete log of error by running the docker-compose in normal mode without `-d` flag (non detach mode):

Jasonyou (Wed, 14 Aug 2019 02:53:35 GMT):

Jasonyou - Tue Aug 13 2019 22:53:12 GMT-0400 (Eastern Daylight Time).txt

nkl199 (Wed, 14 Aug 2019 09:41:51 GMT):
Agenda For Call: - Prometheus status - New repository for caliper-benchmarks - Hopefully will be joined by Julien to present the Corda adapter :ok_hand: - UI progress - AOB

klenik (Wed, 14 Aug 2019 10:44:28 GMT):
It probably had some startup issues, check the container logs (if the container still exists), or maybe it's a resource issue for your machine when running a lot of containers

nkl199 (Thu, 15 Aug 2019 11:09:27 GMT):
Iroha is now at V1.1 ...I guess that means an "official" package for the adapter?

nkl199 (Thu, 15 Aug 2019 12:08:04 GMT):
and it seems that Azure pipelines is the favoured CI engine to replace Travis ... at the moment ... which is nice as you have a build job limitation and not a build minutes limit

klenik (Thu, 15 Aug 2019 22:01:01 GMT):
[ ](https://chat.hyperledger.org/channel/caliper-contributors?msg=LXWdHcMC2PdSeZyqG) Eh, here we go again :D

Jasonyou (Fri, 16 Aug 2019 19:13:05 GMT):
Hi @nkl199 I got some questions when I read the `/caliper/packages/caliper-core/lib/client/caliper-local-client.js`: I found that in the `beforeTest(msg)` function we have a line to configure the Prometheus observer: `this.prometheusClient.configureTarget(msg.label, msg.testRound, msg.clientIdx);`. My question is: does the prometheus observer record the test result for each individual client test, or does it record the whole lifecycle in one place?

Jasonyou (Fri, 16 Aug 2019 19:20:56 GMT):
I found that in the `txUpdate()` function the Prometheus Client constantly receives data by push to `pushUrl`. So I guess the result of Prometheus is also discrete, right?

nkl199 (Mon, 19 Aug 2019 08:27:04 GMT):
@Jasonyou - yes, each client sends results to the Prometheus server, we can then run a query that is the aggregation of all clients for a specific test round

Jasonyou (Mon, 19 Aug 2019 10:12:40 GMT):
Cool! I get it, thanks!

Jasonyou (Mon, 19 Aug 2019 10:20:06 GMT):
Now I can understand the source codes in caliper-core, and is able to import dependencies from it. I'm trying for following the run = async function(absConfigFile, absNetworkFile, admin, clientFactory, workspace) in the caliper-flow.js to produce the test results now. I have a few questions: 1. I can see that the absConfigFile, absNetworkFile, and workspace are just same configure files path as we passed to the caliper client in the terminal. But I don't know how to give the admin and clientFactory arguments to the async run function. 2. After I integrate the run function, do I need to add something more to make it works as expected? (I was wondering if there are other dependency files that I need to add to my server to make the whole testing works)

Jasonyou (Mon, 19 Aug 2019 10:20:06 GMT):
Now I can understand the source codes in caliper-core, and is able to import dependencies from it. I'm trying for following the `run = async function(absConfigFile, absNetworkFile, admin, clientFactory, workspace)` in the `caliper-flow.js` to produce the test results now. I have a few questions: 1. I can see that the `absConfigFile`, `absNetworkFile`, and `workspace` are just same configure files path as we passed to the `caliper` client in the terminal. But I don't know how to give the `admin` and `clientFactory` arguments to the `async run` function. 2. After I integrate the `run` function, do I need to add something more to make it works as expected? (I was wondering if there are other dependency files that I need to add to my server to make the whole testing works)

nkl199 (Mon, 19 Aug 2019 10:37:24 GMT):
if u look in caliper-cli/lib/benchmak/lib/runBenchmark.js you will see how these items are created based on the adaptor imports

Jasonyou (Mon, 19 Aug 2019 11:12:54 GMT):
Awesome! Just read through the source code and imported the `caliper-core` and `caliper-fabric` dependencies through the GUI library! Thanks a lot for the help! Now just a small question on the scope of these two packages: I found that in the `runBenchmark.js` we can access them globally, but I need to use the relative path to import, e.g. `../../caliper-core`. Did we use `npm install -g caliper-core/` when installing the `caliper` command-line client, so they are globally accessible? Can I just assume that they are globally accessible from the `caliper-gui` directory too?

Jasonyou (Mon, 19 Aug 2019 11:12:54 GMT):
Awesome! Just read through the source code and imported the `caliper-core` and `caliper-fabric` dependencies through the GUI library! Thanks a lot for the help! Now just a small question on the scope of these two packages: I found that in the `runBenchmark.js` we can access them globally, but I need to use the relative path to import when I use them right not, e.g. `../../caliper-core`. Did we use `npm install -g caliper-core/` when installing the `caliper` command-line client, so they are globally accessible? Can I just assume that they are globally accessible from the `caliper-gui` directory too?

Jasonyou (Mon, 19 Aug 2019 11:12:54 GMT):
Awesome! Just read through the source code and imported the `caliper-core` and `caliper-fabric` dependencies through the GUI library! Thanks a lot for the help! Now just a small question on the scope of these two packages: I found that in the `runBenchmark.js` we can access them globally, but I need to use the relative path to import when I use them right now, e.g. `../../caliper-core`. Did we use `npm install -g caliper-core/` when installing the `caliper` command-line client, so they are globally accessible? Can I just assume that they are globally accessible from the `caliper-gui` directory too?

nkl199 (Mon, 19 Aug 2019 11:46:27 GMT):
you should only use the import in your GUI package (ie `const {CaliperFlow, CaliperUtils, ConfigUtil} = require('caliper-core');` kinda stuff). Never a relative path to another package. You mark caliper-core (and any other caliper-x package required) as dependancies in *your* package.json file - the `npm run bootstrap` will sort everything for you and means that if someone wanted you-and-only-your gui package, the npm install will fetch everything that is required

Jasonyou (Mon, 19 Aug 2019 12:04:06 GMT):
I get it, and I found how this works in the Lerna github [README](lerna bootstrap) too! Basically `lerna bootstrap` will link all the local dependencies in the lerna repo together and make them accessible globally. I just want to make sure that the repo structure of my current `caliper-gui` structure is "recognizable" by `lerna` as a valid package format. (I saw that almost all major `caliper-*` packages implemented has a `lib` directory in each of them, does that means I will need to add a lib in the `gui-client` and `gui-server` directories? Another question is about the MongoDB database (can be changed to other substitutions in the future). Do I need to write a script to check and run the installation of MongoDB before the installation of Caliper-GUI?

Jasonyou (Mon, 19 Aug 2019 12:04:06 GMT):
I get it, and I found how this works in the Lerna github [README](https://github.com/lerna/lerna/tree/master/commands/bootstrap#how-it-works) too! Basically `lerna bootstrap` will link all the local dependencies in the lerna repo together and make them accessible globally. I just want to make sure that the repo structure of my current `caliper-gui` structure is "recognizable" by `lerna` as a valid package format. (I saw that almost all major `caliper-*` packages implemented has a `lib` directory in each of them, does that means I will need to add a lib in the `gui-client` and `gui-server` directories? Another question is about the MongoDB database (can be changed to other substitutions in the future). Do I need to write a script to check and run the installation of MongoDB before the installation of Caliper-GUI?

Jasonyou (Mon, 19 Aug 2019 12:06:48 GMT):
``` caliper-gui/ ├── gui-client │ └── package.json ├── gui-server │ ├── api │ ├── node_modules │ ├── package.json │ └── package-lock.json ├── package.json ├── package-lock.json ├── scripts ``` This is the basic idea of how the `caliper-gui` repo looks like now

Jasonyou (Mon, 19 Aug 2019 12:07:20 GMT):
Well I will move the `gui-client` in after the server is done.

nkl199 (Mon, 19 Aug 2019 16:00:44 GMT):
hi, i would have expected the format: ``` caliper-gui/ |--lib | . |-gui-client | . |_client_folder | . |_gui-server | . |_api | . |_other_folder |--pacakge.json ```

nkl199 (Mon, 19 Aug 2019 16:01:12 GMT):
one package for all the components that relate to the gui parts

Jasonyou (Mon, 19 Aug 2019 19:38:47 GMT):
That looks neat! Thanks very much for the help Nick!

nkl199 (Wed, 21 Aug 2019 08:07:44 GMT):
HI all, it would appear that someone has published the caliper pacakges without our knowledge. This is bad because: - well, we want to be in control of this - they are not ready - The integration tests will fail (package conflicts) - They are at the wrong level I'm going to try and see what is possible with regards to getting this fixed, but frankly we could do without it

nkl199 (Wed, 21 Aug 2019 08:57:48 GMT):
further to the above, i have started the npm dispute process

klenik (Wed, 21 Aug 2019 13:33:53 GMT):
The official packages/images will be published under the HL org/namespace, so at least we can avoid the naming conflict. But true, the official code base should be released by us to provide a solid UX for the community

klenik (Wed, 21 Aug 2019 13:35:14 GMT):
@nkl199 I can only stay for the first 20 minutes of the call. What's the agenda beside the publishing discussion?

nkl199 (Wed, 21 Aug 2019 13:35:39 GMT):
Possibly some PR triage

Jasonyou (Fri, 23 Aug 2019 01:14:27 GMT):
Just curious: is it possible that the recently published `caliper-core` will affect the installation of local test `caliper`?

Jasonyou (Fri, 23 Aug 2019 01:15:49 GMT):
Some people in the Caliper community said they got error in `npm run start_verdaccio`

nkl199 (Fri, 23 Aug 2019 08:19:39 GMT):
yes, our entire code base is set at v0.1.0 ... but that is now published, so you can't publish again under that version

Jasonyou (Fri, 23 Aug 2019 15:28:54 GMT):
Too bad... haha. Hopefully the community and NPM can fix this issue

klenik (Fri, 23 Aug 2019 20:53:34 GMT):
@Jasonyou We'll publish organization-scoped packages, so there won't be any naming conflict. The official packages will be named `@hyperledger/caliper-...`

Jasonyou (Sat, 24 Aug 2019 03:55:35 GMT):
Sounds great!

Jasonyou (Sat, 24 Aug 2019 17:00:58 GMT):
I realized that some required dependencies in `caliper-core` for `caliper-flow` testing are not exported in the `caliper-core/index.js`, so do I just copy these scripts manually or do I add exports in index.js?

Jasonyou (Sat, 24 Aug 2019 17:01:04 GMT):
``` { BlockchainInterface: [Function: BlockchainInterface], CaliperLocalClient: [Function: CaliperLocalClient], TxStatus: [Function: TxStatus], CaliperUtils: [Function: CaliperUtils], Version: [Function: Version], ConfigUtil: { get: [Function: get], set: [Function: set], keys: { CoreTxUpdateTime: 'caliper-core-txupdatetime', CoreSkipStartScript: 'caliper-core-skipstartscript', CoreSkipEndScript: 'caliper-core-skipendscript', CoreLogging: 'caliper-core-logging', FabricSleepAfterCreateChannel: 'caliper-fabricccp-sleepafter-createchannel', FabricSleepAfterJoinChannel: 'caliper-fabricccp-sleepafter-joinchannel', FabricSleepAfterInstantiateChaincode: 'caliper-fabricccp-sleepafter-instantiatechaincode', FabricVerifyProposalResponse: 'caliper-fabricccp-verify-proposalresponse', FabricVerifyReadWriteSets: 'caliper-fabricccp-verify-readwritesets', FabricTimeoutChaincodeInstantiate: 'caliper-fabricccp-timeout-chaincodeinstantiate', FabricTimeoutChaincodeInstantiateEvent: 'caliper-fabricccp-timeout-chaincodeinstantiateevent', FabricTimeoutInvokeOrQuery: 'caliper-fabricccp-timeout-invokeorquery', FabricLoadBalancing: 'caliper-fabricccp-loadbalancing', FabricOverwriteGopath: 'caliper-fabricccp-overwritegopath', FabricLatencyThreshold: 'caliper-fabricccp-latencythreshold', FabricCountQueryAsLoad: 'caliper-fabricccp-countqueryasload', FabricSkipCreateChannelPrefix: 'caliper-fabricccp-skipcreatechannel-', FabricGateway: 'caliper-fabricccp-usegateway', FabricGatewayLocalHost: 'caliper-fabricccp-gatewaylocalhost', FabricDiscovery: 'caliper-fabricccp-discovery' } }, CaliperFlow: { run: [AsyncFunction] } } ```

Jasonyou (Sat, 24 Aug 2019 17:01:51 GMT):
For example `Blockchain.js` is not exported, but in `caliper-flow` it was used from local import: `const Blockchain = require('./blockchain');`

Jasonyou (Sat, 24 Aug 2019 17:09:32 GMT):
``` const { CaliperUtils, ConfigUtil, } = require('caliper-core'); // Something dependencies need to be added in local /* const Blockchain = require('...'); // required for test const ClientOrchestrator = require('...'); // ?copy? const MonitorOrchestrator = require('...'); // ?copy? const Report = require('...'); // optional const DefaultTest = require('...'); // local copy is fine */ ``` Basically all the commented ones are not exported from `caliper-core` now.

Jasonyou (Sat, 24 Aug 2019 17:09:32 GMT):
``` const { CaliperUtils, ConfigUtil, } = require('caliper-core'); // Some dependencies need to be added in local /* const Blockchain = require('...'); // required for test const ClientOrchestrator = require('...'); // ?copy? const MonitorOrchestrator = require('...'); // ?copy? const Report = require('...'); // optional const DefaultTest = require('...'); // local copy is fine */ ``` Basically all the commented ones are not exported from `caliper-core` now.

Jasonyou (Sat, 24 Aug 2019 23:51:47 GMT):
The importing error for `gui-caliper-flow.js` is fixed perfectly with the following dependencies and added module exports in the `caliper-core/index.js`:

Jasonyou (Sat, 24 Aug 2019 23:51:51 GMT):
``` const { Blockchain, CaliperUtils, ConfigUtil, ClientOrchestrator, MonitorOrchestrator, Report, DefaultTest, LocalObserver, PrometheusObserver, } = require('caliper-core'); ```

Jasonyou (Sat, 24 Aug 2019 23:52:03 GMT):
``` // More dependencies for caliper-flow test module.exports.ClientOrchestrator = require('./lib/client/client-orchestrator'); module.exports.Blockchain = require('./lib/blockchain'); module.exports.MonitorOrchestrator = require('./lib/monitor/monitor-orchestrator'); module.exports.Report = require('./lib/report/report'); module.exports.DefaultTest = require('./lib/test-runners/default-test'); module.exports.LocalObserver = require('./lib/test-observers/local-observer'); module.exports.PrometheusObserver = require('./lib/test-observers/prometheus-observer'); ```

Jasonyou (Sun, 25 Aug 2019 00:04:58 GMT):
Now the problem is in the network config start command: `export FABRIC_VERSION=1.4.0;docker-compose -f networks/fabric/docker-compose/2org1peercouchdb/docker-compose.yaml up -d;sleep 3s` the path it use are relative path, thus cannot directly be applied to the GUI unless: 1) I change the path manually in the config file, or 2) I structure the sample test workspace as the command.

Jasonyou (Sun, 25 Aug 2019 00:06:53 GMT):
In the future I might spend more time to customize these, but now I will just follow option (2) and see how things works first

Jasonyou (Sun, 25 Aug 2019 02:30:50 GMT):
Hi guy! The GUI for the caliper is basically finished, and everything is integrated, except that right now In the `fabric.js` adaptor #1434 `let resp = await admin.queryChannels(peer, true);` the adaptor still have issue to connect to the `URL:grpc://localhost:7051`, which is the `Org1`'s peer nodes. Currently users can upload, save, and edit the test and network config files directly through the GUI, and the visualizations are finished. However, since the local deployment issue still cannot be solved, I cannot guarantee a working GUI with caliper and blockchain network connected right now. But that's the only part that I might skip in this stage, because that doesn't affect too much of the development of GUI. Right now the front-end and back-end of the GUI is basically finished, and all the functionalities are exactly matching the current Caliper-cli. I will finish up my project in the next feel days, and will make this application usable by the Hyperledger community once the local deployment issue was resolved! Thanks very much for every one that helped me in this summer internship, especially @FlyingTiger, @nkl199, and @FlyingTiger! I cannot accomplish this much without your helps! I will keep contribute to the community, and start to explore more in the Blockchain field!

Jasonyou (Sun, 25 Aug 2019 02:30:50 GMT):
Hi guy! The GUI for the caliper is basically finished, and everything is integrated, except that right now In the `fabric.js` adaptor #1434 `let resp = await admin.queryChannels(peer, true);` the adaptor still have issue to connect to the `URL:grpc://localhost:7051`, which is the `Org1`'s peer nodes. Currently users can upload, save, and edit the test and network config files directly through the GUI, and the visualizations are finished. However, since the local deployment issue still cannot be solved, I cannot guarantee a working GUI with caliper and blockchain network connected right now. But that's the only part that I might skip in this stage, because that doesn't affect too much of the development of GUI. Right now the front-end and back-end of the GUI is basically finished, and all the functionalities are exactly matching the current Caliper-cli. I will finish up my project in the next feel days, and will make this application usable by the Hyperledger community once the local deployment issue was resolved! (the caliper-gui source code will soon be posted, and I will make a pull request once I cleaned up the code and made proper documentations)! Thanks very much for every one that helped me in this summer internship, especially @FlyingTiger, @nkl199, and @FlyingTiger! I cannot accomplish this much without your helps! I will keep contribute to the community, and start to explore more in the Blockchain field!

FlyingTiger (Mon, 26 Aug 2019 11:59:37 GMT):
@Jasonyou :thumbsup: The GUI for the caliper is basically finished. Thanks for your hard work, it is much appreciated.

Jasonyou (Mon, 26 Aug 2019 12:54:42 GMT):
No problem! I will keep contributing and learning!

klenik (Tue, 27 Aug 2019 19:39:01 GMT):
Agenda for tomorrow's 3PM UTC call. Contact info: https://github.com/hyperledger/caliper/wiki/Caliper-Calls - Final(ish) debate about publishing (Proposed approach/docs: https://aklenik.github.io/caliper/docs/Installing_Caliper.html) - Prometheus integration update (plus the new observer approach and implication, e.g., UI integration) - AOB

nkl199 (Thu, 29 Aug 2019 16:01:49 GMT):
It's probably about time we start approaching other H'ledger techs and ask if anyone in their community would like to contribute adaptors to Caliper ... there's a fair few missing and I don't think we can be an expert for all of them!

klenik (Wed, 04 Sep 2019 00:38:32 GMT):
Agenda for today's 3PM UTC call. Contact info: https://github.com/hyperledger/caliper/wiki/Caliper-Calls - Final_v2 debate about publishing, based on the last call's productive brainstorming session (Proposed approach/docs: https://aklenik.github.io/caliper/docs/Installing_Caliper.html) - AOB @nkl199 I'll try to join the call on time, but I'll have a meeting just finishing around that time. Feel free to start the above docs without me :)

Utsav_Solanki (Mon, 09 Sep 2019 09:36:31 GMT):
Has joined the channel.

Utsav_Solanki (Mon, 09 Sep 2019 09:36:33 GMT):
any video document to learn caliper

nkl199 (Mon, 09 Sep 2019 09:37:27 GMT):
not yet - the problem with videos is that they become out of date very quickly ... we are working on documented tutorials though

klenik (Mon, 09 Sep 2019 09:40:06 GMT):
@Utsav_Solanki +1 for what Nick says. While Caliper is under heavy development, providing (and maintaining) video tutorials would be a huge overhead. We rather put that effort into improving Caliper and text-based tutorials

Utsav_Solanki (Mon, 09 Sep 2019 09:41:47 GMT):
can i have a link

Utsav_Solanki (Mon, 09 Sep 2019 09:41:47 GMT):
can i have a link for tutorials

klenik (Mon, 09 Sep 2019 09:42:16 GMT):
https://hyperledger.github.io/caliper/

Utsav_Solanki (Mon, 09 Sep 2019 09:42:32 GMT):
[ ](https://chat.hyperledger.org/channel/caliper-contributors?msg=5HZyLn9pFRTEJYCj2) Thank you

klenik (Mon, 09 Sep 2019 09:42:40 GMT):
The tutorials are work in progress, but you'll find them on that site soon

klenik (Mon, 09 Sep 2019 09:43:06 GMT):
But there are some examples about how to use Caliper

Utsav_Solanki (Mon, 09 Sep 2019 09:43:56 GMT):
ok

Utsav_Solanki (Mon, 09 Sep 2019 12:55:31 GMT):
[ ](https://chat.hyperledger.org/channel/caliper-contributors?msg=5HZyLn9pFRTEJYCj2) does it support 1.4.2

Utsav_Solanki (Mon, 09 Sep 2019 12:55:31 GMT):
[ ](https://chat.hyperledger.org/channel/caliper-contributors?msg=5HZyLn9pFRTEJYCj2) does it support 1.4.2 hyperledger fabric

Utsav_Solanki (Mon, 09 Sep 2019 12:59:28 GMT):
in hyperledger caliper repo they mentioned the latest version that has been verified is v1.4.1

klenik (Mon, 09 Sep 2019 13:02:11 GMT):
@Utsav_Solanki Based on the [release notes](https://github.com/hyperledger/fabric/tree/v1.4.3/release_notes) of Fabric, I think there won't be any issues, just bind to the latest SDK version. And let's move this discussion to the general #caliper channel :)

Utsav_Solanki (Mon, 09 Sep 2019 13:08:15 GMT):
ok, thank you

klenik (Mon, 09 Sep 2019 16:12:47 GMT):
@rjones It looks like [probot-stale](https://probot.github.io/apps/stale/) is installed for the HL github org. I guess it needs an admin to enable it for the caliper repo (and bootstrap it with an encrypted github token)? What is the workflow we should follow?

rjones (Mon, 09 Sep 2019 16:12:47 GMT):
Has joined the channel.

rjones (Mon, 09 Sep 2019 16:13:50 GMT):
@klenik I don't think you need any of that. Just add the stale.yaml in the right place

klenik (Mon, 09 Sep 2019 16:14:13 GMT):
@rjones Oh, nice, I'll do that, thx!

rjones (Mon, 09 Sep 2019 16:14:43 GMT):
For example: https://github.com/hyperledger/fabric/blob/master/.github/stale.yml

klenik (Mon, 09 Sep 2019 16:16:48 GMT):
@nkl199 30 days to stale, then 7 days to close sounds about right for the settings, right?

nkl199 (Mon, 09 Sep 2019 16:17:15 GMT):
sounds good to me!

nkl199 (Mon, 09 Sep 2019 16:17:43 GMT):
so long as it doesnt close bug/enhancement tagged items i guess?

klenik (Mon, 09 Sep 2019 16:18:19 GMT):
Certain labels will be excluded of course :ok_hand:

klenik (Tue, 10 Sep 2019 09:57:24 GMT):
Agenda for the 9.11. 3PM UTC+0 Caliper call. Contact info: https://github.com/hyperledger/caliper/wiki/Caliper-Calls - Scheduling the pending adapter PRs for review (Ethereum and FISCO BCOS adapters for starters) - Collection features to focus on for the v0.2.0 milestone. Some ideas * Making some modules more OO (like user modules) * Improving the life-cycle API for some objects (something like `initializeForClient(clientId)`, `initializeForRound(roundId)`, their finalize/end counterparts, where appropriate). * Clean up the benchmark config a bit (the array notation for rounds is a bit hard to read and parse) * Removing Zookeeper support (already WIP) - Documentation improvements: * DLT tutorials (mainly Fabric for now) * A general architecture docs that reflects the current design * Versioned documentation (e.g., choose the doc version from a dropdown list, like the Composer docs) - AOB

klenik (Tue, 10 Sep 2019 11:30:46 GMT):
- Designing an all-in-one example for the CI

Khaled.MH (Thu, 12 Sep 2019 08:34:51 GMT):
Has joined the channel.

klenik (Mon, 16 Sep 2019 13:28:42 GMT):
:exclamation: *This week's Caliper call is cancelled* :exclamation:

klenik (Mon, 16 Sep 2019 13:30:10 GMT):
Wow, we have a :caliper: icon, awesome! :grin:

rjones (Mon, 16 Sep 2019 13:30:19 GMT):
:caliper:

rjones (Mon, 16 Sep 2019 13:30:31 GMT):
I put them in for every project I have art for

rjones (Mon, 16 Sep 2019 15:33:45 GMT):
Has left the channel.

klenik (Thu, 19 Sep 2019 14:45:15 GMT):
Hey All. I put together a draft for the upcoming :caliper: project update. Feel free to add to it (hopefully more people can edit the same draft), comment on it, etc. You can see the draft after logging in: https://wiki.hyperledger.org/pages/resumedraft.action?draftId=20023040&draftShareId=b6eb7388-0967-47d3-92a9-ca3f57811645& I'd like to publish it by the end of this week to give the TSC ample time to digest it before the verbal presentation next Thursday.

klenik (Tue, 24 Sep 2019 10:10:32 GMT):
Agenda for the 9.24. 3PM UTC+0 Caliper call. Contact info: https://github.com/hyperledger/caliper/wiki/Caliper-Calls - Discussion/status update about the current major PRs (Ethereum, FISCO BCOS, Prometheus) - AOB

klenik (Tue, 24 Sep 2019 10:10:32 GMT):
Agenda for the 9.25. 3PM UTC+0 Caliper call. Contact info: https://github.com/hyperledger/caliper/wiki/Caliper-Calls - Discussion/status update about the current major PRs (Ethereum, FISCO BCOS, Prometheus) - AOB

russanto (Thu, 26 Sep 2019 10:42:07 GMT):
Sorry for my absence in the call

russanto (Thu, 26 Sep 2019 10:42:23 GMT):
@klenik I'll fix what you pointed asap

klenik (Thu, 26 Sep 2019 10:46:53 GMT):
@russanto No problem, it was rather short notice :sweat_smile: Let me know if you need any help/guidance

klenik (Tue, 01 Oct 2019 14:47:19 GMT):
@nkl199 Do we have new things to discuss on the call?

klenik (Tue, 01 Oct 2019 14:47:48 GMT):
@AgniCampos Do you need any assistance with the Corda adapter that we should discuss on the call?

AgniCampos (Tue, 01 Oct 2019 14:47:48 GMT):
Has joined the channel.

nkl199 (Tue, 01 Oct 2019 15:24:11 GMT):
:thinking: - PR review process - Desired contents for next release (and an anticipated date) - Focal points for release++

klenik (Tue, 01 Oct 2019 15:47:22 GMT):
In that case, agenda for the 10.02. 3PM UTC+0 Caliper call. Contact info: https://github.com/hyperledger/caliper/wiki/Caliper-Calls 1) PR review process 2) TODOs till next release 3) Focal points for next-next release

klenik (Tue, 01 Oct 2019 15:50:47 GMT):
Some bootstrap thoughts: 1) I think merging the adapter PRs will be the bottlenecks. The other issues I marked for milestone 0.2.0 are smaller. 2) Beside the marked issues for 0.2.0, I'd like to clean up a bit the round definitions (get rid of the array syntax). 3) The reason I want to leave as few issues for 0.3.0 as possible is because we'll have our hands full with the distributed client implementation. I think that issue alone will take up the whole release-cycle time

klenik (Wed, 02 Oct 2019 11:12:20 GMT):
@russanto Let us know if you need any help with the Ethereum adapter :)

russanto (Wed, 02 Oct 2019 13:18:24 GMT):
Thanks, to be honest I had some troubles yesterday to set up the environment to test changes with the 0.1.0 release

russanto (Wed, 02 Oct 2019 13:19:28 GMT):
each time I have either to stop clean and restart verdaccio or to increase the ethereum adapter version in order to have it updated with changes

klenik (Wed, 02 Oct 2019 13:20:46 GMT):
You can run the CLI directly, without verdaccio. If you keep your samples the the packages/caliper-samples dir, then: ``` cd packages/caliper-samples node ../caliper-cli/caliper.js benchmark run ...... ```

klenik (Wed, 02 Oct 2019 13:21:54 GMT):
This way you run the current/master version of Caliper, not the published packages. And it's easier to debug this way. Once you feel confident about your code, you can try the verdaccio way.

klenik (Wed, 02 Oct 2019 13:26:04 GMT):
@russanto https://hyperledger.github.io/caliper/vLatest/installing-caliper/#testing-the-code

russanto (Wed, 02 Oct 2019 13:27:23 GMT):
Thanks, I completely jumped that line in doc

russanto (Fri, 04 Oct 2019 17:12:23 GMT):
about the ethereum adapter PR, I don't understand where the apache license is absent

russanto (Fri, 04 Oct 2019 17:13:58 GMT):
ah sorry, he meant the identifier, it is "SPDX-License-Identifier: Apache-2.0" right ?

klenik (Fri, 04 Oct 2019 18:08:08 GMT):
@russanto Yeah, but don't worry about it for now. We'll add those to every file (and automatic license test settings) before the next release

klenik (Fri, 04 Oct 2019 18:08:22 GMT):
Just concentrate for the current CI requirements :)

russanto (Sun, 06 Oct 2019 16:54:09 GMT):
ok, I update with necessary things that you pointed in the PR. I wait for your last feedback before doing a single commit with all updates

russanto (Sun, 06 Oct 2019 16:54:09 GMT):
ok, I updated with necessary things that you pointed in the PR. I'm waiting for your last feedback before doing a single commit with all updates

klenik (Mon, 07 Oct 2019 09:31:22 GMT):
@russanto I've left some minor comments on your PR. After addressing them, you can also squash the commits :thumbsup:

russanto (Tue, 08 Oct 2019 11:00:00 GMT):
do I add the ethereum test to the travis.yml too ?

klenik (Tue, 08 Oct 2019 11:10:36 GMT):
@russanto Since you already added the ethereum test in the script, an extra `- env: BENCHMARK=ethereum` in the .travis.yml couldn't hurt :thumbsup:

klenik (Tue, 08 Oct 2019 11:44:20 GMT):
Agenda for the 10.09. 3PM UTC+0 Caliper call. Contact info: https://github.com/hyperledger/caliper/wiki/Caliper-Calls 1) PR review progress 2) TODOs till next/imminent release

klenik (Tue, 08 Oct 2019 11:52:44 GMT):
Easy fix: https://github.com/hyperledger/caliper/pull/432#issuecomment-539468822

russanto (Tue, 08 Oct 2019 13:55:09 GMT):
perfect, thanks

russanto (Tue, 08 Oct 2019 14:13:08 GMT):
fabric test is failing, can it be something related to updates on master that I don't have ?

klenik (Tue, 08 Oct 2019 14:13:49 GMT):
Nope, something broke with the new CouchDB image, we'll fix it soon. As long as the ethereum job is passing, all is good

russanto (Tue, 08 Oct 2019 14:49:25 GMT):
Commits squashed, asap I'll take a look to the doc PR

klenik (Thu, 10 Oct 2019 08:41:22 GMT):
@russanto Merged the Ethereum PRs, awesome work!! :rocket: :fireworks:

russanto (Thu, 10 Oct 2019 08:41:45 GMT):
:thumbsup: thanks for your support

klenik (Wed, 16 Oct 2019 10:28:10 GMT):
:exclamation: Today's :caliper: call is canceled

AgniCampos (Fri, 18 Oct 2019 12:48:40 GMT):
thanks for reply on the other channel Attila- i'll continue discussions here

shemnon (Sat, 19 Oct 2019 13:25:58 GMT):
Has joined the channel.

klenik (Wed, 23 Oct 2019 08:36:02 GMT):
Agenda for the 10.23. 3PM UTC+0 Caliper call. Contact info: https://github.com/hyperledger/caliper/wiki/Caliper-Calls 1) PR review/release process 2) Approaching the distributed client implementation

klenik (Wed, 23 Oct 2019 08:36:02 GMT):
Agenda for the 10.23. 3PM UTC+0 Caliper call. Contact info: https://github.com/hyperledger/caliper/wiki/Caliper-Calls 1) PR review/release progress 2) Approaching the distributed client implementation

nkl199 (Wed, 23 Oct 2019 10:30:41 GMT):
Lyne would like to present her current work on a generator for caliper test files

klenik (Wed, 23 Oct 2019 10:31:17 GMT):
Awesome :thumbsup:

klenik (Wed, 23 Oct 2019 10:32:04 GMT):
We also have Jason's UI, maybe the two have something in common

klenik (Thu, 24 Oct 2019 09:15:47 GMT):
@shemnon Once you squash your Besu commits, the PR is good to go :thumbsup:

shemnon (Thu, 24 Oct 2019 11:18:10 GMT):
Done

klenik (Thu, 24 Oct 2019 11:26:24 GMT):
And merged :) Thanks for the quick and awesome contribution :thumbsup: :champagne:

nkl199 (Thu, 24 Oct 2019 14:17:48 GMT):
:point_up: I second that! :D

klenik (Mon, 28 Oct 2019 07:54:06 GMT):
We'll use the development plan wiki page for features that require non-trivial designs or modifications: https://wiki.hyperledger.org/display/caliper/Caliper+Development+Plan The distributed client (re)implementation falls in this category. I'll start by uploading a sequence diagram about the current data/control-flow in the core module. Feel free to comment on anything, ideas, insights are welcome, the wiki is a nice place for such focused discussions

nkl199 (Mon, 28 Oct 2019 14:40:08 GMT):
tha's a busy image!!

klenik (Mon, 28 Oct 2019 14:41:02 GMT):
Yeah. It starts out nice and clear, then the communication between the processes and update processing happens. :boom:

klenik (Mon, 28 Oct 2019 14:44:36 GMT):
I'll write up some concerns in the wiki, mainly about the following: 1. Caliper-flow has round processing, which should belong to DefaultTest 2. DefaulTest really should be called TestOrchestrator or something 3. The "ready" and "promise" Promises embedded in the `ChildProcess` objects are hard to follow. We should limit the process eventing to a thin wrapper/layer, and provide unified functions for the events for higher modules. 4. Observers, Monitors, Reporters, both in master and clients. Maybe we could "formalize" these modules based on the goals, requirements and interactions

nkl199 (Mon, 28 Oct 2019 15:24:10 GMT):
yup

klenik (Tue, 29 Oct 2019 13:47:03 GMT):
Agenda for the 10.30. 3PM UTC+0 Caliper call. Contact info: https://github.com/hyperledger/caliper/wiki/Caliper-Calls 1) Doing an `unstable` rolling release on every PR merge (npm and docker) 2) Distributed client implementation, points of issues. Bootstrap thoughts: https://wiki.hyperledger.org/display/caliper/Caliper+Development+Plan 3) AOB :exclamation: Don't forget about the recent winter time compensation, might affect your country's local time

klenik (Tue, 05 Nov 2019 11:32:53 GMT):
@shemnon The Besu integration test seems to pass, although it actually errors during deploying the contract: https://travis-ci.org/hyperledger/caliper/jobs/607545421#L1680 The basically same Ethereum test passes, so this might be Besu-related? Could you investigate please?

shemnon (Tue, 05 Nov 2019 15:08:01 GMT):
My first attempt at repcicaiton failed, the scripts when run by hand worked.

shemnon (Tue, 05 Nov 2019 17:52:40 GMT):
I think it's startup delay again, which is why the failure is flakey. I got it when I did a compose up and went straight to the test.

klenik (Tue, 05 Nov 2019 19:12:11 GMT):
And you also got the `Invalid params` error?

shemnon (Tue, 05 Nov 2019 20:28:16 GMT):
Yep. Thats the only way I see that particular error. However I got some more errors I'm prepping a PR for separate from this.

klenik (Tue, 05 Nov 2019 20:56:38 GMT):
I'll increase the sleep time in my PR (messing around with the configs anyway)

klenik (Wed, 06 Nov 2019 10:29:22 GMT):
Agenda for the 11.06. 3PM UTC+0 Caliper call. Contact info: https://github.com/hyperledger/caliper/wiki/Caliper-Calls 1) Progress updates about pending items (unstable publishing, core refactoring, etc) 2) Discuss next short-term items

klenik (Tue, 12 Nov 2019 11:24:52 GMT):
:exclamation: Tomorrow's :caliper: call is canceled. Questions and updates can be discussed in this channel instead. :exclamation:

shemnon (Tue, 12 Nov 2019 21:34:19 GMT):
@mackcom @GregTheGreek How long did syncing to the ETC chain head take? Did you do archival or fast sync?

GregTheGreek (Tue, 12 Nov 2019 21:34:19 GMT):
Has joined the channel.

mackcom (Tue, 12 Nov 2019 21:34:19 GMT):
Has joined the channel.

shemnon (Wed, 13 Nov 2019 06:41:47 GMT):
The recent round refactoring seems to have broken arguments for multiple clients. For example: the iroha simple test won't run as the second client gets 'undefined' as it's args - ``` node ./packages/caliper-cli/caliper.js benchmark run --caliper-workspace ./packages/caliper-samples --caliper-benchconfig benchmark/simple/config.yaml --caliper-networkconfig network/iroha/simplenetwork/iroha.json [...] 2019.11.12-23:38:06.663 error [caliper] [iroha.js] Error within getContext: TypeError: Cannot read property 'hasOwnProperty' of undefined at Iroha.getContext (/Users/dannoferrin/git/github.com/shemnon/caliper/packages/caliper-iroha/lib/iroha.js:343:22) at Blockchain.getContext (/Users/dannoferrin/git/github.com/shemnon/caliper/packages/caliper-core/lib/common/core/blockchain.js:76:33) at CaliperLocalClient.doTest (/Users/dannoferrin/git/github.com/shemnon/caliper/packages/caliper-core/lib/worker/client/caliper-local-client.js:320:49) at MessageHandler.test [as testHandler] (/Users/dannoferrin/git/github.com/shemnon/caliper/packages/caliper-core/lib/worker/client/message-handler.js:104:37) at Function.handle (/Users/dannoferrin/git/github.com/shemnon/caliper/packages/caliper-core/lib/worker/client/message-handler.js:131:52) at process._tickCallback (internal/process/next_tick.js:68:7) 2019.11.12-23:38:06.664 info [caliper] [local-client.js] Client[5522] encountered an error: Error: Failed when finding access point or user key at Iroha.getContext (/Users/dannoferrin/git/github.com/shemnon/caliper/packages/caliper-iroha/lib/iroha.js:391:35) at Blockchain.getContext (/Users/dannoferrin/git/github.com/shemnon/caliper/packages/caliper-core/lib/common/core/blockchain.js:76:33) at CaliperLocalClient.doTest (/Users/dannoferrin/git/github.com/shemnon/caliper/packages/caliper-core/lib/worker/client/caliper-local-client.js:320:49) at MessageHandler.test [as testHandler] (/Users/dannoferrin/git/github.com/shemnon/caliper/packages/caliper-core/lib/worker/client/message-handler.js:104:37) at Function.handle (/Users/dannoferrin/git/github.com/shemnon/caliper/packages/caliper-core/lib/worker/client/message-handler.js:131:52) at process._tickCallback (internal/process/next_tick.js:68:7) 2019.11.12-23:38:06.664 error [caliper] [message-handler] Error while handling "test" message: Error: Failed when finding access point or user key at Iroha.getContext (/Users/dannoferrin/git/github.com/shemnon/caliper/packages/caliper-iroha/lib/iroha.js:391:35) at Blockchain.getContext (/Users/dannoferrin/git/github.com/shemnon/caliper/packages/caliper-core/lib/common/core/blockchain.js:76:33) at CaliperLocalClient.doTest (/Users/dannoferrin/git/github.com/shemnon/caliper/packages/caliper-core/lib/worker/client/caliper-local-client.js:320:49) at MessageHandler.test [as testHandler] (/Users/dannoferrin/git/github.com/shemnon/caliper/packages/caliper-core/lib/worker/client/message-handler.js:104:37) at Function.handle (/Users/dannoferrin/git/github.com/shemnon/caliper/packages/caliper-core/lib/worker/client/message-handler.js:131:52) at process._tickCallback (internal/process/next_tick.js:68:7) [...] ```

shemnon (Wed, 13 Nov 2019 06:42:26 GMT):
My work on besu is impacted becuase we also are depending on non-undefined args values. Like Iroha we need to track sequence numbers for transactions per client for multi-client setup.

klenik (Wed, 13 Nov 2019 07:35:47 GMT):
Will look into it. Or if you found the place of the bug, feel free to raise a pr

klenik (Wed, 13 Nov 2019 09:42:37 GMT):
@shemnon I'm trying to replicate it now

klenik (Wed, 13 Nov 2019 10:58:19 GMT):
@shemnon @nkl199 Raised the fix PR (https://github.com/hyperledger/caliper/pull/649)

nkl199 (Wed, 13 Nov 2019 11:01:58 GMT):
:thumbsup:

nkl199 (Wed, 13 Nov 2019 11:03:05 GMT):
did u intend to leave the comments in the config-record-replay-rate.yaml file?

klenik (Wed, 13 Nov 2019 11:04:08 GMT):
Yeah, it kind of shows what needs to be filled/overridden at the place of use (could have clarified it a bit though :sweat_smile: ).

nkl199 (Wed, 13 Nov 2019 11:11:30 GMT):
:thumbsup:

klenik (Wed, 13 Nov 2019 11:34:11 GMT):
@shemnon The fix is merged

klenik (Thu, 14 Nov 2019 10:03:07 GMT):
[at]here: Some thoughts about the new TX stat collection and the trimming feature. - Trimming is used to exclude some TXs from the stats in the beginning of a round (which is typically considered a warmup period, often resulting in some irregular TX stats). - The new TX stat collection mechanism has precise access to the number and create time of TXs in the worker nodes (which is closest to the TX sending). - Trimming can be implemented in this collector on a per-TX basis, and not on already aggregated stats (as currently, and the precision of trimming depends on the aggregation time). - The effects of such precise TX trimming is that the trimmed TXs are completely hidden from every part of Caliper (since the results are technically dropped): 1. They're not included in the the submitted TX number (used for the `txNumber`-based TX generation, or used by rate controllers) 2. They're not included in the gathered statistics, which means that the TxObservers (which can report to the manager node, or push the current stats to a Prometheus push gateway, or just log them somewhere) won't see them. 3. This effect will be consistent across every sub-module of Caliper (both in the master and worker nodes). Even though trimming can be achieved through Prometheus (if used), it only supports time-based trimming. I think this would be a nice-to-have feature. It is opt-in, of course, and any user can do her/his own post-processing on the somehow exported results (allowing more exotic data manipulation, not just trimming). But if we can drop some TXs out of the box, maybe it helps the further processing of data. Any thoughts on this are welcome :)

shemnon (Thu, 14 Nov 2019 19:42:00 GMT):
Any node experts that can tell me why my CI build failed? https://travis-ci.org/hyperledger/caliper/jobs/612026412?utm_medium=notification&utm_source=github_status - shouldn't the build grab the websocket dependency autmatically?

klenik (Thu, 14 Nov 2019 20:18:02 GMT):
@shemnon I don't see it in the package deps. Was it explicitly listed before? Maybe it was accidentally removed?

shemnon (Thu, 14 Nov 2019 20:19:38 GMT):
No, it wasn't listed. I added a dep to `"ethereumjs-wallet": "^0.6.3"`, which has a dep to `web3` which is listed as a dev dep. Do I need to move web3?

klenik (Thu, 14 Nov 2019 20:19:44 GMT):
Hmm, according to git blame, it was never listed. Maybe it was loaded indirectly, and that deps' deps changed. You should add it to deps

nkl199 (Fri, 15 Nov 2019 09:56:28 GMT):
web3 had a version bump 18hrs ago, i've dropped the version in the ethereum adaptor down to 1.2 and spinning a build to see if that was the issue

nkl199 (Fri, 15 Nov 2019 09:56:28 GMT):
web3 had a version bump 18hrs ago, i've dropped the version in the ethereum adaptor down to 1.2.2 and spinning a build to see if that was the issue

klenik (Fri, 15 Nov 2019 09:58:08 GMT):
And that's why the binding pulls explicit versions :joy: We should adopt the same for important deps to avoid further surprises

nkl199 (Fri, 15 Nov 2019 09:58:25 GMT):
yh, that would be a smart move

nkl199 (Fri, 15 Nov 2019 09:59:32 GMT):
would also explain why everything is passing locally, because i have a package.lock file enforcing what it pulled last time :joy:

klenik (Fri, 15 Nov 2019 10:00:19 GMT):
Yeah, maybe it's time we commit a lock file, as npm suggests :D

nkl199 (Fri, 15 Nov 2019 10:11:10 GMT):
builds passed - raising PR

nkl199 (Fri, 15 Nov 2019 10:29:57 GMT):
@shemnon - If you rebase and force push, your build should pass now :thumbup:

klenik (Wed, 20 Nov 2019 06:53:54 GMT):
Agenda for the 11.20. 3PM UTC+0 Caliper call. Contact info: https://github.com/hyperledger/caliper/wiki/Caliper-Calls 1) A quick presentation about the vNext architecture 2) Monitoring and reporting status (now that it's been cleaned up a bit) 3) AOB

klenik (Wed, 20 Nov 2019 10:46:15 GMT):
Uploaded the presentation to the dev wiki page: https://wiki.hyperledger.org/pages/viewpageattachments.action?pageId=9110170&metadataLink=true

klenik (Wed, 27 Nov 2019 11:13:09 GMT):
Agenda for the 11.20. 3PM UTC+0 Caliper call. Contact info: https://github.com/hyperledger/caliper/wiki/Caliper-Calls 1) Progress report on the manager-worker refactoring (mainly for @nkl199, my tasks haven't really progressed since last week :sweat_smile: ) 2) AOB

klenik (Wed, 27 Nov 2019 11:13:09 GMT):
Agenda for the 11.27. 3PM UTC+0 Caliper call. Contact info: https://github.com/hyperledger/caliper/wiki/Caliper-Calls 1) Progress report on the manager-worker refactoring (mainly for @nkl199, my tasks haven't really progressed since last week :sweat_smile: ) 2) AOB

nkl199 (Wed, 27 Nov 2019 11:42:57 GMT):
(might be time to update the planning document linked in the header for this channel ... )

klenik (Wed, 27 Nov 2019 12:15:24 GMT):
@rjones I think only you have access to the channel topic settings, could you change the planning document link to this? https://wiki.hyperledger.org/display/caliper/Caliper+Development+Plan

rjones (Wed, 27 Nov 2019 12:15:24 GMT):
Has joined the channel.

rjones (Wed, 27 Nov 2019 17:10:59 GMT):
klenik

rjones (Wed, 27 Nov 2019 17:11:21 GMT):
@klenik now you have the power :)

klenik (Thu, 28 Nov 2019 11:02:52 GMT):
Discuss implementation details of Hyperledger Caliper. See [planning document](https://wiki.hyperledger.org/display/caliper/Caliper+Development+Plan)

klenik (Thu, 28 Nov 2019 11:02:52 GMT):
Used for contributors of Hyperledger Caliper to discuss implementation details. See [planning document](https://wiki.hyperledger.org/display/caliper/Caliper+Development+Plan)

klenik (Thu, 28 Nov 2019 11:07:05 GMT):
Yes, I have *insert unlimited power meme here*

klenik (Thu, 28 Nov 2019 11:07:05 GMT):
Yes, I have /insert unlimited power meme here/

RahulEth (Fri, 29 Nov 2019 10:40:14 GMT):
Has joined the channel.

RahulEth (Fri, 29 Nov 2019 10:40:15 GMT):
hi all i am following below repostry https://github.com/hyperledger/caliper/ want to setup caliper for hyperledger fabric. please help thanks

nkl199 (Mon, 02 Dec 2019 09:15:43 GMT):
@RahulEth https://hyperledger.github.io/caliper/vLatest/getting-started/ ?

ArjunC 2 (Tue, 03 Dec 2019 06:12:48 GMT):
Has joined the channel.

ArjunC 2 (Tue, 03 Dec 2019 06:12:51 GMT):
hii have anyone tried nextledger accelerator with caliper for fabric network 1.4?

nkl199 (Tue, 03 Dec 2019 17:05:18 GMT):
I think that is a no from those that monitor this channel

nkl199 (Tue, 03 Dec 2019 17:05:44 GMT):
-> agenda item for tomorrow: mqtt broker configuration

klenik (Tue, 03 Dec 2019 17:50:39 GMT):
Haven't heard of such integration yet. The TX blur would be challenging to handle on Caliper side. I'll have a look at the whitepaper later.

klenik (Tue, 03 Dec 2019 17:51:46 GMT):
Agenda for the 12.04. 3PM UTC+0 Caliper call. Contact info: https://github.com/hyperledger/caliper/wiki/Caliper-Calls 1) MQTT broker configuration 2) AOB

RahulEth (Wed, 04 Dec 2019 06:33:23 GMT):
i want to run test cases for fabric but not able to setup all the things

nkl199 (Wed, 04 Dec 2019 09:16:49 GMT):
can you elaborate on "allthe things" ?

nkl199 (Wed, 04 Dec 2019 09:16:49 GMT):
can you elaborate on "all the things" ?

RahulEth (Wed, 04 Dec 2019 09:18:05 GMT):
i am my own custom network with two orgs, raft ordering service and my own chaincode .

nkl199 (Wed, 04 Dec 2019 09:18:06 GMT):
Accelerator itself (iirc) is a batching mechanism that sits in front of Fabric, and would require its own adaptor component.

RahulEth (Wed, 04 Dec 2019 09:18:46 GMT):
using fabric 1.4.2

nkl199 (Wed, 04 Dec 2019 09:18:53 GMT):
https://developer.ibm.com/tutorials/blockchain-performance-testing-smart-contracts-vscode-caliper/

nkl199 (Wed, 04 Dec 2019 09:18:56 GMT):
this might help

RahulEth (Wed, 04 Dec 2019 09:19:38 GMT):
will look it thanks . have you tried it?

nkl199 (Wed, 04 Dec 2019 09:19:47 GMT):
kinda ... I wrote it :joy:

RahulEth (Wed, 04 Dec 2019 09:20:18 GMT):
thanks again boss!! :grinning:

nkl199 (Wed, 04 Dec 2019 09:20:25 GMT):
lol, no worries

RahulEth (Wed, 04 Dec 2019 12:41:33 GMT):
will contact you if required

nkl199 (Mon, 09 Dec 2019 16:55:38 GMT):
well, the modularisation of the worker-orchestrator messenger is now in a PR :scream:

klenik (Mon, 09 Dec 2019 16:57:22 GMT):
Yeah, got the notification, with mixed feelings as a reviewer :sweat_smile: But now I can rebase in peace and mess around freely with the dataflow :D

nkl199 (Mon, 09 Dec 2019 17:07:47 GMT):
hahahah

nkl199 (Mon, 09 Dec 2019 17:08:04 GMT):
aye, i'm hoping that the modularization makes it easier?

klenik (Mon, 09 Dec 2019 17:08:39 GMT):
Yeah, it definitely will :chart_with_upwards_trend:

klenik (Wed, 11 Dec 2019 11:18:10 GMT):
Agenda for the 12.11. 3PM UTC+0 Caliper call. Contact info: https://github.com/hyperledger/caliper/wiki/Caliper-Calls 1) Discussion about supporting multiple version of the same adapter, while still exposing only a single adapter (use cases: Ethereum adapter with fundamentally different SDK/programming models, single Fabric adapter for v1 and v2) 2) AOB

atoulme (Mon, 16 Dec 2019 21:23:56 GMT):
Has joined the channel.

klenik (Wed, 18 Dec 2019 12:08:18 GMT):
:exclamation: The 12.18. 3PM UTC+0 Caliper call is canceled :exclamation:

JerielFrancisMercado (Thu, 19 Dec 2019 01:44:55 GMT):
Has joined the channel.

klenik (Mon, 23 Dec 2019 10:18:44 GMT):
@rjones When did the migration to travis-ci.com happened? And what are the implications/benefits compared to the org version?

rjones (Mon, 23 Dec 2019 13:59:22 GMT):
@klenik last week. there should be no changes. the reason to migrate was a number of projects existed both in .org and .com and were showing inconsistent results

klenik (Mon, 23 Dec 2019 15:42:13 GMT):
Okay, thanks for the clarification :thumbsup:

nkl199 (Tue, 24 Dec 2019 10:59:37 GMT):
I strongly suspect that the Caliper call will not go ahead tomorrow (25/12) on account of the day it is falling on :santa:

klenik (Tue, 24 Dec 2019 11:04:44 GMT):
I think I share your suspicion, so tomorrow's call is canceled and happy holidays everyone! :)

FlyingTiger (Tue, 24 Dec 2019 12:15:01 GMT):
Merry Christmas :grinning:

klenik (Tue, 31 Dec 2019 12:10:23 GMT):
Tomorrow is also a holiday, so the next call will happen at 8th of January. Happy new year everyone 🎉🎊

nkl199 (Wed, 08 Jan 2020 12:17:10 GMT):
Maintainers call agenda (feel free to add/extend under a thread): - Current PRs - New CLI commands following the ability to have stand alone clients that are communicated via a messaging protocol

klenik (Wed, 08 Jan 2020 14:08:22 GMT):
- The orderer connection management issue might also worth a few sentences - Silona recently suggested a blog post about Caliper

klenik (Wed, 15 Jan 2020 13:04:27 GMT):
Maintainers call agenda (feel free to add/extend under a thread): 1. Feature set for v0.3.0 (or issues to fix until release) 2. Unified plugin handling (message transports, observers, monitors, rate controllers, adapters) 3. CLI structure (master, worker, binding, generators, etc)

klenik (Wed, 15 Jan 2020 14:19:28 GMT):
4. Documentation improvements :)

nkl199 (Thu, 23 Jan 2020 17:01:03 GMT):
following from doc improvements, i think we should have the "install from npm" at the top, and assume someone has installed from npm globally. We can go into the install from source at the bottom as usual but people seem to be hitting the issue where they are running unstable code against benchmarks that are incompatible

nkl199 (Thu, 23 Jan 2020 17:01:03 GMT):
following from doc improvements, i think we should have the "install from npm" at the top of the `nstalling and Running Caliper` page, and assume someone has installed from npm globally. We can go into the CLI command as per a global install and move to the install from source at the bottom as usual but people seem to be hitting the issue where they are running unstable code against benchmarks that are incompatible

nkl199 (Thu, 23 Jan 2020 17:01:03 GMT):
following from doc improvements, i think we should have the "install from npm" at the top of the `Installing and Running Caliper` page, and assume someone has installed from npm globally. We can go into the CLI command as per a global install and move to the install from source at the bottom as usual but people seem to be hitting the issue where they are running unstable code against benchmarks that are incompatible

klenik (Thu, 23 Jan 2020 17:27:35 GMT):
TL;DR; We need to tag the bench repo. I think there is some confusion in the terminology that should be clarified (I don't know the official npm terminology). The `Local NPM install` in the docs means that the `@hyperledger/caliper-cli` package is installed into the current/workspace directory, not to the global npm package dir on the machine/user-level. But it's still installed from the npm registry, without needing the cloned Caliper repo. The `Global NPM install` is the same, but the package (from the npm registry) is installed into the machine/user-level npm package dir. If the user only wants to exercise the same SUT (both type and version) multiple times, then there's no practical difference between the two approaches (in other cases, see the notes in the docs about project separation). The `Installing locally from source` part is mainly intended for developers, and true, it can lead to mismatches compared to the bench repo files, but this is the "unstable" area, so all bets are off. I think the reason for the recent user problems is that we modified configuration files in the bench repo according to the changes in the current unstable/unreleased version. Which contains breaking changes compared to v0.2.0. The correct but cumbersome fix would be to make the bench repo compatible again with v0.2.0 and tag it as v0.2.0. Then re-apply the changes corresponding to the new changes in the main repo since the release. And never forget to tag the bench repo upon a new release :)

nkl199 (Thu, 06 Feb 2020 17:18:30 GMT):
Putting out some thoughts relating to the current way benchmarks are defined. At the moment we have: ``` test: rounds: - - - ``` So the notion is that we have a test, with a series of rounds. I don't really feel that this is entirely correct, or perhaps it is and i'm not sticking to an assumed convention. Are we in fact saying that we have a `test-suite` that is comprised of a series of (one or more) `tests`? Because the notion of `rounds` implies that I'm going to be doing the same thing again and again, which I can achieve by running the benchmark multiple times. I guess what I'm getting at is ... shall we change this benchmark configuration file naming convention to reflect that the configuration defines a test suite, comprised of multiple tests, where the output can be used for benchmarking?

klenik (Thu, 06 Feb 2020 17:23:17 GMT):
Valid question. Although test-suites and tests have their own solid definitions in the testing domain, usually implying the independence between tests. So I'd choose a third naming convention :D something like "benchmark" and "phases"

klenik (Thu, 06 Feb 2020 17:24:05 GMT):
Just like the data ingestion phase of the tpc benchmarks. We should probably look up the terminology in those standards

klenik (Thu, 06 Feb 2020 17:25:00 GMT):
But I agree that rounds implicate something like fight rounds :sweat_smile:

nkl199 (Thu, 06 Feb 2020 17:26:17 GMT):
yeah, it's something I keep looking at and my inner voice tells me it is wrong, but I can't think of a better term, so I forget it

klenik (Thu, 06 Feb 2020 17:27:27 GMT):
I have some colleagues in the benchmark domain, I'll ask around, just ping me before the next call :)

klenik (Thu, 06 Feb 2020 20:41:21 GMT):
A more imminent issue is the benchmark repo's versioning. We should revert some commits since the v0.2.0 release, tag the repo, and reapply the commits. Or since 0.3.0 is around the corner, we could do it then

nkl199 (Fri, 07 Feb 2020 09:12:03 GMT):
either way, but yes ... should be easy as we can revert the last few changes that broke people

stone-ch (Tue, 11 Feb 2020 08:21:57 GMT):
Has joined the channel.

nkl199 (Wed, 12 Feb 2020 13:26:45 GMT):
unfortunately I am unable to attend today's meeting due to a schedule conflict :(

klenik (Wed, 12 Feb 2020 14:01:07 GMT):
Same for me (sick day), so I guess the call is canceled

CatarinaDiegues (Wed, 19 Feb 2020 23:06:30 GMT):
Has joined the channel.

rjones (Mon, 24 Feb 2020 20:26:13 GMT):
[Caliper question on Reddit](https://old.reddit.com/r/hyperledger/comments/f8wmhd/does_anyone_have_an_eta_on_when_caliper_will_be/)

nkl199 (Tue, 25 Feb 2020 13:20:01 GMT):
it's a confusing question - fabric-sdk is compatible with Fabric 2.0 ... so it is already supported

rjones (Tue, 25 Feb 2020 15:44:46 GMT):
I dunno. tell them that, maybe?

rjones (Tue, 25 Feb 2020 16:31:43 GMT):
hmm. All of the documentation says fabric 1.4

rjones (Tue, 25 Feb 2020 18:51:54 GMT):
does Caliper rely on composer for Fabric support? all of the Composer packages were unpublished on NPM

nkl199 (Wed, 26 Feb 2020 09:12:36 GMT):
we have marked the composer package as deprecated and removed it from the caliper code base

nkl199 (Wed, 26 Feb 2020 09:12:36 GMT):
we have marked the composer addaptor as deprecated and removed it from the caliper code base

nkl199 (Wed, 26 Feb 2020 09:12:36 GMT):
we have marked the composer adaptor as deprecated and removed it from the caliper code base

klenik (Wed, 26 Feb 2020 09:58:51 GMT):
The composer packages are still there: https://www.npmjs.com/package/@hyperledger/caliper-composer

klenik (Wed, 26 Feb 2020 09:59:31 GMT):
NPM doesn't allow unpublishing after 72 hours

rjones (Wed, 26 Feb 2020 16:39:55 GMT):
Do you want them to go away?

rjones (Wed, 26 Feb 2020 16:40:31 GMT):
I was trying to build caliper yesterday, and I couldn't. Part of the failure was hitting composer issues.

rjones (Wed, 26 Feb 2020 16:40:51 GMT):
@klenik I can have that package unpublished if you want.

rjones (Wed, 26 Feb 2020 16:42:05 GMT):
I was able to have a bunch of NPM stuff removed for Caliper.

rjones (Wed, 26 Feb 2020 16:42:05 GMT):
I was able to have a bunch of NPM stuff removed for COmposer.

klenik (Wed, 26 Feb 2020 16:52:41 GMT):
Ah, I though you meant that the caliper-composer packages were unpublished... My bad.

klenik (Wed, 26 Feb 2020 16:53:54 GMT):
If the composer packages are gone, then that could cause a problem

rjones (Wed, 26 Feb 2020 16:54:00 GMT):
they're gone.

rjones (Wed, 26 Feb 2020 16:54:42 GMT):
I was confused, sorry.

rjones (Wed, 26 Feb 2020 16:54:51 GMT):
I deleted all of the docker hub images, not composer.

klenik (Wed, 26 Feb 2020 16:54:55 GMT):
Although the install instructions on the Caliper page still work. The key part now is the `--prod-only` npm install. That will skip the SDK installs. And then just don't bind to composer :joy:

klenik (Wed, 26 Feb 2020 16:56:04 GMT):
https://www.npmjs.com/package/composer-client Still there, but deprecated. But this won't be a problem anymore once Caliper v0.3.0 is released soon

rjones (Wed, 26 Feb 2020 16:56:29 GMT):
yeah, I was confused. I'm sorry for the static. However... it still wouldn't build :)

klenik (Wed, 26 Feb 2020 16:56:38 GMT):
[ ](https://chat.hyperledger.org/channel/caliper-contributors?msg=3CkrLwRwTmQYQ5BRE) How did you try to build Caliper?

rjones (Wed, 26 Feb 2020 16:56:39 GMT):
there were errors about getting gPRC

rjones (Wed, 26 Feb 2020 16:57:00 GMT):
I set up a fresh Ubuntu VM and started plowing through

rjones (Wed, 26 Feb 2020 16:57:25 GMT):
(I'm opening the VM now)

rjones (Wed, 26 Feb 2020 16:58:21 GMT):
```node-pre-gyp ERR! Tried to download(403): https://storage.googleapis.com/grpc-precompiled-binaries/node/grpc/v1.10.1/node-v79-linux-x64-glibc.tar.gz node-pre-gyp ERR! Pre-built binaries not found for grpc@1.10.1 and node@13.9.0 (node-v79 ABI, glibc) (falling back to source compile with node-gyp) node-pre-gyp ERR! Pre-built binaries not installable for grpc@1.10.1 and node@13.9.0 (node-v79 ABI, glibc) (falling back to source compile with node-gyp) node-pre-gyp ERR! Hit error Connection closed while downloading tarball file ```

rjones (Wed, 26 Feb 2020 16:58:53 GMT):
`ry@ubu-i:~/caliper$ npx caliper-cli --version`

klenik (Wed, 26 Feb 2020 16:59:21 GMT):
Hmm, what's the node version? 13.x?

rjones (Wed, 26 Feb 2020 17:00:12 GMT):
`node-pre-gyp ERR! cwd /home/ry/.npm/_npx/7372/lib/node_modules/caliper-cli/node_modules/composer-connector-hlfv1/node_modules/grpc node-pre-gyp ERR! node -v v13.9.0 node-pre-gyp ERR! node-pre-gyp -v v0.7.0 node-pre-gyp ERR! not ok `

rjones (Wed, 26 Feb 2020 17:00:12 GMT):
```node-pre-gyp ERR! cwd /home/ry/.npm/_npx/7372/lib/node_modules/caliper-cli/node_modules/composer-connector-hlfv1/node_modules/grpc node-pre-gyp ERR! node -v v13.9.0 node-pre-gyp ERR! node-pre-gyp -v v0.7.0 node-pre-gyp ERR! not ok ```

klenik (Wed, 26 Feb 2020 17:00:16 GMT):
Official support is 8.x LTS and 10.x LTS.

rjones (Wed, 26 Feb 2020 17:01:03 GMT):
OK.

rjones (Wed, 26 Feb 2020 17:01:46 GMT):
honestly, this whole experience was because I was trying to prove that your comment about it should work with fabric 2.0 was true :)

klenik (Wed, 26 Feb 2020 17:01:47 GMT):
We've had a lot of issues with grpc and node. Currently we stabilized it for 8 and 10. We'll look into the newer versions, but probably not for this release

klenik (Wed, 26 Feb 2020 17:03:52 GMT):
I think we can put a how-to on the Fabric config page for 2.0. Nick got it working, but the SDK is changing constantly AFAIK, so it's not easy to keep up

rjones (Wed, 26 Feb 2020 17:05:16 GMT):
yeah, I hear you. I would think the Fabric 2.0 should be stable now that it's released, no?

klenik (Wed, 26 Feb 2020 17:07:57 GMT):
Fabric 2 might be stable, but the SDK still doesn't have a stable 2.0.0 version (only beta), according to npm: https://www.npmjs.com/package/fabric-network

rjones (Wed, 26 Feb 2020 17:08:44 GMT):
ah yes I see. https://www.npmjs.com/package/fabric-network/v/2.0.0-snapshot.309 shows that.

klenik (Tue, 03 Mar 2020 21:33:48 GMT):
:exclamation: This week's :caliper: call is canceled due to the GF. :exclamation:

klenik (Mon, 09 Mar 2020 15:49:21 GMT):
Maintainers call agenda for 03.11. (feel free to add/extend under a thread): 1. Updating the benchmark repo for v0.3.0 2. Consider providing doc pages for the previous stable release (so have vPrev, vLatest and vNext). 3. Add Fabric v2 binding and test the v2 adapter with the official v2 images/sdk (for v0.3.1)

klenik (Mon, 09 Mar 2020 15:56:06 GMT):
4. Install core package separately in the docker image, so it is available for workload modules (otherwise it is just a dep of other packages, and npm can't resolve it when required from a workload module). 5. Enhanced start/end scripts across config files.

klenik (Mon, 09 Mar 2020 16:32:39 GMT):
6. Discuss following potential features: - start/end scripts separetely for master/worker (item 5 above) - some basic data broadcast among worker modules: master -> workers and workers -> workers (similarly to how the adapters can send data to workers). - The above will require to update the life-cycle management of workload modules, and finally convert them to classes if possible (there's an issue for that) - Introduce a `sequential` rate controller, which is not actually a rate controller, but a separate loop-variant in the workers without a rate controller. Most benchmarks require seqential requests on the worker/client/terminal level. Currently this requires some advanced Caliper magic with controllers and workload modules. - Introduce `conditional` TX drive mode beside txDuration and txNumber. This would mean passing an extra "cancellation token" to rate controllers and workload modules, so anyone can cancel a round upon some conditional completion. The txNumber or txDuration is sometimes hard to calculate, especially for data loading rounds before the actual benchmark

klenik (Wed, 01 Apr 2020 13:01:51 GMT):
Just to clarify because of the summer time, the call starts in 2 hours. Right? :sweat_smile: @nkl199 @shemnon

nkl199 (Wed, 01 Apr 2020 13:02:15 GMT):
I can confirm!

shemnon (Wed, 01 Apr 2020 15:40:56 GMT):
So for besu versions, are `1.3.x` and `1.4.x` acceptable identifiers? or do you want all versions spelled out?

rjones (Wed, 01 Apr 2020 15:44:32 GMT):
I would advise using SEMVER (not listing the final digit)

klenik (Wed, 01 Apr 2020 15:49:13 GMT):
The versions in the binding are mainly determined by the SDK. So if the same package can be used for all 1.3.x Besu version, then I'd also suggest using 1.3 and 1.4 as versions

klenik (Wed, 01 Apr 2020 15:52:10 GMT):
Since there's already a 1.3.2 binding, maybe that could be changed to 1.3.0 (to be consistent with the other versions), and note in the docs, that it also works for all patch versions

shemnon (Wed, 01 Apr 2020 16:14:53 GMT):
I'll propose patches for `1.3` and `1.4` later today along with the docs note. This will be two patches, one in gh-pages and one in master.

klenik (Wed, 01 Apr 2020 16:31:01 GMT):
:thumbsup:

shemnon (Thu, 02 Apr 2020 13:57:47 GMT):
The TSC project update for caliper is a week or two late: https://wiki.hyperledger.org/display/TSC/2020+TSC+Project+Update+Calendar

klenik (Thu, 02 Apr 2020 19:05:40 GMT):
Oh, you're right, didn't notice the email notification :sweat_smile: Thanks for the reminder, I'll submit it tomorrow :thumbsup:

shemnon (Fri, 03 Apr 2020 16:43:40 GMT):
review request - docs updates for besu versions - https://github.com/hyperledger/caliper/pull/786 and https://github.com/hyperledger/caliper/pull/787__

shemnon (Fri, 03 Apr 2020 16:43:40 GMT):
review request - docs updates for besu versions - https://github.com/hyperledger/caliper/pull/786 and https://github.com/hyperledger/caliper/pull/787

shemnon (Fri, 03 Apr 2020 16:44:31 GMT):
review request - change besu version in benchmarks - https://github.com/hyperledger/caliper-benchmarks/pull/52

nkl199 (Mon, 06 Apr 2020 11:08:21 GMT):
minor mod request on the bindings only - there are currently 2 `*besu-latest` tags ... should only be one (i assume appleid to the 1.4 version)

jramps9 (Tue, 07 Apr 2020 15:53:03 GMT):
Has joined the channel.

jramps9 (Tue, 07 Apr 2020 15:53:04 GMT):
Hello Caliper contributors! Friendly reminder to pls join us for the Marketing Committee-DevRel call tomorrow, 4/8 at 9am PT. This is a great opportunity to learn how you can get involved in many aspects of Hyperledger marketing including overall messaging, events, content, social media, PR etc. Feel free to add items to the agenda: https://wiki.hyperledger.org/x/_QjcAQ Hope to see you there :slight_smile:

klenik (Tue, 07 Apr 2020 21:58:24 GMT):
Submitted the project update, feel free to extend it if I forgot something: https://wiki.hyperledger.org/display/TSC/2020+Q1+Hyperledger+Caliper

klenik (Wed, 08 Apr 2020 09:41:50 GMT):
@rjones Just out of curiosity, how does the [CII Best Practices](https://bestpractices.coreinfrastructure.org/en/projects/2381) review process work. Are the criteria evaluated from time to time by LF/HL people, should we trigger such reviews, or we trigger it by editing certain entries?

klenik (Wed, 08 Apr 2020 09:41:50 GMT):
@rjones Just out of curiosity, how does the [CII Best Practices](https://bestpractices.coreinfrastructure.org/en/projects/2381) review process work? Are the criteria evaluated from time to time by LF/HL people, should we trigger such reviews, or we trigger it by editing certain entries?

rjones (Wed, 08 Apr 2020 15:34:05 GMT):
@klenik it's done via assertion

rjones (Wed, 08 Apr 2020 15:34:05 GMT):
@klenik it's done via assertion by you.

klenik (Wed, 15 Apr 2020 14:16:58 GMT):
@nkl199 @shemnon Can't make today's call, sorry. Progress report on my part: reviewing the benchmark repo update PR, then I'll start preparing the release/changes note for v0.3.1 just in case :)

nkl199 (Wed, 15 Apr 2020 14:17:18 GMT):
:thumbsup:

shemnon (Wed, 15 Apr 2020 14:23:08 GMT):
:+1:

shemnon (Wed, 15 Apr 2020 14:23:24 GMT):
I don't have anything to discuss myself

nkl199 (Wed, 15 Apr 2020 15:15:23 GMT):
and it looks like I fell foul of my calendar not alerting me (have i ever mentioned how much I dislike lotus notes?)

MHBauer (Thu, 16 Apr 2020 21:27:01 GMT):
Has left the channel.

benjamincburns (Mon, 20 Apr 2020 21:23:38 GMT):
Has joined the channel.

benjamincburns (Mon, 20 Apr 2020 21:23:57 GMT):
review request: https://github.com/hyperledger/caliper/pull/808

nkl199 (Wed, 22 Apr 2020 07:58:32 GMT):
Maintainers meeting topics: - Are we ready for a new release? - Pending the above, what should be targeted for the next release cycle? - Is it time for some tutorials?

klenik (Wed, 22 Apr 2020 16:44:43 GMT):
Sorry for missing the call. My quick take on the above points: yes, core improvements, yes :D

nkl199 (Wed, 29 Apr 2020 13:51:01 GMT):
Topics: - caliper PRs to merge - when to release? - Alternative report options (not html)

benjamincburns (Mon, 04 May 2020 21:59:33 GMT):
@nkl199 can I please request an invite to the maintainer's meeting? I've more or less taken over the work that @shemnon was doing previously. Although if he's presently on the invite list, I think he should stay

benjamincburns (Mon, 04 May 2020 22:01:41 GMT):
In other news - by any chance was there a recent change that would've broke/added command line arguments? Up until recently I've been running a master process with the command `caliper launch master --caliper-benchconfig config/benchmark.yaml --caliper-networkconfig config/networkconfig.json --caliper-workspace config/`, and on the latest `HEAD` of `master` it errors and prints usage text

benjamincburns (Mon, 04 May 2020 22:01:58 GMT):
no worries if this is expected - just figured I'd ask here before raising an issue

rjones (Tue, 05 May 2020 00:36:53 GMT):
it looks like `burrow` was changed? https://github.com/hyperledger/caliper/commit/2c55b6d5da7f75cadb3a32776747bb820135ac5f

rjones (Tue, 05 May 2020 00:37:48 GMT):
@benjamincburns here is the calendar: https://wiki.hyperledger.org/display/HYP/Calendar+of+Public+Meetings

rjones (Tue, 05 May 2020 00:38:16 GMT):
it looks like Caliper is Wednesday at 1500

rjones (Tue, 05 May 2020 00:39:07 GMT):
(0800 Pacific)

benjamincburns (Tue, 05 May 2020 01:13:57 GMT):
Thanks @rjones!

nkl199 (Tue, 05 May 2020 08:25:15 GMT):
no changes that should imapact that command, we did have an issue with yargs, where a version change was required to permit the command to run correctly. Have you recently performed a repoclean/bootstrap?

benjamincburns (Tue, 05 May 2020 08:27:33 GMT):
Yes - every benchmark I run is provisioned on a fresh VM by terraform. As a workaround for now I just set it to use the merge commit from my most recent PR. I'll try again tomorrow and file an issue (and likely a PR) if the problem persists.

kiview (Tue, 05 May 2020 15:56:16 GMT):
Has joined the channel.

benjamincburns (Tue, 05 May 2020 20:52:01 GMT):
> Error while performing "init" step: TypeError: this.provider.send is not a function looks like something changed w.r.t. the web3 dependency

benjamincburns (Tue, 05 May 2020 20:54:09 GMT):
https://github.com/hyperledger/caliper/blame/master/packages/caliper-ethereum/package.json#L29

benjamincburns (Tue, 05 May 2020 20:54:15 GMT):
That'd be the problem

benjamincburns (Tue, 05 May 2020 21:01:50 GMT):
review request: https://github.com/hyperledger/caliper/pull/825 - ethereum adapter is broken until merged :-(

kiview (Wed, 06 May 2020 05:53:38 GMT):
Although not a maintainer, I commented :) Makes me wonder, missing CI for this?

klenik (Wed, 06 May 2020 13:00:02 GMT):
The CI performs an explicit binding step, which basically install some runtime deps. And the deps in the binding specification remained correct, so the above bug was a dev-time bug fortunately/hopefully.

klenik (Wed, 06 May 2020 13:52:53 GMT):
Prepared the version bump PR and release draft for v0.3.1 https://github.com/hyperledger/caliper/pull/827 https://github.com/hyperledger/caliper/releases/tag/untagged-867c111738e9237b0582

nkl199 (Wed, 06 May 2020 14:55:37 GMT):
wohoo!!

klenik (Wed, 06 May 2020 14:58:58 GMT):
I've also started organizing the 0.4.0 milestone: https://github.com/hyperledger/caliper/milestone/4 The general topics are: consistent logging, OO workload modules, config validations, and the big refactor idea I presented some time ago (pptx in the wiki dev: https://wiki.hyperledger.org/pages/viewpageattachments.action?pageId=9110170&metadataLink=true)

nkl199 (Wed, 06 May 2020 15:03:37 GMT):
am i currently late for a meeting

nkl199 (Wed, 06 May 2020 15:03:39 GMT):
?

klenik (Wed, 06 May 2020 15:03:49 GMT):
Yes :D

benjamincburns (Wed, 06 May 2020 17:35:42 GMT):
review request (paging @klenik, as we discussed this change on the contributor call): https://github.com/hyperledger/caliper/pull/829

benjamincburns (Wed, 06 May 2020 17:47:11 GMT):
Note: CI failed for that PR, however the test failure was in `caliper-core` in tests that check prometheus monitoring. I've double checked `caliper-core` to be certain that there's no way that my PR could have broken those test. I'd try rerunning the CI build, but I don't have rights to do so.

rjones (Wed, 06 May 2020 17:48:08 GMT):
@benjamincburns I clicked the "do it again" button

benjamincburns (Wed, 06 May 2020 17:48:20 GMT):
thanks

benjamincburns (Wed, 06 May 2020 17:50:20 GMT):
On a related topic - is there any reason for the block quote formatting on this page? As a reader it doesn't seem to call out anything specific to me. https://github.com/hyperledger/caliper/blob/gh-pages/docs/vNext/Ethereum_Configuration.md

benjamincburns (Wed, 06 May 2020 17:52:36 GMT):
Looks like some of it came from Antonio Russon (not sure what his username is here, sorry), and some came from @shemnon

benjamincburns (Wed, 06 May 2020 17:52:36 GMT):
Looks like some of it came from Antonio Russo (not sure what his username is here, sorry), and some came from @shemnon

benjamincburns (Wed, 06 May 2020 17:53:26 GMT):
@russanto :point_up:

russanto (Wed, 06 May 2020 18:08:00 GMT):
Hi, do you mean same specific part ?

benjamincburns (Wed, 06 May 2020 18:08:50 GMT):
I mean the paragraphs starting with the `>` character (e.g. the very first paragraph of the document)

benjamincburns (Wed, 06 May 2020 18:09:30 GMT):
I'm updating that page some and I wanted to understand the reason for the formatting choice so that I can either make my changes consistent with it, or remove the block quote formatting.

russanto (Wed, 06 May 2020 18:19:02 GMT):
On my side I only tried to consistent with Fabric configuration pages that reports compatibility and feature summary in that way. No reason beyond that.

benjamincburns (Wed, 06 May 2020 18:19:30 GMT):
OK, thanks. For now I've just reordered things a bit, but left the formatting as-is

benjamincburns (Wed, 06 May 2020 18:22:01 GMT):
Another review request, please: https://github.com/hyperledger/caliper/pull/830

benjamincburns (Wed, 06 May 2020 18:49:22 GMT):
@klenik per our call today, here is the issue I promised to submit regarding decoupling of contract configuration data from the network config for the ethereum adapter: https://github.com/hyperledger/caliper/issues/831

klenik (Wed, 13 May 2020 11:46:40 GMT):
@benjamincburns @nkl199 I added my 5 cents to the above issue, heading towards a general solution. Unfortunately, I can't attend today's call, but feel free to discuss the pros/cons, because it would be a nice enhancement

jramps9 (Wed, 13 May 2020 12:11:37 GMT):
Hello Caliper contributors! Reminder to please join the DevRel Marketing Committee call at 9am PT TODAY, 5/13! Take a look at the agenda and add items here: https://wiki.hyperledger.org/display/Marketing/2020-05-13+Meeting+notes

nkl199 (Wed, 13 May 2020 15:29:09 GMT):
Was a rather quiet meeting today :joy: - Also keen to see the separation of network and contract information, it is a natural partition and should lead to easier use of Caliper

benjamincburns (Wed, 13 May 2020 16:48:17 GMT):
Apologies for missing the meeting today - I forgot to make my calendar entry recurring last time :-(

nkl199 (Thu, 14 May 2020 11:20:06 GMT):
no worries, I'm notorious for missing them - my calendar has decided to no longer alert me to meetings ... which is somewhat frustrating!

nkl199 (Thu, 14 May 2020 11:22:11 GMT):
I think we could/should start grouping the breaking changes for the next release, based on the above, the onces we should try to get into the same release are: - breaking out smart contracts to a separate file - enabling different report formats - modify "backlog" controllers to permit nomination of back total on the SUT, not per worker (ie align with TPS drivers) - make callbacks object oriented

klenik (Wed, 20 May 2020 12:13:58 GMT):
I think the above points make a nice agenda for today

nkl199 (Wed, 20 May 2020 12:16:29 GMT):
cool - maybe add a "worker lifecycle" when using MQTT?

klenik (Wed, 20 May 2020 14:57:31 GMT):
I might be 2-3 minutes late :sweat_smile:

nkl199 (Wed, 20 May 2020 15:03:01 GMT):
righto :thumbup:

benjamincburns (Wed, 20 May 2020 15:25:39 GMT):
re: enabling different report formats - I recommend Jinja2

nkl199 (Wed, 20 May 2020 15:26:46 GMT):
righto - currently usines Mustache

nkl199 (Wed, 20 May 2020 15:26:46 GMT):
righto - currently usies Mustache

nkl199 (Wed, 20 May 2020 15:26:46 GMT):
righto - currently uses Mustache

benjamincburns (Wed, 20 May 2020 15:27:33 GMT):
Mustache is good, too - just something that has support for rendering templates based on a well known templating language

nkl199 (Thu, 21 May 2020 07:54:26 GMT):
extended the description for the callback->class issue https://github.com/hyperledger/caliper/issues/576

klenik (Thu, 21 May 2020 10:43:55 GMT):
Although this raises the question of "cross-platform" workload modules. The TX arguments are SUT-specific. A base class for each SUT might be overkill. A possible solution would be if the SUT adapters provide a general TX builder API, and assemble the correct object under the hood. But coming up with a common format for the common interaction (target contract and arguments, invoker, target nodes) might be a challenge.

nkl199 (Thu, 21 May 2020 11:26:39 GMT):
that's fair - though I think most (if not all) things passed into the base blockchain class `invokeSmartContract` (or query) can be supplied at the highest level ... we consume `(context, contractID, contractVer, args, timeout)` for these, my thoughts were that: - a single base class is to be used for all SUT txn driving - context, contractID and maybe version be provided at benchmark config level - investigate if all adaptors result in a single function being called within the smart contract, and if so have that provided via benchmark file too - the user is always responsible for correctly formulating adaptor specific args I suspect super users would override the base class invoke/query route :shrug:

nkl199 (Thu, 21 May 2020 11:26:39 GMT):
that's fair - though I think most (if not all) things passed into the base blockchain class `invokeSmartContract` (or query) can be supplied at the highest level ... we consume `(context, contractID, contractVer, args, timeout)` for these, my thoughts were that: - a single base class is to be used for all SUT txn driving - context, contractID and maybe version be provided at benchmark config level - investigate if all adaptors result in a single function being called within the smart contract, and if so have that provided via benchmark file too - the user is always responsible for correctly formulating adaptor specific args (as they do currently) I suspect super users would override the base class invoke/query route :shrug:

nkl199 (Thu, 21 May 2020 11:27:39 GMT):
looks like the base class would be rather thin

covepark (Sat, 23 May 2020 14:33:57 GMT):
Has joined the channel.

covepark (Sat, 23 May 2020 14:33:59 GMT):
hello. I have set my own fabric network through VMs(2org 4peer 1orderer). however, I don't know how to measure fabric network with multi-channel. I wanna set multi-channel which is executed simultaneously or alternately by caliper. how can I write config.yaml or fabric-go.yaml file? or Is there any file that I have to modify? Thanks.

qwertlimframe (Tue, 26 May 2020 08:43:58 GMT):
Has joined the channel.

qwertlimframe (Tue, 26 May 2020 08:45:57 GMT):
How can i run caliper on byfn?

klenik (Wed, 27 May 2020 09:44:26 GMT):
@nkl199 @benjamincburns Can't make today's call, crazy busy week. But I started working on the workload module refactor, should be ready by next week.

benjamincburns (Wed, 27 May 2020 16:12:02 GMT):
Apologies, I didn't make it either. Unfortunately I've been tied up with other things and I haven't had any time to focus on caliper-related tasks this week. I'm not sure if I'll have much time over the next week to do much, either.

nkl199 (Wed, 03 Jun 2020 13:52:20 GMT):
are people able to attend today?

klenik (Wed, 03 Jun 2020 14:02:57 GMT):
Yep, I have a PoC for the workload module refractor. Although I also have some design decision questions :)

klenik (Thu, 04 Jun 2020 23:17:47 GMT):
Had some thoughts about our goal to make workload modules less painful, and came up with the following steps to achieve it: 1. Converting the modules to classes is almost done (https://github.com/hyperledger/caliper/pull/856) 2. A bit of config restructuring to pave the road for the next enhancements (https://github.com/hyperledger/caliper/issues/857) 3. Introducing TX builders to make the workloads more or less SUT agnostic (https://github.com/hyperledger/caliper/issues/858) 4. Implement a smart base class for workloads that takes advantage of the builder pattern (https://github.com/hyperledger/caliper/issues/859) Any remarks are appreciated :)

nkl199 (Tue, 09 Jun 2020 15:10:43 GMT):
Suggested agenda for tomorrow: - Current issues that warrant a 0.3.2 release - Do we want to start having release branches? Or maybe feature branches that are merged to master when complete? - Docs and the versioning of releases: do we want a drop down for every release, or only for the highest minor version of each release? (1.0 | 1.1 | 1.2 | 1.3.2 | vNext) in this case - Current PR backlog - Shall we rename `caliper-client-local` to be `caliper-worker` ?

nkl199 (Tue, 09 Jun 2020 15:10:43 GMT):
Suggested agenda for tomorrow: - Current issues that warrant a 0.3.2 release - Do we want to start having release branches? Or maybe feature branches that are merged to master when complete? - Docs and the versioning of releases: do we want a drop down for every release, or only for the highest minor version of each release? (0.1 | 0.2 | 0.3.2 | vNext) in this case - Current PR backlog - Shall we rename `caliper-client-local` to be `caliper-worker` ?

nkl199 (Tue, 09 Jun 2020 15:10:43 GMT):
Suggested agenda for tomorrow: - Current issues that warrant a 0.3.2 release - Do we want to start having release branches? Or maybe feature branches that are merged to master when complete? - Docs and the versioning of releases: do we want a drop down for every release, or only for the highest minor version of each release? (0.1 | 0.2 | 0.3.2 | vNext) in this case - Current PR backlog - Shall we rename `caliper-client-local` to be `caliper-worker` ? - Binding: should we only be binding at the semver minor release level, and not patch releases? (use of a config file can give any binding)

jramps9 (Tue, 09 Jun 2020 15:52:21 GMT):
Hi Caliper contributors! Reminder the Marketing Committee-Dev Relations call is tomorrow, 6/10 at 9am PT. Hope to see you there! https://wiki.hyperledger.org/display/Marketing/2020-06-010+Meeting+notes

nkl199 (Wed, 10 Jun 2020 11:02:12 GMT):
Not sure what I just managed to do to the suggested agenda for today, so here it is again: Suggested agenda for tomorrow: - Current issues that warrant a 0.3.2 release - Do we want to start having release branches? Or maybe feature branches that are merged to master when complete? - Docs and the versioning of releases: do we want a drop down for every release, or only for the highest minor version of each release? (0.1 | 0.2 | 0.3.2 | vNext) in this case - Current PR backlog - Shall we rename caliper-client-local to be caliper-worker ? - Binding: should we only be binding at the semver minor release level, and not patch releases? (use of a config file can give any binding)

nkl199 (Wed, 10 Jun 2020 15:30:37 GMT):
Minutes of the above: - items for inclusion of the 0.3.2 release agreed, will include current non-breaking PRs plus the 1.4.8 fabric binding - yes, we should start looking at release branches sooner rather than later, so that we can backport patches to prior releases while we work on significant breaking changes on a different branch - docs to be versioned at the highest patch semver level - caliper-client-local to renamed inline with the orchestrators to be `round-executor` - binding options to be passed to the community for opinions before we move to a specific binding option strategy

nkl199 (Thu, 11 Jun 2020 13:52:44 GMT):
:thinking: so what is the process to go to a v1.0.0 release?

klenik (Thu, 11 Jun 2020 14:14:11 GMT):
Hmm, I think the tsc wiki has/had some info about this.

nkl199 (Thu, 11 Jun 2020 14:15:20 GMT):
OK, i'll have a peek. I *think* (or is it feel?) that the 0.4.x release could be a viable 1.0 candidate

nkl199 (Thu, 11 Jun 2020 14:15:47 GMT):
I certainly think we're approaching it

klenik (Thu, 11 Jun 2020 14:18:59 GMT):
Yeah, we're definitely getting closer. Regarding the core functionalities at least. The adapters are a different question though.

rjones (Thu, 11 Jun 2020 14:20:41 GMT):
https://hyperledger.github.io/tsc/project-incubation-exit.html

rjones (Thu, 11 Jun 2020 14:21:04 GMT):
https://hyperledger.github.io/tsc/project-lifecycle.html#promoted-release

nkl199 (Thu, 11 Jun 2020 14:21:36 GMT):
yh, agree there with regards to the adaptors

rjones (Sat, 13 Jun 2020 12:21:32 GMT):
https://lfanalytics.io/projects/hyperledger%2Fcaliper/dashboard

klenik (Tue, 16 Jun 2020 14:03:06 GMT):
Proposed agenda items for tomorrow (feel free to add to the list): - Naming conventions/ideas for the new workload module updates (https://github.com/hyperledger/caliper/issues/857 and https://github.com/hyperledger/caliper/issues/859) - TX builder API details (https://github.com/hyperledger/caliper/issues/858)

Adhavpavan (Sat, 20 Jun 2020 09:57:24 GMT):
Has joined the channel.

Adhavpavan (Sat, 20 Jun 2020 10:06:33 GMT):
Hello, I am getting error as below: Caliper currently only supports gateway based operation using the 2.1.0 Fabric-SDK. Please retry with the gateway flag Where I have to provide gateway flag? COnfiguration: services: caliper: container_name: caliper image: hyperledger/caliper:0.3.1 command: launch master environment: - CALIPER_BIND_SUT=fabric:2.1.0 - CALIPER_BENCHCONFIG=benchmarks/scenario/simple/pavan/config.yaml - CALIPER_NETWORKCONFIG=networks/fabric/pavan/network-config.yaml volumes: - ./caliper-benchmarks:/hyperledger/caliper/workspace network_mode: host Please let me know for more information. Thanks in advance.

NamanGupta1003 (Mon, 22 Jun 2020 07:42:40 GMT):
Has joined the channel.

NamanGupta1003 (Mon, 22 Jun 2020 07:43:20 GMT):
Hey, i was setting up caliper using the docker image and run the following command: user@ubuntu:~/caliper-benchmarks$ docker run \ -v $PWD:/hyperledger/caliper/workspace \ -e CALIPER_BIND_SUT=fabric:1.4.0 \ -e CALIPER_BENCHCONFIG=benchmarks/scenario/simple/config.yaml \ -e CALIPER_NETWORKCONFIG=networks/fabric/fabric-v1.4.1/2org1peergoleveldb/fabric-go.yaml \ --name caliper hyperledger/caliper:0.3.0 launch master I got the following error: 2020-06-20T14:20:29.617Z - error: [FabricCAClientService.js]: Failed to enroll admin, error:%o message=Calling enrollment endpoint failed with error [Error: connect ECONNREFUSED 127.0.0.1:7054], stack=Error: Calling enrollment endpoint failed with error [Error: connect ECONNREFUSED 127.0.0.1:7054] at ClientRequest.request.on (/home/node/.npm-global/lib/node_modules/fabric-ca-client/lib/FabricCAClient.js:487:12) at ClientRequest.emit (events.js:198:13) at Socket.socketErrorListener (_http_client.js:392:9) at Socket.emit (events.js:198:13) at emitErrorNT (internal/streams/destroy.js:91:8) at emitErrorAndCloseNT (internal/streams/destroy.js:59:3) at process._tickCallback (internal/process/next_tick.js:63:19) 2020.06.20-14:20:29.618 error [caliper] [caliper-engine] Error while performing "init" step: Error: Couldn't enroll Org1's registrar or set it as user context: Calling enrollment endpoint failed with error [Error: connect ECONNREFUSED 127.0.0.1:7054] 2020.06.20-14:20:29.618 info [caliper] [caliper-engine] Executed "init" step in 0.406 seconds 2020.06.20-14:20:29.618 info [caliper] [caliper-utils] Executing command: cd /hyperledger/caliper/workspace; docker-compose -f networks/fabric/docker-compose/2org1peergoleveldb/docker-compose.yaml down;(test -z \"$(docker ps -aq)\") || docker rm $(docker ps -aq);(test -z \"$(docker images dev -q)\") || docker rmi $(docker images dev -q);rm -rf /tmp/hfc-* /bin/sh: docker-compose: not found /bin/sh: docker: not found /bin/sh: docker: not found /bin/sh: docker: not found /bin/sh: docker: not found /bin/sh: docker: not found /bin/sh: docker: not found 2020.06.20-14:20:29.625 info [caliper] [caliper-engine] Executed end command in 0.007 seconds 2020.06.20-14:20:29.626 error [caliper] [cli-launch-master] Benchmark failed with error code 4 Usage: caliper launch master --caliper-bind-sut fabric:1.4.1 [other options] Options: --help, -h Show usage information [boolean] --version Show version information [boolean] --caliper-bind-sut The name and version of the platform to bind to [string] --caliper-bind-cwd The working directory for performing the SDK install [string] --caliper-bind-args Additional arguments to pass to "npm install". Use the "=" notation when setting this parameter [string] --caliper-bind-file Yaml file to override default (supported) package versions when binding an SDK [string] Error: Benchmark failed with error code 4 at Function.handler (/home/node/.npm-global/lib/node_modules/@hyperledger/caliper-cli/lib/launch/lib/launchMaster.js:70:23) at process._tickCallback (internal/process/next_tick.js:68:7) Can someone please help?

nkl199 (Mon, 22 Jun 2020 08:06:08 GMT):
replied in caliper channel

rjones (Mon, 22 Jun 2020 17:54:36 GMT):
Has left the channel.

klenik (Wed, 24 Jun 2020 09:35:26 GMT):
@nkl199 Do you have some agenda items in mind for today? For my part, I'm working on the workload module config refactor (https://github.com/hyperledger/caliper/issues/857), so not much to report yet :)

nkl199 (Wed, 24 Jun 2020 09:36:14 GMT):
hmm, currently working on the docs mod, but maybe we could prioritise a few items?

klenik (Wed, 24 Jun 2020 09:38:58 GMT):
Sure. These are the current v0.4.0 items: https://github.com/hyperledger/caliper/milestone/4 Good basis for discussion

NamanGupta1003 (Fri, 26 Jun 2020 14:36:44 GMT):
Hey this error just doesn't seem to go away. I started a network using "./startFabric.sh javascript" in fabcar in fabric-samples. It used fabric version 2.1.0. I mapped the credentials in peerOrganizations and ordererOrganizations. I used docker-compose to run caliper and got the following error caliper | 2020.06.26-14:15:31.398 info [caliper] [adapters/fabric] No admin.Org1 found in caliper configuration file - unable to perform admin options caliper | 2020.06.26-14:15:31.398 info [caliper] [adapters/fabric] No admin.Org2 found in caliper configuration file - unable to perform admin options caliper | 2020-06-26T14:15:31.749Z - error: [FabricCAClientService.js]: Failed to enroll admin, error:%o message=Calling enrollment endpoint failed with error [Error: connect ECONNREFUSED 127.0.0.1:7054], stack=Error: Calling enrollment endpoint failed with error [Error: connect ECONNREFUSED 127.0.0.1:7054] caliper | at ClientRequest.request.on (/home/node/.npm-global/lib/node_modules/fabric-ca-client/lib/FabricCAClient.js:484:12) caliper | at ClientRequest.emit (events.js:198:13) caliper | at TLSSocket.socketErrorListener (_http_client.js:392:9) caliper | at TLSSocket.emit (events.js:198:13) caliper | at emitErrorNT (internal/streams/destroy.js:91:8) caliper | at emitErrorAndCloseNT (internal/streams/destroy.js:59:3) caliper | at process._tickCallback (internal/process/next_tick.js:63:19) Can someone pls help?

klenik (Mon, 29 Jun 2020 14:09:34 GMT):
The current release-cycle aims (among other things) to make the workload modules easier to code. The new, cleaner design also affects the API of the SUT connectors, hopefully in a good way :) I opened an issue for the proposed connector API changes, feel free to add any thoughts and suggestions: https://github.com/hyperledger/caliper/issues/888

nkl199 (Mon, 29 Jun 2020 14:17:18 GMT):
added my thoughts :thumbup:

klenik (Wed, 01 Jul 2020 09:59:02 GMT):
I can't attend today's call. As a progress update, I'm working on this issue: https://github.com/hyperledger/caliper/issues/888 Hopefully this will result in a clean, life-cycle aware and event-driven API for the connectors.

wangdong (Fri, 03 Jul 2020 05:38:58 GMT):
Has left the channel.

nkl199 (Thu, 09 Jul 2020 17:11:58 GMT):
hmmm, fabric 2.2 is out it would seem

klenik (Thu, 09 Jul 2020 18:14:33 GMT):
As I see it's mostly bug fixes, no notable feature from our point of view (kind of relieved)

nkl199 (Thu, 09 Jul 2020 18:14:43 GMT):
yup

nkl199 (Thu, 09 Jul 2020 18:15:12 GMT):
i'm waiting for them to pull the "old chaincode lifecycle" within a patch release tho :/

nkl199 (Thu, 09 Jul 2020 18:15:44 GMT):
they have nailed all the 1.4.x networks tho

nkl199 (Thu, 09 Jul 2020 18:16:32 GMT):
well, set themselves up to fail there it would seem - the chaincode build in the 1.4 imagesalways pulls ccenv:latest, and not the build it needs

nkl199 (Thu, 09 Jul 2020 18:16:32 GMT):
well, set themselves up to fail there it would seem - the chaincode build in the 1.4 images always pulls ccenv:latest, and not the build it needs

nkl199 (Thu, 09 Jul 2020 18:16:47 GMT):
shame that `latest` is now for 2.2

nkl199 (Thu, 09 Jul 2020 18:16:54 GMT):
:facepalm:

klenik (Thu, 09 Jul 2020 18:22:11 GMT):
Yeah, and even if the 1.4 image is present, the 2.x is used. I had to clean up my test env a few times...

klenik (Thu, 09 Jul 2020 18:22:11 GMT):
(that's why I actively oppose the latest tag for the caliper image, it's a big antipattern)

davidkel (Wed, 15 Jul 2020 15:20:07 GMT):
Has joined the channel.

abhi201191 (Mon, 20 Jul 2020 15:22:59 GMT):
Has joined the channel.

abhi201191 (Mon, 20 Jul 2020 15:22:59 GMT):
Hi.. when I try to run ./publish.js verdaccio start terminal gets stucked at PM2_HOME=.pm2 npx pm2 start verdaccio -- -l ${BIND} -c artifacts/verdaccio-config.yaml with this rollbackfailedoptional verb npm-session. Please help me out

abhi201191 (Mon, 20 Jul 2020 15:22:59 GMT):
Hi.. when I try to run `./publish.js verdaccio start` terminal gets stucked at `PM2_HOME=.pm2 npx pm2 start verdaccio -- -l ${BIND} -c artifacts/verdaccio-config.yaml` with this `rollbackfailedoptional verb npm-session`. Please help me out

klenik (Tue, 21 Jul 2020 16:36:17 GMT):
I'm on vacation(ish) from tomorrow, so can't attend the call

davidkel (Tue, 21 Jul 2020 17:28:59 GMT):
No worries, Nick is on vacation as well

lehors (Wed, 22 Jul 2020 19:51:03 GMT):
Has joined the channel.

lehors (Wed, 22 Jul 2020 19:51:03 GMT):
Hi there

lehors (Wed, 22 Jul 2020 19:51:39 GMT):
The quarterly report to the TSC for *Q2* is overdue, could someone please submit one?

lehors (Wed, 22 Jul 2020 19:52:29 GMT):
The one for Q1 is here: https://wiki.hyperledger.org/display/TSC/2020+Q1+Hyperledger+Caliper

lehors (Wed, 22 Jul 2020 19:53:02 GMT):
any maintainer should be able to make a copy of that and update it as needed

lehors (Wed, 22 Jul 2020 19:53:16 GMT):
thanks

klenik (Wed, 22 Jul 2020 20:00:04 GMT):
Yeah, I'll submit it tomorrow, sorry for the delay

lehors (Wed, 22 Jul 2020 20:00:10 GMT):
thanks

klenik (Thu, 23 Jul 2020 09:59:41 GMT):
@lehors Submitted the update: https://wiki.hyperledger.org/display/TSC/2020+Q2+Hyperledger+Caliper

lehors (Thu, 23 Jul 2020 14:02:10 GMT):
👍

klenik (Wed, 29 Jul 2020 12:49:42 GMT):
From my part, I'll just give a quick update on the connector API refactoring on the call

nkl199 (Wed, 29 Jul 2020 13:14:19 GMT):
Other items: - I can give an update on the txObservers refactor - Rosie is to give an update on perf testing "fabric test network" with the aim that we can make it easier to connect and test - Dave is to propose a modification to the network file for fabric connectors (if unavailable, i can give a high level summary)

nkl199 (Thu, 30 Jul 2020 10:27:49 GMT):
rate controller options convention ... we have a mix of camel case and underscore separation for option names ... shall we vote on a single choice?

klenik (Thu, 30 Jul 2020 10:36:32 GMT):
For attributes, I prefer camel casing (underscores are sooo python :D ). We also have mixes in file names, for which I have no preference, but I usually go with camel casing, to almost match the class names in the module.

davidkel (Fri, 31 Jul 2020 12:58:23 GMT):
I've been thinking about how to restructure the network-config for the fabric connector and the way I'm thinking is to make the network-config org specific. ie a single network-config file is representative of a single organisation. My thinking behind this is because 1. the plan is to have a network config reference a connection profile which will be org specific 2. identities will be org specific (but in the network config you will give it an alias that can doesn't need to be so workloads can reference that alias) 3. organisations are only really interested in the performance from their org point of view 4. if you want to test a network for multiple orgs then from a real world point of view you would have 2 distinct orgs interacting with the network currently via 2 different caliper launches.

davidkel (Fri, 31 Jul 2020 12:58:23 GMT):
I've been thinking about how to restructure the network-config for the fabric connector and the way I'm thinking is to make the network-config org specific. ie a single network-config file is representative of a single organisation. My thinking behind this is because 1. the plan is to have a network config reference a connection profile which will be org specific 2. identities will be org specific (but in the network config you will give it an alias that doesn't need to be so workloads can reference that alias) 3. organisations are only really interested in the performance from their org point of view 4. if you want to test a network for multiple orgs then from a real world point of view you would have 2 distinct orgs interacting with the network currently via 2 different caliper launches. I was wondering what people think about this approach ?

nkl199 (Fri, 31 Jul 2020 12:59:07 GMT):
:point_up_2: @klenik

klenik (Fri, 31 Jul 2020 13:24:46 GMT):
That makes a lot of sense, and raises further questions, tasks and things to consider (because why would life be easy) : a) I'd really like to keep the single networkconfig file approach (to be consistent across connectors), but in that file we can reference one or more org-specific connection profiles, just as you described. b) I partially agree with point 3., but we can't just ignore the other players (and their workload) on the network, since they consume the same resources. Hence the next, very interesting point. c) What you describe in point 4 is (kind of) the "blockchainification" of benchmarking itself, something like "privacy preserving distributed benchmarking". c-1) Everyone plugs in their own private workload logic, uses the network at the same time, and see the aggregated results in the form of a report. c-2) Hiding the workload logic is almost possible, since orgs can host their own workers, just a central (or more complicated) message broker is needed. The workers (can) have separated config files, but currently the round configs are determined by the manager (this would be easy to fix, and very useful). So independent, org-hosted workers are not far away. c-3) Analyzing the result is also interesting in this case. The manager only provides an aggregated report, which is kind of boring. But with the coming worker-scoped TX Monitors, you can do anything with your own TX results, so org-specific analysis is possible. It was a bit long, but I think we need to consider this in real enterprise scenarios. So I await your inputs, since you're the enterprise people :grin:

nkl199 (Fri, 31 Jul 2020 14:38:23 GMT):
just trying to dig a bit into c-x above. I think (correct me if i'm wrong) Dave is hinting towards having Fabric network config files restricted to a "per org" scenario. So org1 would (in an enterprise scenario) run a scheduled load test and see what they achieve, orgX would do similar at a different time and if they wanted they could compare/share results. In these cases, orgs would potentially have some form of overlap in resources, but org1 might well have way more resources than orgX - so their results would certainly vary. I dont think the suggestion is to merge concurrent caliper runs at completion (or at the very least, shotgun not me!) At the moment caliper can pick from any client for an org within the profile, so there is a breaking change in behaviour by moving to a single org focus, but i'm keen to explore what we would lose here .. thoughts? I can only think of asset manipulation from cross organizational actors, such as Org1 creating something based on agreements/contributions from Orgs2 and 3. That itself is edging towards lifecyle performance tests, which in general are a bit tricky to encompass in a workload module

nkl199 (Fri, 31 Jul 2020 14:39:58 GMT):
I'll drop the caveat that my current experience has been people interested from within their own organization, so am keen to explore potential cross-org scenarios

nkl199 (Fri, 31 Jul 2020 14:39:58 GMT):
I'll drop the caveat that my current experience has been based on people interested from within their own organization, so am keen to explore potential cross-org scenarios

davidkel (Fri, 31 Jul 2020 15:18:48 GMT):
@klenik thanks for the reply, so how about this for an example of a proposal ? It allows you to do both but doesn't add too much overhead if you want to separate orgs into different files ``` --- name: Fabric version: "1.0" mutual-tls: true caliper: blockchain: fabric channels: mychannel: created: true discover: true contracts: - id: marbles contractID: mymarbles version: v0 organizations: - mspid: org1MSP identities: # only used for fabric connector not gateway (can also be defined in connection profile) credentialStore: path: /tmp/hfc-kvs/org1 cryptoStore: path: /tmp/hfc-cvs/org1 # assume that entries in wallet are all of the same org # only for use by gateway wallet: path: 'some/path/to/org-specific-wallet' hsm: library: 'path/to/pkcs11library' slot: 3 pin: 123456789 usertype: 1 localMSP: alias: 'aUser' path: 'some/path/to/org-specific-localMSP' certificates: - alias: 'User1' clientPrivateKey: # can use PEM as well path: ../config/crypto-config/peerOrganizations/org1.example.com/users/User1@org1.example.com/msp/keystore/key.pem clientSignedCert: path: ../config/crypto-config/peerOrganizations/org1.example.com/users/User1@org1.example.com/msp/signcerts/User1@org1.example.com-cert.pem # auto register and enroll to come.... # could validate this connection profile matches declared mspid connectionProfile: 'some/path/to/org-specific-profile' - mspid: org2MSP # and so forth ```

klenik (Fri, 31 Jul 2020 15:23:16 GMT):
Can't really see the high-level structure from my phone, but I think I get the concept, and it looks good. Btw, since workers can pick up different network configs, the single-org config could also work. But it would complicate smaller experiments (and centralized benchmarks). I'll give the above remarks some thought, maybe we should come up with some scenarios in figure format, to helps us see the artifact constraints

nkl199 (Tue, 04 Aug 2020 11:06:23 GMT):
@klenik -any more thoughts on the above?

klenik (Tue, 04 Aug 2020 11:29:07 GMT):
As I see it's functionally equivalent (but more organized) to the current network config, so I fully support it :thumbsup:

klenik (Tue, 04 Aug 2020 11:30:12 GMT):
And hopefully we can delegate most of the validation to the SDK :grin:

klenik (Wed, 05 Aug 2020 13:05:29 GMT):
@nkl199 Not much tangible progress from me (I've started reviewing the TX observers PR, and went through the Tutorial PR), so I'll leave the agenda for you and the others :)

nkl199 (Wed, 05 Aug 2020 13:59:09 GMT):
I have another meeting to attend at half past the hour, so i'm hoping it will be a short meeting!

nkl199 (Wed, 05 Aug 2020 14:56:10 GMT):
agenda items: - progress on network config file change in fabric connector - coding standards ... shall we formalise these? ie stricter eslinting, variable names, etc - work in progress

klenik (Wed, 05 Aug 2020 15:08:47 GMT):
I'll be late, sorry

nkl199 (Wed, 05 Aug 2020 15:09:27 GMT):
no worries - stalled a little with "hey look at the shiny things"

nkl199 (Wed, 05 Aug 2020 15:10:14 GMT):
and completely missed the initial "introduce self to others and vice-versa" thing :/

Lynn14m (Thu, 06 Aug 2020 20:09:52 GMT):
Has joined the channel.

klenik (Fri, 07 Aug 2020 13:27:05 GMT):
Did some thinking about loading external modules (since the major components are now pluggable :caliper: ), any remarks are welcome :) https://github.com/hyperledger/caliper/issues/948

nkl199 (Mon, 10 Aug 2020 07:53:25 GMT):
Taht would be handy

nkl199 (Mon, 10 Aug 2020 07:53:25 GMT):
That would be handy, as part of that, I'd be keen to ensure that all the intended pluggable parts follow the same pattern and be explicit in what parts will and will not be pluggable

jramps9 (Tue, 11 Aug 2020 00:08:36 GMT):
Hello Caliper contributors! Reminder to please join the DevRel Marketing Committee call at 9am PT this Weds 8/12. Take a look at the agenda and add items here: https://wiki.hyperledger.org/display/Marketing/2020-08-12+Meeting+notes

klenik (Wed, 19 Aug 2020 11:48:47 GMT):
I'll be traveling this afternoon, so I'll skip the call (very bad wifi on the trains....)

nkl199 (Wed, 19 Aug 2020 15:18:49 GMT):
not much to update - Prs inbound and under review, items we hope to contain in the 0.4.0 release

mzeeshanhaider (Mon, 24 Aug 2020 06:41:56 GMT):
Has joined the channel.

nkl199 (Wed, 26 Aug 2020 13:02:16 GMT):
Suggested agenda topics: - pending PRs - work in flight - is the prometheus monitor config correct/ in need of changing? - 0.4.0 release requirements

nkl199 (Wed, 26 Aug 2020 16:25:01 GMT):
User User_1 added by nkl199.

nkl199 (Thu, 03 Sep 2020 10:05:28 GMT):
I've been looking into the "bump node versions" issue in more depth. It would appear that: - the logical move is to bump to v14 - doing so requires the removal of the sawtooth connector, which uses the sawtooth-sdk that has been untouched for 2yrs, and fails on zeromq We can't really be beholding to an out of date connector, which we have already been forced to remove from the integration tests ... so the suggestion here is that we officially remove it. Thoughts?

klenik (Thu, 03 Sep 2020 11:05:50 GMT):
Well, yeah, node v8/10 is no longer lts, so we should explore the 12/14 lts supports. AFAIK, we're still running fine on 8/10, so I wouldn't necessarily remove those (with the engine restriction). Older GRPC packages had issues with newer node versions, that was resolved with the smart binding args/env based on the node version. So I'm all for exploring the two new versions :thumbsup:

klenik (Thu, 03 Sep 2020 11:10:08 GMT):
https://sawtooth.hyperledger.org/docs/core/releases/1.0/app_developers_guide/sdk_table.html Based on this, the JS SDK is stable, but true, untouched for 1-2 year.

klenik (Thu, 03 Sep 2020 11:11:17 GMT):
Currently, the only stable connectors are: Fabric, Besu/Ethereum and FISCO-BCOS.

nkl199 (Thu, 03 Sep 2020 11:14:12 GMT):
yeah, with their format not being compatible with the current caliper API, at the very least we will have to deprecate sawtooth and iroha, as well as exclude them from publish and the CLI binding knowlege

nkl199 (Thu, 03 Sep 2020 11:19:56 GMT):
there's a question about the bindings too, at the moment we have "quite a few" bindings available for fabric versions, as well as permitting a file based override for "any version combination". This seems to be a bit much now :joy: we could follow the pattern in the besu binding that provides the "most recent legacy release" and the full range of current major release

nkl199 (Thu, 03 Sep 2020 11:21:48 GMT):
So for fabric there would be bindings for [1.0, 1.1, 1.2, 1.3, 1.4, 2.0, 2.1, (2.2.x -> 2.2.y)]

nkl199 (Thu, 03 Sep 2020 11:22:12 GMT):
(or even perhaps cull 1.0 -> 1.4 ?)

nkl199 (Thu, 03 Sep 2020 11:22:12 GMT):
(or even perhaps cull 1.0 -> 1.3 ?)

klenik (Thu, 03 Sep 2020 11:28:36 GMT):
Yeah, I was thinking the same (currently reviewing the install page) :D We could follow the Besu convention. We could provide the latest SDK version of a compatibility range. For example, we have `1.4` in the binding spec, which will install the latest patch, so the user API won't change. I think 1.0, 1.1 and 1.2 are strictly separate, since each introduced a new feature (channel-based eventing, private collections, CDB indices). The next new feature is the gateway mode with 1.4.x.

klenik (Thu, 03 Sep 2020 11:29:27 GMT):
So the question is, do we want to stick to only LTS (or not so old) releases, or support what we can with sensible path handling?

klenik (Thu, 03 Sep 2020 11:29:27 GMT):
So the question is, do we want to stick to only LTS (or not so old) releases, or support what we can with sensible patch handling?

nkl199 (Thu, 03 Sep 2020 11:29:57 GMT):
hmm

nkl199 (Thu, 03 Sep 2020 11:30:44 GMT):
so, in theory, if we move the caliper dev onward with officially supported versions, if someone wanted to test an older version, they could grab a previous release of caliper that had it supported, and make use of the multi-versino docs

nkl199 (Thu, 03 Sep 2020 11:31:23 GMT):
so i think that put me in the support of LTS versions only camp

klenik (Thu, 03 Sep 2020 11:32:19 GMT):
Yeah, in theory yes, but we also introduce awesome, connector-independent features (TX monitors), which would be useful even if we benchmark an old SUT version.

klenik (Thu, 03 Sep 2020 11:32:48 GMT):
What is the implication of the LTS-only support? Just removing stuff from the binding?

nkl199 (Thu, 03 Sep 2020 11:32:53 GMT):
true, that would be lost, unless the explicit binding with older versions was version compatible

nkl199 (Thu, 03 Sep 2020 11:32:53 GMT):
true, that would be lost, unless the explicit file-based binding with older versions was version compatible

nkl199 (Thu, 03 Sep 2020 11:33:35 GMT):
yes - remove from binding and ensure packages are dev-dependancy on latest

nkl199 (Thu, 03 Sep 2020 11:33:48 GMT):
(and update integration test)

klenik (Thu, 03 Sep 2020 11:34:08 GMT):
Fabric SDK 1.4.x is feature complete (can do everything the older releases can), the only exception is 1.0.0, with its peer-based eventing. So we only need 3 versions: 1.0, 1.4.x, 2.x (?)

klenik (Thu, 03 Sep 2020 11:35:57 GMT):
If we look at the version checks in the connector code, the major paths are gateway/low-level, the rest is feature check (private data, etc). So by supporting 1.0, 1.4 and 2, I think we cover everything

klenik (Thu, 03 Sep 2020 11:36:38 GMT):
Was there any API/SDK change between 2.x and 2.y?

nkl199 (Thu, 03 Sep 2020 11:37:08 GMT):
not to my knowledge, certainly nothing that will break the gateway based connector

nkl199 (Thu, 03 Sep 2020 11:38:41 GMT):
aha, and by bumping the fabric-connector package deps to 1.4.11, it seems the grpc error is no more (so can build project on node12)

nkl199 (Thu, 03 Sep 2020 11:39:14 GMT):
v2 sdk uses the grpcjs package, so no more building of native modules

nkl199 (Thu, 03 Sep 2020 11:39:14 GMT):
fabric node v2 sdk uses the grpcjs package, so no more building of native modules

klenik (Thu, 03 Sep 2020 11:42:36 GMT):
cool, so let's keep the most stable SDK versions that can cover the previous versions functionally, and upon every release, we bump these (probably just behind the scene, so the `1.4` binding will point to the latest patch). We could do the usual deprecated play with the binding versions. Now we declare some 1.x versions deprecated, then merge them in the next release.

klenik (Thu, 03 Sep 2020 11:43:17 GMT):
So now we have `2.1` and `2.2`, now we deprecate 2.1, and remove it in the next release

nkl199 (Thu, 03 Sep 2020 11:48:15 GMT):
sounds like a plan :thumbsup:

nkl199 (Thu, 03 Sep 2020 11:48:49 GMT):
we should probably doc that somewhere so that all connectors follow the convention

klenik (Thu, 03 Sep 2020 11:49:46 GMT):
I'm doing some minor misc updates to the docs currently, so I can include this strategy in the binding command section.

nkl199 (Thu, 03 Sep 2020 12:38:10 GMT):
OK, looks like dave has been working on the bindings, so i'll ask him to incorporate the above binding pattern in the options

davidkel (Thu, 03 Sep 2020 12:52:01 GMT):
@klenik @nkl199 why would you want to keep 1.0 ? 1.0 is so old, out of date and not supported and a lot of breaking changes occurred between 1.0 and 1.1 that I'm not sure I can even remember how it works :-)

nkl199 (Thu, 03 Sep 2020 13:05:35 GMT):
i'm not personally aware of people using it, but people tend not to complain until something is no longer there :joy:

klenik (Thu, 03 Sep 2020 13:06:32 GMT):
Something along the following reasoning: https://lists.hyperledger.org/g/fabric/topic/76540833

klenik (Thu, 03 Sep 2020 13:07:42 GMT):
The only extra code it requires is the event registration, so not much of a maintenance overhead (and it definitely won't change :joy: )

nkl199 (Thu, 03 Sep 2020 13:10:40 GMT):
OK, so I think the suggestion is: ``` 1.0: packages: ['grpc@1.10.1', 'fabric-ca-client@1.1.0', 'fabric-client@1.1.0', 'fabric-protos@2.0.0-snapshot.1'] settings: - *new-node-old-grpc 1.4.11: &fabric-latest packages: ['fabric-client@1.4.11', 'fabric-protos@2.0.0', 'fabric-network@1.4.11'] 1.4: packages: ['fabric-client@1.4.11', 'fabric-protos@2.0.0', 'fabric-network@1.4.11'] 2.1.0: packages: ['fabric-common@2.1.0', 'fabric-protos@2.1.0', 'fabric-network@2.1.0', 'fabric-ca-client@2.1.0'] 2.2.0: packages: ['fabric-common@2.2.0', 'fabric-protos@2.2.0', 'fabric-network@2.2.0', 'fabric-ca-client@2.2.0'] 2.2.1: packages: ['fabric-common@2.2.1', 'fabric-protos@2.2.1', 'fabric-network@2.2.1', 'fabric-ca-client@2.2.1'] 2.2.2: &fabric-latest-v2 packages: ['fabric-common@2.2.2', 'fabric-protos@2.2.2', 'fabric-network@2.2.2', 'fabric-ca-client@2.2.2'] 2.2: packages: ['fabric-common@2.2.2', 'fabric-protos@2.2.2', 'fabric-network@2.2.2', 'fabric-ca-client@2.2.2'] latest: *fabric-latest latest-v2: *fabric-latest-v2 ``` ?

nkl199 (Thu, 03 Sep 2020 13:11:07 GMT):
(didn't exactly print well, but hopefully is indicative)

nkl199 (Thu, 03 Sep 2020 13:13:33 GMT):
i will confess to be a little confused how the current 1.0.0 binding actually uses 1.1.0 sdk versions tho :joy:

davidkel (Thu, 03 Sep 2020 13:16:03 GMT):
@klenik so the current caliper definitions trying to use node-sdk 1.1 against a 1.0 fabric I can't imagine working. It would have to be a node-sdk 1.0 release as I'm pretty sure that the protobufs changed a lot between 1.0 and 1.1 (in a breaking way). IIRC There was no concept of chaincode install/instantiate in 1.0 so certain capabilities in caliper just wouldn't work.

klenik (Thu, 03 Sep 2020 13:17:09 GMT):
It still has the old event hub class, so it works. The 1.0 SDK had some serious grpc issues (surprise), that's why the 1.1 is used. But different API parts are invoked

nkl199 (Thu, 03 Sep 2020 13:17:53 GMT):
ah

nkl199 (Thu, 03 Sep 2020 13:17:53 GMT):
hmm

klenik (Thu, 03 Sep 2020 13:18:41 GMT):
but let's run some tests, maybe it broke since then :joy:

nkl199 (Thu, 03 Sep 2020 13:18:43 GMT):
so if we retained the 1.1 binding then, which is compatible with 1.0 fabric?

klenik (Thu, 03 Sep 2020 13:19:27 GMT):
That should also work. I think the `eventUrl` attribute in the networkconfig triggers the "legacy" path

nkl199 (Thu, 03 Sep 2020 13:20:09 GMT):
i think i would prefer that, because then the pattern of "which sdk" is retained for all items

nkl199 (Thu, 03 Sep 2020 13:20:09 GMT):
i think i would prefer that, because then the pattern of "which sdk" is retained for all items, instead of mixing the platform/sdk aspect

nkl199 (Thu, 03 Sep 2020 13:20:58 GMT):
(and a user looks up their required sdk based on target)

klenik (Thu, 03 Sep 2020 13:21:29 GMT):
true, aligns with the "latest SDK over all previous compatible" pattern

klenik (Thu, 03 Sep 2020 13:22:10 GMT):
So what's left is 1.1 (installing 1.1), 1.4 (installing 1.4.11) and 2.2 (installing 2.2.2)?

klenik (Thu, 03 Sep 2020 13:22:25 GMT):
Or we shouldn't trim so many for this release?

nkl199 (Thu, 03 Sep 2020 13:22:34 GMT):
hmm

nkl199 (Thu, 03 Sep 2020 13:23:00 GMT):
i think definitely trim the 1.4 options

klenik (Thu, 03 Sep 2020 13:23:16 GMT):
yeah, that's for sure :joy:

davidkel (Thu, 03 Sep 2020 13:32:51 GMT):
is this then what is wanted ? ``` sut: # Specifies the available SDK bindings and their Node.js-specific install settings fabric: # The name/key of the SDK binding 1.1.0: # Specifies the packages and their versions to install packages: ['grpc@1.10.1', 'fabric-ca-client@1.1.0', 'fabric-client@1.1.0', 'fabric-protos@2.0.0-snapshot.1'] # The settings to apply during binding. The first setting whose versionRegexp matches the node version will be used settings: - *new-node-old-grpc 1.1: # Specifies the packages and their versions to install packages: ['grpc@1.10.1', 'fabric-ca-client@1.1.0', 'fabric-client@1.1.0', 'fabric-protos@2.0.0-snapshot.1'] # The settings to apply during binding. The first setting whose versionRegexp matches the node version will be used settings: - *new-node-old-grpc 1.4.11: &fabric-latest packages: ['fabric-client@1.4.11', 'fabric-protos@2.0.0-snapshot.1', 'fabric-network@1.4.11'] 1.4: packages: ['fabric-client@1.4.11', 'fabric-protos@2.0.0-snapshot.1', 'fabric-network@1.4.11'] 2.1.0: packages: ['fabric-common@2.1.0', 'fabric-protos@2.1.0', 'fabric-network@2.1.0', 'fabric-ca-client@2.1.0'] 2.2.0: packages: ['fabric-common@2.2.0', 'fabric-protos@2.2.0', 'fabric-network@2.2.0', 'fabric-ca-client@2.2.0'] 2.2.1: packages: ['fabric-common@2.2.1', 'fabric-protos@2.2.1', 'fabric-network@2.2.1', 'fabric-ca-client@2.2.1'] 2.2.2: &fabric-latest-v2 packages: ['fabric-common@2.2.2', 'fabric-protos@2.2.2', 'fabric-network@2.2.2', 'fabric-ca-client@2.2.2'] 2.2: packages: ['fabric-common@2.2.2', 'fabric-protos@2.2.2', 'fabric-network@2.2.2', 'fabric-ca-client@2.2.2'] latest: *fabric-latest latest-v2: *fabric-latest-v2 ```

gregdhill (Thu, 03 Sep 2020 13:35:24 GMT):
Has left the channel.

nkl199 (Thu, 03 Sep 2020 13:44:50 GMT):
:wave: bye Greg!

nkl199 (Thu, 03 Sep 2020 13:45:27 GMT):
@davidkel - yes, looks like it to me :thumbsup:

klenik (Thu, 03 Sep 2020 13:47:17 GMT):
yep :thumbsup: Although 1.4 and 2.2 could use `*fabric-latest(-v2)`, so it always points to the latest patch when `latest` is moved

davidkel (Thu, 03 Sep 2020 13:51:28 GMT):
ok, submitted a PR

klenik (Fri, 04 Sep 2020 08:22:28 GMT):
Now that the bindings are cleaned up, the other big question is the "functioning/maintained" connector list. I think the following connectors should be marked "abandoned" with the upcoming release: Burrow, Iroha, Sawtooth (what remains: Besu, Fabric, Ethereum, FISCO-BCOS)

nkl199 (Fri, 04 Sep 2020 08:32:14 GMT):
That sounds reasonable, we should perhaps approach the maintainers of the abandoned connectors to see if they want to reinstate them at some point in the future

klenik (Fri, 04 Sep 2020 08:35:56 GMT):
Yeah, definitely. I guess v0.5.0 will further clean up the surroundings of the connectors (cleaner API, noted best practices). Plus the Fabric connector refactor will come with additional experience we can share. So I think after v0.5.0 the core Caliper code will be stable enough so we can start reaching out to platform experts about the new/renewed connectors.

klenik (Fri, 04 Sep 2020 08:36:51 GMT):
Until then I don't think we have the capacity to do our own tasks AND provide consulting for new connector implementations

nkl199 (Fri, 04 Sep 2020 08:40:42 GMT):
ok, ... should we remove the relevant sections from the vNext docs?

klenik (Fri, 04 Sep 2020 08:44:23 GMT):
Since there's API incompatibility, no need to do the deprecate-then-remove steps. So shall we remove code/docs now?

nkl199 (Fri, 04 Sep 2020 08:51:44 GMT):
seems the reasonable thing to do to be honest

nkl199 (Fri, 04 Sep 2020 13:59:57 GMT):
Just a heads up that I wont be able to attend the maintainers meeting on weds next week :thumbsup:

klenik (Wed, 09 Sep 2020 10:51:20 GMT):
I have a conflicting meeting today, so can't attend the call. Progress update: I'm going through the docs looking for outdated examples or concepts, or anything that requires clarification. The next days I'll also have a look at the prs, there's quite a few now (but mostly reviewed as I see, thanks Dave :tada:)

davidkel (Mon, 14 Sep 2020 08:29:03 GMT):
@klenik @nkl199 Hi in the BlockChainConnector implementation there is this definition ``` /** * Send a request to the backing SUT. Must be overridden by derived classes. * @param {object} request The object containing the options of the request. * @return {Promise} The array of data about the executed requests. * @async */ async _sendSingleRequest(request) { throw new Error('_sendSingleRequest is not implemented for this blockchain connector'); } ``` `_sendSingleRequest` is a protected method (ie semi-public) and not really private, so can I propose we remove the underscore pattern from this method as that makes more sense to me

klenik (Mon, 14 Sep 2020 08:34:40 GMT):
Makes sense, but does JS have a notation for protected methods? Without the underscore it looks like a public API function. (Btw, it could be, but then sendRequest should be sendRequestBatch, and this should be sendRequest)

nkl199 (Mon, 14 Sep 2020 08:53:14 GMT):
:thinking: I think having both as public methods would work nicely, iirc there is no "protected" method in javascript, though you can emulate it through some hackery using call/apply on a function outside of the exported class (shudder). I like the idea of `sendRequest` and `sendRequestBatch` as being public methods that a user may select

davidkel (Mon, 14 Sep 2020 09:01:46 GMT):
As a side, some people suggest that you use the `_` for both protected and private methods which is subjective really. Of course some OO purists insist that protected was a bad idea in the first place and you should only have public and private :-) As with Javascript all methods are really public unless you perform clever tricks to hide the methods then I would say the best suggestion I have to describe these would be through the use of JSDoc and don't expose or mark as private all methods that should not be used by caliper or end user developers, others would then be documented to target the audience who would use them and you can also mark them as `@protected` as well.

klenik (Mon, 14 Sep 2020 09:04:54 GMT):
Okay, I support removing the underscore. Most IDEs can process and notify the users about protected/private tags anyway.

klenik (Mon, 14 Sep 2020 09:04:54 GMT):
Okay, I support removing the underscore. Most IDEs can process and notify the users about protected/private tags anyway. For now I'd keep the single `sendRequest` public method, so users don't have to care about the number of TXs.

jramps9 (Mon, 14 Sep 2020 17:36:02 GMT):
Hello Caliper contributors! Reminder to please join the DevRel Marketing Committee call at 9am PT on 9/16 this week. Take a look at the agenda and add items here: https://wiki.hyperledger.org/display/Marketing/2020-09-16+Meeting+notes

davidkel (Wed, 16 Sep 2020 09:18:18 GMT):
I would like to put forward a proposal for discussion. It's about using newer capabilities of node. With the release of 0.4 there is an opportunity to define supported versions of node and with that allows us to know what newer capabilities of node we could exploit. The node LTS Table looks like this ``` v10 Maintenance LTS - EOL: 2021-04-30 v12 Active LTS - EOL: 2022-04-30 v14 Current- SOL: 2020-10-27 (EOL = End of Life, SOL = Start of Life) ``` Depending on when V0.4 is planned for release I think that would affect whether node V14 could be included, but I would hope that 0.5 wouldn't be too long after 0.4 So could consider that 0.5 includes V14 support. We should definitely drop support for V8 and the minimum should be V10. I think the limiting factors here will be the versions of node that the connectors themselves declare support for and how we manage that considering you can bind different versions. But I would expect connectors to always support the minimum node LTS version at time of release in order to be included in a release. One bonus is that with the removal of node 8, we can make use of at least node 10 features which include `fs.promises` support, meaning we don't have to rely on fsextra anymore. Whether we want to go back and remove all uses of fsextra as well as all uses of fs sync use (unless there is a good reason to use the sync methods) I think is worth talking about as well.

klenik (Wed, 16 Sep 2020 09:30:49 GMT):
Great idea! If we make the Dockerfile path configurable in [this script](https://github.com/hyperledger/caliper/blob/master/packages/caliper-publish/artifacts/docker-build.sh) (and in the publishing CLI), then we can build and test Caliper locally as a PoC with the [local docker test](https://github.com/hyperledger/caliper/tree/master/packages/caliper-tests-integration/fabric_docker_distributed_tests) by providing custom Dockerfiles, using different node versions. I think this way would be faster than the CI path, and we can run the test with different `(SUT x SUT version x Node version)` tuples to detect potential incompatibilities :thinking:

klenik (Wed, 16 Sep 2020 09:32:47 GMT):
And then we can argue about range of Node version support instead of a single version (using a `SUT version x Node version` compatibility table for each SUT). Although for the published docker image we should probably stick with a single/newest Node version.

klenik (Wed, 16 Sep 2020 10:22:18 GMT):
[ ](https://chat.hyperledger.org/channel/caliper-contributors?msg=g9wpJHzow5RT6X5xy) Added some thoughts to this in the issue: https://github.com/hyperledger/caliper/issues/998

nkl199 (Wed, 16 Sep 2020 10:47:13 GMT):
oooo - next issue raised is #1000 ... is that a good or bad thing?! :thinking:

nkl199 (Wed, 16 Sep 2020 10:49:22 GMT):
... and it's taken with a PR :joy:

klenik (Wed, 16 Sep 2020 10:49:45 GMT):
Since every software has issues, no issues means stale software :D We're not far from the #1000 closed issue, and that's definitely a good thing :grin:

nkl199 (Wed, 16 Sep 2020 14:42:20 GMT):
also a conversation on: - connector removal - bindings (those users may call, and defaults in the container image)

nkl199 (Fri, 18 Sep 2020 11:39:18 GMT):
candidate item for next weeks call - release schedules. I think it is fair to say that we don't release very often. At the moment it is because we want to contain the majority of breaking changes in the next version bump. However, after the 0.4 release we might want to include a "do we release" conversation every month, which could be based on: - an essential bug fix to go out - enough minor bug fixes to warrant a patch release (to remove the niggles) - completion of a breaking change/feature that we want out

klenik (Fri, 18 Sep 2020 16:31:21 GMT):
I think we managed to keep the breaking changes under control for this release. The only major change from a user's PoV is the new workload module approach. Everything else is either a smaller config change, or new feature. (The internal changes are waaaay bigger, but I think the maintainers and contributors like the new design better :slight_smile: )

klenik (Fri, 18 Sep 2020 17:37:20 GMT):
There are still some API breaking changes down the road, these might signal fast release cycles after v0.4.0 :thinking: In no particular order, and without being exhaustive: 1. Make messengers configurable/selectable through config file (currently they're configured through runtime configs, which is kind of weird) 2. Separate monitors and reporters in a clean way, and make reporters pluggable in the process :) 3. Introduce a "Module Config" file, that gathers every single-place-of-use plugins: messengers, TX monitors, resource monitors, reporters. (1. depends on this.) 4. Ongoing Fabric connector refactor ~5. Idea of "Contract Config" file, so the network config is separated from the deployed artifacts~ 5. is crossed out, because it can be easily achieved inside the connector by using a runtime config, so I think there's no need to pollute the architecture with it.

klenik (Sat, 19 Sep 2020 08:36:40 GMT):
Raised a draft PR for the connector docs: https://github.com/hyperledger/caliper/pull/1007 I plan a complete rewrite for this page, since we will need help with reimplementing the recently removed connectors. Plus a consensus of requirements is always helpful. Some of these thoughts are subjective, open for discussion and extension.

klenik (Mon, 21 Sep 2020 08:05:40 GMT):
Update: the PR is live :newspaper:

klenik (Mon, 21 Sep 2020 09:36:53 GMT):
And now the changes are actually added to the commit :joy: :sweat_smile:

nkl199 (Mon, 21 Sep 2020 09:37:24 GMT):
lol, been there ... many times :joy:

nkl199 (Mon, 21 Sep 2020 15:15:21 GMT):
38 files :worried: suddenly prefer how that PR was before :joy:

klenik (Mon, 21 Sep 2020 15:25:52 GMT):
Most of them are only renaming, don't worry :grin:

klenik (Mon, 21 Sep 2020 15:26:38 GMT):
[ ](https://chat.hyperledger.org/channel/caliper-contributors?msg=me82Dsjy4gZDSzhJ5) I was actually talking about the docs PR, the code PR was already live on Friday evening :D

nkl199 (Mon, 21 Sep 2020 15:26:48 GMT):
ah!

klenik (Wed, 23 Sep 2020 06:13:20 GMT):
@davidkel We probably need some fresh/unbiased eyes on the new connector docs (https://github.com/hyperledger/caliper/pull/1007). Could you take a quick look at it please, whether it's clear what I'm talking about? :) https://aklenik.github.io/caliper/vNext/writing-connectors/

davidkel (Wed, 23 Sep 2020 10:36:21 GMT):
@klenik I will take a look in more detail but I think I have already hit a problem with the interface. It looks like the interface defines the lifecycle on a connector as ``` - synchronous constructor (expected it ready for use) - optional init depending on master or worker ``` The problem I have is that in some of the work we are doing to change the network configuration it's not possible to construct a connector that is ready for use because asynchronous activities take place to make a connector available. This would mean having to push the logic into the init method but the docs say this is optional. As Caliper is managing the lifecycle of this connector it dictates when a connector is ready for use and if the assumption is purely down to the results of newing a synchronous constructors then connectors may very well have to do lazy initialising on all other calls. The other thing I came across was around getContext. It looks like getContext can be called twice which means connectors need to potentially protect against this (as was done in the fabric v1 connector) so would be worth mentioning in the docs

davidkel (Wed, 23 Sep 2020 10:36:21 GMT):
@klenik I will take a look in more detail but I think I have already hit a problem with the interface. It looks like the interface defines the lifecycle on a connector as ``` - synchronous constructor (expected it ready for use) - optional init depending on master or worker ``` The problem I have is that in some of the work we are doing to change the network configuration it's not possible to construct a connector that is ready for use because asynchronous activities take place to make a connector available. This would mean having to push the logic into the init method but the docs say this is optional. As Caliper is managing the lifecycle of this connector it dictates when a connector is ready for use and if the assumption is purely down to the results of newing a synchronous constructor then connectors may very well have to do lazy initialising on all other calls. The other thing I came across was around getContext. It looks like getContext can be called twice which means connectors need to potentially protect against this (as was done in the fabric v1 connector) so would be worth mentioning in the docs

klenik (Wed, 23 Sep 2020 11:03:33 GMT):
Yes, this dual environment is a bit confusing. 1. So, the manager always calls init and install on the connector, the worker never (maybe optional isn't the right word in the docs). 2. For the worker instances, the async factory method must return a ready-to-use connector, and the worker will call `getContext` on it next. 3. There's still some unfortunate naming in the API: get/releaseContext really should be roundStarting/roundFinished (or something similar). 4. The context concept is still a legacy approach from the early times of Caliper. I'm not sure this is the best way to expose data to workload modules. The connectors should expose additional API functions for additional features. Then the start/finish rounds calls could be events initiated by the worker, never fired twice. 5. As I was writing the docs, I really had the feeling that two APIs would be needed for this dual environment. Common code could be extracted meaningfully I think, and the concepts would be clear. Thoughts are welcome

davidkel (Wed, 23 Sep 2020 11:55:43 GMT):
I've just found the connector factory :-) so that helps, but I do wonder on the purpose of the init method then and probably calling it `init` is not the right method name here as it implies a certain meaning

klenik (Wed, 23 Sep 2020 11:57:22 GMT):
Yep. Nick proposed the initConnector and initSut distinctions. The bottom line is we have to make these concepts cleaner a bit...

nkl199 (Wed, 23 Sep 2020 12:07:30 GMT):
do we want this clarification in 0.4 or 0.5?

klenik (Wed, 23 Sep 2020 12:13:20 GMT):
I say 0.5, we should really roll out 0.4 already

nkl199 (Wed, 23 Sep 2020 12:13:37 GMT):
phew, was hoping u would say that!

klenik (Wed, 23 Sep 2020 14:59:07 GMT):
(might be a few minutes late, and drop off after 30 minutes :sweat_smile:)

nkl199 (Thu, 24 Sep 2020 11:17:55 GMT):
phew, had a crack at re-writing the tutorial :scream: there were many comments so i might have missed a few. My plan is to give it a few hours, then run through the tutorial on a fresh VM to check it works

nkl199 (Thu, 24 Sep 2020 11:32:31 GMT):
any changes required in the changelog PR?

klenik (Thu, 24 Sep 2020 13:56:37 GMT):
I think it contains everything. Maybe a doc link for new features and broken APIs would be useful, but I'm okay without it

nkl199 (Thu, 24 Sep 2020 14:00:37 GMT):
hmmm, API docs links can get broken ... but a "go look at the docs" would work

nkl199 (Thu, 24 Sep 2020 14:00:50 GMT):
or link the the PR with ur additions to the docs

klenik (Thu, 24 Sep 2020 14:02:05 GMT):
[ ](https://chat.hyperledger.org/channel/caliper-contributors?msg=CEbF4Jc5fbgrqqcTu) Can't we "pre-link" to the v0.4.0 paths? The docs will be bumped anyway, and won't change after that.

klenik (Thu, 24 Sep 2020 14:02:42 GMT):
Or the users can just look for the keywords in the change list :man_shrugging_light_skin_tone:

nkl199 (Thu, 24 Sep 2020 14:03:13 GMT):
hmmm, good point! (bumping docs)

nkl199 (Thu, 24 Sep 2020 14:03:56 GMT):
ah, but we track the current "latest minor" such as 0.3.2, vNext

nkl199 (Thu, 24 Sep 2020 14:04:07 GMT):
we will soon have 0.3.2, 0.4, vNext

klenik (Thu, 24 Sep 2020 14:04:43 GMT):
Then let's leave it as it is

nkl199 (Thu, 24 Sep 2020 14:04:54 GMT):
righto

nkl199 (Thu, 24 Sep 2020 14:15:21 GMT):
OK, i've reached tutorial saturation poing :joy:

nkl199 (Thu, 24 Sep 2020 14:15:21 GMT):
OK, i've reached tutorial saturation point :joy:

nkl199 (Thu, 24 Sep 2020 14:15:48 GMT):
https://nklincoln.github.io/caliper/vNext/fabric-tutorial/tutorials-fabric-existing/ for the interest parties

nkl199 (Thu, 24 Sep 2020 14:15:48 GMT):
https://nklincoln.github.io/caliper/vNext/fabric-tutorial/tutorials-fabric-existing/ for the interested parties

klenik (Thu, 24 Sep 2020 14:19:40 GMT):
Just a quick remark for the beginning: 1. I would note the repo tag for the Fabric sample, in case the example will change again later 2. I would switch step 1 & 2. First the workspace is created, then Caliper is installed locally there (is `npm init` needed or the install will create it?)

nkl199 (Thu, 24 Sep 2020 14:27:09 GMT):
good points!

nkl199 (Thu, 24 Sep 2020 14:31:38 GMT):
i think i will combine those steps to make it more explicit that the bind is localin that workspace

nkl199 (Thu, 24 Sep 2020 14:31:38 GMT):
i think i will combine those steps to make it more explicit that the bind is local in that workspace

nkl199 (Thu, 24 Sep 2020 14:36:16 GMT):
(above change is now live)

klenik (Thu, 24 Sep 2020 16:33:47 GMT):
Btw, should we add a `tutorials` folder to the bench repo? It could contain the final artifacts

lucassaldanha (Fri, 25 Sep 2020 00:53:21 GMT):
Has joined the channel.

lucassaldanha (Fri, 25 Sep 2020 00:53:22 GMT):
Hi folks! I am working on a update to the ethereum-connector to add support for Hyperledger Besu private transactions. I would love some feedback from you on my approach. I'm planning opening a draft PR with my changes to get some feedback. Does that work for you guys?

klenik (Fri, 25 Sep 2020 06:32:59 GMT):
@lucassaldanha Sure, and if you could give a short summary of it (private txs in general, how clients use it, some technical details, etc) next Wednesday on the Caliper call, that would be even better :) Let me just tag the Besu experts @benjamincburns and @shemnon so they're also aware of this

nkl199 (Fri, 25 Sep 2020 08:18:32 GMT):
https://hacktoberfest.digitalocean.com/ <- time to start tagging issues? :joy:

nkl199 (Fri, 25 Sep 2020 09:22:37 GMT):
@rjones Hi Ry, it seems that the Caliper repository has lost connection to the Travis builds. Is this something that was intended to occur? This is a little problematic for us, as not only do we use Travis to run the unit/integration tests, it is also used to publish Caliper for both unstable and official releases

rjones (Fri, 25 Sep 2020 09:22:37 GMT):
Has joined the channel.

nkl199 (Fri, 25 Sep 2020 09:35:41 GMT):
ok ... so it seems the builds are back?

klenik (Fri, 25 Sep 2020 09:38:30 GMT):
Hopefully it stays that way

rjones (Fri, 25 Sep 2020 12:14:41 GMT):
I didn’t do anything

rjones (Fri, 25 Sep 2020 12:15:07 GMT):
please migrate to azure pipelines or GitHub actions tho

nkl199 (Fri, 25 Sep 2020 12:18:53 GMT):
noted- will add an issue into the repo to do that

rjones (Fri, 25 Sep 2020 12:19:58 GMT):
Thanks. Have you played with the minifab integration? Good stuff

nkl199 (Fri, 25 Sep 2020 12:23:37 GMT):
Assisted tong with a few small bits when he moved it in, but not used it myself yet :thumbsup:

lucassaldanha (Sat, 26 Sep 2020 06:39:21 GMT):
Actually, @benjamincburns is aware of the changes. We work together at ConsenSys :) At what time do you hold the contributor calls? I have a feeling they might be in the middle of the night for me (I'

klenik (Sat, 26 Sep 2020 15:03:04 GMT):
3PM UTC. If you sum up the above points in the pr, that's also fine

klenik (Mon, 28 Sep 2020 16:12:19 GMT):
@nkl199 @davidkel Which `fabric-samples` tag are you using for the tutorial? I used the latest non-master `v2.1.1` tag, and it uses fabcar... We should probably specify a commit on which the tutorial has been tested (and update it to a named release once available).

nkl199 (Mon, 28 Sep 2020 16:32:42 GMT):
:thumbsup: good idea - will grab the bits I was using ... brb

nkl199 (Mon, 28 Sep 2020 16:33:43 GMT):
https://hyperledger-fabric.readthedocs.io/en/release-2.2/tutorials.html

nkl199 (Mon, 28 Sep 2020 16:33:57 GMT):
and this: https://github.com/hyperledger/fabric-samples/blob/master/asset-transfer-basic/chaincode-javascript/lib/assetTransfer.js

nkl199 (Mon, 28 Sep 2020 16:37:49 GMT):
so it looks like the 2.2 fabric sdks are broken with respect to concurrent queries. Suggestion here is that we pull the binding from the CLI, and add it back in when it works

klenik (Mon, 28 Sep 2020 17:18:47 GMT):
How is that possible? :O

nkl199 (Tue, 29 Sep 2020 08:17:20 GMT):
good question

nkl199 (Tue, 29 Sep 2020 14:12:09 GMT):
oof, yet another push on the tutorial :joy:

klenik (Wed, 30 Sep 2020 07:17:57 GMT):
I also left a few smaller notes, but technically I could run the tutorial without a glitch, so hurray :tada:

nkl199 (Wed, 30 Sep 2020 07:33:07 GMT):
not sure why, but i just can't get the "code snippet" to work

klenik (Wed, 30 Sep 2020 07:34:58 GMT):
I changed the v2.something tag to the commit hash I wrote in the remarks (current master), and it worked like a charm.

nkl199 (Wed, 30 Sep 2020 07:36:51 GMT):
ok, will give it a go

nkl199 (Wed, 30 Sep 2020 07:54:57 GMT):
aha, i haz network :D

nkl199 (Wed, 30 Sep 2020 07:55:04 GMT):
and deployed contract

klenik (Wed, 30 Sep 2020 07:56:59 GMT):
:tada:

klenik (Wed, 30 Sep 2020 14:09:45 GMT):
I can only drop in only for like 10 minutes, maybe enough time for a quick release-check (tasks concerning me) :sweat_smile:

klenik (Wed, 30 Sep 2020 14:09:45 GMT):
I can only drop in only for like 10 minutes for the call, maybe enough time for a quick release-check (tasks concerning me) :sweat_smile:

nkl199 (Wed, 30 Sep 2020 14:34:56 GMT):
OK, suggested agenda: - questions around current PRs - any pending inclusions for 0.4 release - possible discussion on when to release :tada:

klenik (Wed, 30 Sep 2020 15:01:50 GMT):
Zoom channel with pass: https://zoom.us/my/hyperledger.community?pwd=STZQd0xMZU9xRVVOVnpQM3JNQ2dqZz09

klenik (Thu, 01 Oct 2020 12:50:50 GMT):
There is something wrong with the ethereum networks in the bench repo. I'll remove them and add the setup we use for the CI, that has no issues.

nkl199 (Thu, 01 Oct 2020 14:03:28 GMT):
righto, hadn't noticed those as ... tbh, i dont use them

nkl199 (Thu, 01 Oct 2020 14:03:46 GMT):
would be good to get a CI for all items in the benchmarks post 0.4 release

klenik (Thu, 01 Oct 2020 14:04:17 GMT):
that would be a long running CI :sweat_smile:

nkl199 (Thu, 01 Oct 2020 14:04:52 GMT):
yh, maybe just one test from each connector type?

klenik (Thu, 01 Oct 2020 14:05:40 GMT):
I was thinking about unit tests for the workload modules, using mocked connectors. So at least we can detect API changes

klenik (Thu, 01 Oct 2020 14:05:40 GMT):
I was thinking about unit tests for the workload modules, using mocked connectors. So at least we can detect API changes. And one integration test per SUT type maybe.

nkl199 (Thu, 01 Oct 2020 14:14:30 GMT):
oooo, much neater

lucassaldanha (Thu, 01 Oct 2020 23:15:46 GMT):
Hi guys, I have opened the PR adding support to Besu private transactions. Let me know what you think! https://github.com/hyperledger/caliper/pull/1030

nkl199 (Fri, 02 Oct 2020 07:02:34 GMT):
thanks, we're aiming to release 0.4 today, so hopefully it will be "officially available" in 0.5 (but we also publish unstable versions on every PR inclusion)

lucassaldanha (Mon, 05 Oct 2020 01:28:08 GMT):
Thanks! I have just updated the PR :+1:

klenik (Wed, 07 Oct 2020 14:51:48 GMT):
Used for contributors of Hyperledger Caliper to discuss implementation details. [Weekly Zoom calls.](https://zoom.us/my/hyperledger.community?pwd=STZQd0xMZU9xRVVOVnpQM3JNQ2dqZz09)

nkl199 (Thu, 08 Oct 2020 07:49:17 GMT):
a consideration for when performance testing (new) fabric networks .. use "roll back" instead of rebuilding the entire network from scratch: https://cloud.ibm.com/docs/blockchain?topic=blockchain-backup-restore

klenik (Thu, 08 Oct 2020 08:05:53 GMT):
Does this method also save the state database content? We tried the same approach using swarm (point a "restarted" peer to a volume containing its previous chain), but recommitting the TX write sets to the state DB took the same time (or even more, I don't remember exactly) than filling the DB from scratch with traditional TXs (which was really surprising that we didn't gain much/anything).

klenik (Thu, 08 Oct 2020 08:19:11 GMT):
(Off topic: the concept of rolling back entire networks sound like a cross-organizational hell to me. Why should org1 lose its CC1 TXs on channel1, just because a faulty CC2 was deployed by someone else on a different channel? Especially if the failure surfaces only later. I think BPE-like compensation mechanisms would be the correct way to go. A distributed ledger is always backed up by design. A network-wide (or just channel-wide, if possible) rollback... well, good luck pushing that through with the participants and auditors. Defeats the whole purpose.)

nkl199 (Thu, 08 Oct 2020 09:27:21 GMT):
yh, agree that cross organisational roll back reeks of "lets hide that" :joy:

klenik (Thu, 08 Oct 2020 13:58:45 GMT):
Any objections for configuring stale-bot for the bench repo?

nkl199 (Thu, 08 Oct 2020 15:59:41 GMT):
none at all :thumbup:

nkl199 (Thu, 08 Oct 2020 15:59:59 GMT):
2 day timeout? :thinking: :joy:

klenik (Wed, 14 Oct 2020 08:50:49 GMT):
So there were two significant fixes recently: 1) Bumping the node version in the Docker image. Both v0.3.2 and v0.4.0 can benefit from this. Only the image needs to be rebuilt and repushed, can be done locally. 2) A TX stat merging bug fix is on its way, which only affects v0.4.0. I would suggest the following: * Repush v0.3.2 docker image using the updated docker file (no PR required). * Switching back to a v0.4.1 patch version after the fix is in, which will push a fix to both npm and docker hub.

nkl199 (Wed, 14 Oct 2020 08:52:04 GMT):
At the moment there are no breaking changes, so we can keep with the 0.4.2-unstable once the patch has gone out :thumbup:

nkl199 (Wed, 14 Oct 2020 09:34:15 GMT):
Added agenda item: node version bump and implications for caliper docker image

jramps9 (Wed, 14 Oct 2020 14:07:33 GMT):
Hello Caliber contributors! Reminder to please join the DevRel Marketing Committee call at 9am PT TODAY. Take a look at the agenda and add items if you'd like here: https://wiki.hyperledger.org/display/Marketing/2020-10-14+Meeting+notes

klenik (Wed, 14 Oct 2020 14:16:15 GMT):
@nkl199 I have an almost conflicting meeting today, so if we start with the update of docker images, I'd be grateful :grin:

klenik (Wed, 14 Oct 2020 14:16:15 GMT):
@nkl199 I have an almost conflicting meeting today, so if we could start with the update of docker images, I'd be grateful :grin:

nkl199 (Wed, 14 Oct 2020 14:16:28 GMT):
sure :thumbup:

nkl199 (Wed, 14 Oct 2020 14:16:45 GMT):
i suspect it will be a quick meeting

Gary_Hsing (Tue, 20 Oct 2020 19:07:26 GMT):
Has joined the channel.

Gary_Hsing (Tue, 20 Oct 2020 19:07:26 GMT):
Hello! We are a team of graduate students from Duke University looking into doing a project extending Caliper to be able to gather statistics from Hyperledger Sawtooth. We saw from online and chat history here that Sawtooth used to be supported and is no longer. Is this a feasible project to accomplish in about a month and a half? Is there anyone we can chat with more to explore this option?

rjones (Tue, 20 Oct 2020 19:13:45 GMT):
@Gary_Hsing I think, in general, it should be doable. IIRC Caliper support for Sawtooth was removed due to lack of interest on Sawtooth's part, not any architectural change

Gary_Hsing (Tue, 20 Oct 2020 19:16:48 GMT):
@rjones Thank you very much!

klenik (Tue, 20 Oct 2020 19:27:51 GMT):
@Gary_Hsing It's absolutely doable! The previous Sawtooth connector was removed due to a combination of hard-to-maintain/limited code-base + high-level API change + lack of community/expert interest/support. But if you'd like to reimplement the sawtooth support, we'd be happy to assist you along the way. The first step is getting familiar with the way connectors should behave: https://hyperledger.github.io/caliper/v0.4.1/writing-connectors/

Gary_Hsing (Tue, 20 Oct 2020 20:45:49 GMT):
Thanks for the information! Appreciate it a lot, we will start from reading those materials!

davidkel (Wed, 21 Oct 2020 14:05:36 GMT):
While thinking more about the equivalent replacement of the operational capabilities for the 1.4 fabric connectors, the current proposed specification was ``` - channelName: otherchannel # [Optional] create the channel create: true definition: # [Either/Or] if you create a channel you must provide a prebuiltTransaction ore capabilities/consortium/msps/version prebuiltTransaction: 'some/binary.tx' # capabilities: [] # consortium: 'SampleConsortium2' # msps: ['Org1MSP', 'Org2MSP'] # version: 0 # Array of contracts to be installed/instantiated on the named channel and available for use by the workload module contracts: - id: marbles contractID: myMarbles # the following are required if create is true version: v0 language: golang path: marbles/go metadataPath: src/marbles/go/metadata ``` However now I understand it a bit more, I think that we could make this section simplier. ``` - channelName: otherchannel definition: # [Either/Or] if you create a channel you must provide a prebuiltTransaction or capabilities/consortium/msps/version prebuiltTransaction: 'some/binary.tx' # capabilities: [] # consortium: 'SampleConsortium2' # msps: ['Org1MSP', 'Org2MSP'] # version: 0 # Array of contracts to be installed/instantiated on the named channel and available for use by the workload module contracts: - id: marbles contractID: myMarbles # the following are required if create is true version: v0 language: golang path: marbles/go metadataPath: src/marbles/go/metadata ``` ie remove the create property altogether. If you want channel creation, then you need to specify the definition property of a channel (you only need to provide a definition if you plan to create a channel) if you want chaincode install/instantiate them you need to provide version, language,path and optionally metadataPath (these are only required for chaincode install/instantiate) Suggestions welcome, but this capability is something that I don't think should be in caliper moving forward, just putting it in for backward compatibility and the integration tests really

davidkel (Wed, 21 Oct 2020 14:05:36 GMT):
While thinking more about the equivalent replacement of the operational capabilities for the 1.4 fabric connectors, the current proposed specification was ``` - channelName: otherchannel # [Optional] create the channel create: true definition: # [Either/Or] if you create a channel you must provide a prebuiltTransaction ore capabilities/consortium/msps/version prebuiltTransaction: 'some/binary.tx' # capabilities: [] # consortium: 'SampleConsortium2' # msps: ['Org1MSP', 'Org2MSP'] # version: 0 # Array of contracts to be installed/instantiated on the named channel and available for use by the workload module contracts: - id: marbles contractID: myMarbles # the following are required if create is true version: v0 language: golang path: marbles/go metadataPath: src/marbles/go/metadata ``` However now I understand it a bit more, I think that we could make this section simplier. ``` - channelName: otherchannel definition: # [Either/Or] if you create a channel you must provide a prebuiltTransaction or capabilities/consortium/msps/version prebuiltTransaction: 'some/binary.tx' # capabilities: [] # consortium: 'SampleConsortium2' # msps: ['Org1MSP', 'Org2MSP'] # version: 0 # Array of contracts available on the channel contracts: - id: marbles contractID: myMarbles # the following are required to install/instantiate version: v0 language: golang path: marbles/go metadataPath: src/marbles/go/metadata ``` ie remove the create property altogether. If you want channel creation, then you need to specify the definition property of a channel (you only need to provide a definition if you plan to create a channel) if you want chaincode install/instantiate them you need to provide version, language,path and optionally metadataPath (these are only required for chaincode install/instantiate) Suggestions welcome, but this capability is something that I don't think should be in caliper moving forward, just putting it in for backward compatibility and the integration tests really

nkl199 (Wed, 21 Oct 2020 14:20:15 GMT):
what about shadowing the "definition" in contracts too then? So as a minimal, you get the channelName, with contracts[id/contractId] ... but can explicitly add definitions for both?

nkl199 (Wed, 21 Oct 2020 14:22:17 GMT):
``` - channelName: otherchannel definition: # [Either/Or] if you create a channel you must provide a prebuiltTransaction or capabilities/consortium/msps/version prebuiltTransaction: 'some/binary.tx' # capabilities: [] # consortium: 'SampleConsortium2' # msps: ['Org1MSP', 'Org2MSP'] # version: 0 # Array of contracts available on the channel contracts: - id: marbles contractID: myMarbles definition: version: v0 language: golang path: marbles/go metadataPath: src/marbles/go/metadata ```

klenik (Wed, 21 Oct 2020 14:22:24 GMT):
I agree that channel and contract management in Fabric are reaching a complexity that is hard to mirror in Caliper (and keeping the configuration simple at the same time). Remark about the `created` attribute: it was intended to ease re-running benchmark when the channel was already created in the first run. It's only useful for JSON configs, where you can't comment the "create block". But the connector also supports skipping channel creation based on runtime configs. So the `created` attribute seems a bit obsolete now.

davidkel (Wed, 21 Oct 2020 14:23:29 GMT):
There is also this in the code base ``` if (ConfigUtil.get(ConfigUtil.keys.Fabric.SkipCreateChannelPrefix + channelNameToCreate, false)) { logger.info(`Creation of Channel '${channelNameToCreate}' is configured to skip`); continue; } ``` Which isn't on the Join part of the creation, wondered if we would still want to keep it and also have it check in the join as well

davidkel (Wed, 21 Oct 2020 14:26:44 GMT):
So is the concensus that there is no need now for a create/created flag in the config ?

klenik (Wed, 21 Oct 2020 14:26:45 GMT):
The initial problem was/is that the existence of a channel is considered private information, so it cannot be queried. You can list the channels a peer is part of, but even if you merge every query result for every peer, it's not a definitive answer to "Does this channel exist?". If a known peer is a part of it, then you can answer it, but you can't detect empty channels for example (AFAIK). And the user side of the problem is that Fabric returns general/obfuscated errors when dealing with channel creation/join. So we delegated this knowledge to the users :)

klenik (Wed, 21 Oct 2020 14:27:32 GMT):
[ ](https://chat.hyperledger.org/channel/caliper-contributors?msg=eWjA6r7RRvmNzJ7DF) I won't miss it. But then we should assemble a FAQ about possible channel related errors (see my comment above)

davidkel (Wed, 21 Oct 2020 14:31:04 GMT):
Ok, so we can leave it out for now. How about determining whether a chaincode should be installed/instantiated. Do we want a section as per Nicks suggestion or a more minimal one as per my original suggestion ?

klenik (Wed, 21 Oct 2020 14:34:24 GMT):
When I was putting together the joi validation schema, it was cumbersome to code the OR/AND/XOR/IMPLIES relations among sibling attributes of channels and contracts. So I'd suggest some extra sections to make it clear which operation needs which attributes. Something like this: ``` channels: - channelName: mychannel create: prebuiltTransaction: 'some/binary.tx' buildTransaction: capabilities: [] consortium: 'SampleConsortium2' msps: ['Org1MSP', 'Org2MSP'] version: 0 contracts: - id: marbles contractID: mymarbles install: version: v0 language: golang path: marbles/go metadataPath: src/marbles/go/metadata installWithIdentity: admin@org1.exampl.com instantiate: initFunction: init initArguments: [] instantiateWithIdentity: admin@org1.exampl.com ```

klenik (Wed, 21 Oct 2020 14:36:54 GMT):
Operations are easy to skip based on the absence of the related attributes (create, install, instantiate). Plus I always missed the flexibility to specify who should perform these tasks :) Not that useful for performance benchmarking, but useful for testing different management policies. But since we would like to remove such admin capabilities on the long run, I'm fine without the identity specs

klenik (Wed, 21 Oct 2020 14:38:44 GMT):
(a join section could be added to the channel similarly, but I think there's a peer list for that already in the CCP)

klenik (Wed, 21 Oct 2020 14:38:44 GMT):
(a join section could be added to the channel similarly, but I think there's a peer list for that already in the CCP, but we could provide a better schema :) )

davidkel (Wed, 21 Oct 2020 14:43:11 GMT):
Ok, so change `definition` --> `create`, put the channel creation stuff into it's own subsection and split contract deployment into 2 separate subsections.

davidkel (Wed, 21 Oct 2020 14:43:36 GMT):
Fine with me (sorry won't include the instantiateWithIdentity though)

davidkel (Wed, 21 Oct 2020 14:43:36 GMT):
Fine with me (sorry won't include the install/nstantiateWithIdentity though)

klenik (Wed, 21 Oct 2020 14:44:29 GMT):
I'm fine with this compromise :grin:

davidkel (Wed, 21 Oct 2020 14:47:37 GMT):
@klenik @nkl199 Thanks guys. will adjust it to the above format

klenik (Wed, 21 Oct 2020 15:00:21 GMT):
Conflicting task, so I have to skip the call

nkl199 (Wed, 21 Oct 2020 15:00:51 GMT):
righto - we have conflicting meetings this side too, so I think the call will have to be cancelled for this week

klenik (Wed, 21 Oct 2020 15:01:09 GMT):
Just in time :joy:

nkl199 (Wed, 21 Oct 2020 15:01:38 GMT):
:sweat_smile:

nkl199 (Wed, 28 Oct 2020 14:27:16 GMT):
OK, so bearing in mind there has been a time-saving change in the UK, i'm blatantly going to be either an hour early, or an hour late to the call today :joy:

klenik (Wed, 28 Oct 2020 14:29:20 GMT):
Same here 😂 Based on my rigorous calculations, the call should start in 30 minutes, right? 😅

nkl199 (Wed, 28 Oct 2020 14:29:34 GMT):
that was my thoughts too :shrug:

nkl199 (Wed, 28 Oct 2020 14:29:51 GMT):
let's go with it - way better than being in an hour and a half!

klenik (Wed, 28 Oct 2020 14:30:14 GMT):
Cool, so I won't be alone then 😂

nkl199 (Wed, 28 Oct 2020 14:30:21 GMT):
(NodeJS v14 just went LTS)

klenik (Wed, 28 Oct 2020 14:42:34 GMT):
Some ideas for agenda items: 1. Consequences for moving to NodeJS v12 2. Using release branches starting from v0.5.0 3. Improving naming conventions regarding module life-cycles (TX/resource monitors -> worker/manager monitors, etc)

rjones (Wed, 28 Oct 2020 14:44:55 GMT):
Check out this tool:

jramps9 (Thu, 29 Oct 2020 13:53:14 GMT):
Hello! we're low on project news and updates for the dev/weekly newsletter going out tomorrow 10/30. If there's anything you'd like to suggest, please do so in the comments here: https://wiki.hyperledger.org/pages/viewpage.action?pageId=41584474

klenik (Thu, 29 Oct 2020 16:43:12 GMT):
@nkl199 This, with a "Better late than never" caption :joy: https://github.com/hyperledger/caliper/issues/1065

klenik (Wed, 04 Nov 2020 14:24:16 GMT):
I've consolidated the logging-related issues for v0.4.2: https://github.com/hyperledger/caliper/milestone/6 The workflow will be: make logging module OO -> add capability of dropping logs -> scatter the code-base with debug logs :grin: We can discuss these on the call, if there're any objections

JongWanSeo (Sat, 07 Nov 2020 17:17:52 GMT):
Has joined the channel.

jramps9 (Mon, 09 Nov 2020 18:12:44 GMT):
Hello Caliper contributors! Reminder to please join the DevRel Marketing Committee call at 9am PT on 11/11 this week. Take a look at the agenda and add items if you'd like here: https://wiki.hyperledger.org/display/Marketing/2020-11-11+Meeting+notes

mxevgenis (Wed, 11 Nov 2020 09:48:22 GMT):
Has joined the channel.

mxevgenis (Wed, 11 Nov 2020 09:48:22 GMT):
Hi everyone! I am Michael and i use caliper to measure the performance of my custom Ethereum-based blockchain network and the Smart Contract (SC) i have deployed. I have successfully run some tests where i execute each function of the SC i.e. firstFunction 50 times , secondFunction 50 and thirdFunction 50. Because of the nature of my SC , the result of each function are used as an input to the following function, therefore i would like to ask you if caliper can execute several functions of a SC serially and print the result in terms of throughput and latency? Thank you in advance!

nkl199 (Wed, 11 Nov 2020 10:05:08 GMT):
hi, welcome to the channel :thumbup: you're not the first person to mention this, so we may well have to consider it as an option. My current understanding is that you would like to essentially "start the timer", submit sequenced transactions with the result of one going into the next, and then "stop the timer" once the transaction sequence has completed?

klenik (Wed, 11 Nov 2020 10:31:36 GMT):
@mxevgenis Could you please provide a detailed example containing the following? 1. The sequence of TXs you want to submit: whether you want to submit them in the same `submitTransaction` call, or same round. 2. How do the TXs relate to each other: dependency among them, can they run in parallel, or they would conflict 3. What would you like to see in the report in addition to the current TX-level stats? Stats for groups of TXs? We can't really start designing the feature without these use case details. So please try to "emulate/describe" a Caliper run that would meet your criteria :)

mxevgenis (Wed, 11 Nov 2020 12:03:03 GMT):
This is the main concept. I would like to use values produced by the firstFunction (which is a transaction) as an input to the secondFunction(also a transaction) and check the results. I would like to test the network then by changing the number of this transaction sequence and check the results.

klenik (Wed, 11 Nov 2020 12:05:48 GMT):
@nkl199 @davidkel I have a conflicting call today, so can't attend the Caliper call

mxevgenis (Thu, 12 Nov 2020 14:22:43 GMT):
@klenik When i start a round in caliper i want to call the first function of the SC. This function actually produces a number that is used in the second function of my SC. I would like to perform those two functions serially. Then i would like to increase the number of this transaction sequence and change the rate contoller's value. The report of the caliper will contain information regarding the Avg Latency and Avg Throughput of the network. Is this something that is supported in the current Caliper version? Also from what i have seen Caliper produces results for Transactions made in the network. I have created a function that reads data from the blockchain without performing a transaction. Is there a way to measure the Reads per Second (RPS). Thank you in advance!

nkl199 (Wed, 25 Nov 2020 15:08:00 GMT):
:alarm_clock: Call cut short today, very little attendance, with not too much to talk about aside from the completion of the new fabric adaptor

klenik (Wed, 02 Dec 2020 15:31:33 GMT):
As discussed in the call, future calls will happen biweekly from now on. Updated calendar: https://lists.hyperledger.org/calendar

klenik (Wed, 02 Dec 2020 15:32:06 GMT):
Used for contributors of Hyperledger Caliper to discuss implementation details. [Biweekly Zoom calls.](https://zoom.us/my/hyperledger.community?pwd=STZQd0xMZU9xRVVOVnpQM3JNQ2dqZz09)

nkl199 (Wed, 02 Dec 2020 15:32:41 GMT):
:thumbup: just to clarify, that is once every two weeks

nkl199 (Wed, 02 Dec 2020 15:32:41 GMT):
:thumbup: just to clarify, that is once every two weeks and not twice a week

klenik (Wed, 02 Dec 2020 15:35:24 GMT):
Exactly, and the next call will happen on 16th of December, 15:00 PM UTC0 :)

klenik (Wed, 02 Dec 2020 15:35:24 GMT):
Exactly, and the next call will happen on 16th of December, 3:00 PM UTC0 :)

klenik (Wed, 02 Dec 2020 15:40:03 GMT):
@rjones, will the wiki calendar (https://wiki.hyperledger.org/display/HYP/Calendar+of+Public+Meetings) sync automatically? It embeds a google calendar, I don't know how that relates to the canonical lists.hyperledger.org calendar :man_shrugging_light_skin_tone:

rjones (Wed, 02 Dec 2020 20:29:46 GMT):
@klenik it is a long story of pain

rjones (Wed, 02 Dec 2020 20:33:47 GMT):
@klenik please use `https://lists.hyperledger.org/ics/1902217/3131022486068381536/feed.ics` as your calendar feed

rjones (Wed, 02 Dec 2020 20:34:01 GMT):
if you add that URL to google, it will break.

rjones (Wed, 02 Dec 2020 20:39:08 GMT):
@klenik I've edited the event that was causing syncing to break, so let's see how long it takes to fix it

klenik (Thu, 03 Dec 2020 09:59:31 GMT):
The next week call disappeared from the wiki calendar, so I'd say your magic worked, thx :)

jramps9 (Mon, 07 Dec 2020 18:00:11 GMT):
Hello Caliper contributors! Reminder to please join the DevRel Marketing Committee call at 9am PT on 12/9 this week. Take a look at the agenda and add items if you'd like here: https://wiki.hyperledger.org/display/Marketing/2020-12-09+Meeting+notes

rjones (Wed, 16 Dec 2020 17:26:08 GMT):
@nkl199 @klenik who publishes to DockerHub? I need to get the user count down, do you both need accounts?

klenik (Wed, 16 Dec 2020 17:30:37 GMT):
Publishing to dockerhub and npm is done through my accounts/tokens

rjones (Wed, 16 Dec 2020 17:31:47 GMT):
OK. We'll need to get those converted to one of our bots, but that's for a later day.

klenik (Wed, 16 Dec 2020 17:32:47 GMT):
Cool, great idea :thumbsup:

nkl199 (Thu, 17 Dec 2020 08:16:55 GMT):
(can confirm removal from the dockerhub account ... sniff sniff :joy: )

nkl199 (Wed, 23 Dec 2020 15:55:41 GMT):
@rjones - the move to Azure pipelines is underway, though I do not have sufficient access to the caliper repo to link up to Azure builds ... are you able to complete this aspect?

rjones (Wed, 23 Dec 2020 18:30:26 GMT):
@nkl199 what is your azure account ID?

nkl199 (Wed, 23 Dec 2020 19:28:58 GMT):
same ID as rocket chat - nkl199

jramps9 (Mon, 11 Jan 2021 15:27:55 GMT):
Hello Caliper contributors! Reminder to please join the DevRel Marketing Committee call at 9am PT on 1/13 this week. Take a look at the agenda and add items if you'd like here: https://wiki.hyperledger.org/display/Marketing/2021-01-13+Meeting+notes

prakharsr (Fri, 15 Jan 2021 13:35:07 GMT):
Has joined the channel.

nkl199 (Tue, 26 Jan 2021 11:12:59 GMT):
ny suggested content for the call tomorrow?

nkl199 (Tue, 26 Jan 2021 11:12:59 GMT):
any suggested content for the call tomorrow?

nkl199 (Tue, 26 Jan 2021 11:12:59 GMT):
any suggested content for the call tomorrow? - https://github.com/github/renaming

rjones (Tue, 26 Jan 2021 16:06:00 GMT):
@nkl199 both caliper projects are using `master`: `caliper` and `caliper-benchmarks`

nkl199 (Wed, 27 Jan 2021 11:01:14 GMT):
they certainly do, main decision now is to pick `sith` or `jedi` as the replacement name :joy:

klenik (Wed, 27 Jan 2021 12:24:31 GMT):
master -> sith gh-pages -> jedi :thinking_face:

nkl199 (Fri, 29 Jan 2021 15:36:05 GMT):
@rjones - have I heard correctly that a move from Azure Pipelines to github actions is being encouraged?

rjones (Fri, 29 Jan 2021 15:37:02 GMT):
no?

rjones (Fri, 29 Jan 2021 15:37:17 GMT):
I would prefer you are in AZP

nkl199 (Fri, 29 Jan 2021 15:39:38 GMT):
thanks :+1:

rjones (Fri, 29 Jan 2021 15:40:47 GMT):
where did you hear the opposite?

rjones (Fri, 29 Jan 2021 15:41:19 GMT):
The advantage of AZP is higher performance and longer artifact retention.

nkl199 (Fri, 29 Jan 2021 15:41:28 GMT):
On a call - rumours of azure pipelines being deprecated

nkl199 (Fri, 29 Jan 2021 15:41:36 GMT):
yes - have been enjoying the AZP

rjones (Fri, 29 Jan 2021 15:41:37 GMT):
The advantage of GHA is tighter coupling to GitHub.

jramps9 (Mon, 08 Feb 2021 15:15:40 GMT):
Hello Caliper contributors! Reminder to please join the DevRel Marketing Committee call at 9am PT on 2/10 this week. Take a look at the agenda and add items if you'd like here: https://wiki.hyperledger.org/display/Marketing/2021-02-10+Meeting+notes

nkl199 (Wed, 10 Feb 2021 13:43:47 GMT):
any content for today?

rjones (Wed, 10 Feb 2021 14:23:17 GMT):
Dunno. Have you looked at insights?

rjones (Wed, 10 Feb 2021 14:23:44 GMT):
@nkl199 I can drop in and do a short demo, but if it would be the only item, we can wait :)

klenik (Wed, 10 Feb 2021 14:24:34 GMT):
Nothing for me, crazy busy nowadays :/

klenik (Wed, 10 Feb 2021 14:24:34 GMT):
Nothing from me, crazy busy nowadays :/

davidkel (Wed, 10 Feb 2021 14:45:46 GMT):
Nothing from me

Helen_Garneau (Fri, 12 Feb 2021 13:57:11 GMT):
Has joined the channel.

klenik (Thu, 25 Feb 2021 16:31:47 GMT):
Today I noticed in my local Caliper test that the MQTT messenger initialization hangs indefinitely. And now I also see this in the Azure build log: https://dev.azure.com/Hyperledger/Caliper/_build/results?buildId=30836&view=logs&j=bed1cfea-7bfd-5b32-a5f9-358166d415d0&t=f5c8edc3-bb24-5330-0757-a49ae09da093&l=678

klenik (Thu, 25 Feb 2021 16:33:33 GMT):
@nkl199 Do you have the same issue? Could it be related to mosquitto? I noticed some "Address not available" error locally in the container, but then it logged that the server started :thinking:

davidkel (Thu, 25 Feb 2021 20:24:18 GMT):
@klenik mosquitto released v2 recently and that has almost certainly broken caliper. Fix would be to force the use of mosquitto 1.6 for now.

klenik (Fri, 26 Feb 2021 07:53:38 GMT):
Yeah, now it requires explicit connection configuration https://mosquitto.org/blog/2020/12/version-2-0-0-released/

klenik (Fri, 26 Feb 2021 07:55:53 GMT):
https://mosquitto.org/documentation/migrating-to-2-0/

klenik (Fri, 26 Feb 2021 08:42:14 GMT):
Raised a PR with the necessary mosq. configs: https://github.com/hyperledger/caliper/pull/1128

rjones (Wed, 03 Mar 2021 18:45:27 GMT):
Sad news about AZP: https://github.com/hyperledger/fabric/discussions/2456

klenik (Wed, 10 Mar 2021 12:12:12 GMT):
FYI: I proposed a Caliper-related HL mentorship project, which targets the streamlining of workload modules. Hopefully it'll also make it easy to migrate workload modules between Caliper versions next time :sweat_smile: https://wiki.hyperledger.org/display/INTERN/Declarative+workload+behavior+definition+for+Hyperledger+Caliper

Helen_Garneau (Wed, 10 Mar 2021 12:59:36 GMT):
Hello Caliper contributors: Reminder to please join the DevRel Marketing Committee call at 9am PT today. Take a look at the agenda and add items if you'd like here: https://wiki.hyperledger.org/x/Nqx6Ag

Helen_Garneau (Wed, 10 Mar 2021 14:44:47 GMT):
(Please note new zoom info!)

rjones (Thu, 01 Apr 2021 22:26:00 GMT):
@roxhens maybe ask here?

roxhens (Thu, 01 Apr 2021 22:26:00 GMT):
Has joined the channel.

roxhens (Fri, 02 Apr 2021 08:18:19 GMT):
Is there a tutorial how to benchmark Fabric 2.3.0 with Hyperledger Caliper? I have been searching for 2 weeks and found nothing. I found a tutorial on Caliper website for version 2.2, but when I try withe version 2.3 it does not work and gives me the following error: `error [caliper] [connectors/v2/FabricGateway] Failed to perform submit transaction [CreateAsset] using arguments [0_1,blue,20,penguin,500], with error: Error: No valid responses from any peers. Errors: peer=undefined, status=grpc, message=Peer endorsements do not match`

roxhens (Fri, 02 Apr 2021 08:18:19 GMT):
@all Is there a tutorial how to benchmark Fabric 2.3.0 with Hyperledger Caliper? I have been searching for 2 weeks and found nothing. I found a tutorial on Caliper website for version 2.2, but when I try withe version 2.3 it does not work and gives me the following error: `error [caliper] [connectors/v2/FabricGateway] Failed to perform submit transaction [CreateAsset] using arguments [0_1,blue,20,penguin,500], with error: Error: No valid responses from any peers. Errors: peer=undefined, status=grpc, message=Peer endorsements do not match`

nkl199 (Fri, 02 Apr 2021 18:22:52 GMT):
the v2 gateway should be 100% compatible with fabric 2.3.0 - no breaking changes went into either fabric or the SDK used to connect (we simply consume the Fabric SDk and use it via the gateway method) Was the above from following the tutorial, or a seperately configured fabric network? Have you looked into the peer/chaincode logs to see if there is anything evident there?

roxhens (Fri, 02 Apr 2021 18:25:50 GMT):
Hello, thank you very much for your reply. After looking into it, I think the problem might be that when I bind it using: npx caliper bind --caliper-bind-sut fabric:2.3, it says it is not possible

roxhens (Fri, 02 Apr 2021 18:27:03 GMT):
Then with npx caliper bind --caliper-bind-sut fabric:2.2 it also gets the v2.2.3 of fabric-ca-client from npm library, but version 2.3 of fabric I have uses the fabric-ca-client v1.4.9

roxhens (Fri, 02 Apr 2021 18:28:27 GMT):
Then if I just edit the package.json of caliper-workspace, then it says it does not work because I have to use the caliper bind first and when I use that, it overrides the fabric-ca-client back to v2.2.3

roxhens (Fri, 02 Apr 2021 18:30:08 GMT):
.... This is a separately edited fabric version. So I took fabric v2.3.0 and made some changes and then built the respective binaries and docker images and replaced them in the fabric-samples

bistaastha (Tue, 06 Apr 2021 05:40:27 GMT):
Has joined the channel.

nkl199 (Tue, 06 Apr 2021 15:57:35 GMT):
there is no 2.3 sdk to bind to, you can however use an override file to specify the exact versions of all sdks to bind to if you so desire - using a caliper-bind-file https://hyperledger.github.io/caliper/v0.4.2/installing-caliper/#the-bind-command

bistaastha (Wed, 28 Apr 2021 19:36:53 GMT):
Hello! I am Aastha Bist. I intend to apply for this programme and I'm currently going over the Caliper architecture doc and learning how things work :) Are there any sections of the code/documentation that are more relevant to the project idea?

nkl199 (Thu, 29 Apr 2021 16:45:24 GMT):
Hi, I would certainly recommend all sections relating to applying workloads to the SUT. So this would (i believe) mean looking at the rate controllers and yaml specification of benchmarks ... especially how the caliper core procesess the yaml files to orchestrate a test run

bistaastha (Wed, 05 May 2021 17:21:13 GMT):
Hi! I put together a very barebones doc for personal reference here: https://docs.google.com/document/d/1CCDWlNhGrL7kS1065LCKDhZGB-zciVnMkBgGl98kYr8/edit# Can you please let me know if my perspective aligns with what is the original goal for the project idea to achieve? I am also referencing the sample at: https://github.com/hyperledger/caliper-benchmarks/tree/master/benchmarks/samples/fabric/fabcar to see how the docs compare to actual implementation. I have been unable to set up the test network with fabric because of some system level constraints that I will be working on resolving this weekend, so I am going by references at this point to understand everything. Please let me know your thoughts. Thanks a lot :)

bistaastha (Wed, 05 May 2021 17:21:13 GMT):
Hi! I put together a very bare-bones doc for personal reference here: https://docs.google.com/document/d/1CCDWlNhGrL7kS1065LCKDhZGB-zciVnMkBgGl98kYr8/edit# Can you please let me know if my perspective aligns with what is the original goal for the project idea to achieve? EDIT: Just added the initial points for possible approaches to handle workload module generation. Hopefully they are not too off the mark and would be useful for any kind of future work on the idea :)

bistaastha (Wed, 05 May 2021 17:29:23 GMT):
Also, I don't think I completely understand what "workload module migration" means at this point, so the project objectives in the doc are only half accurate from my own perspective.

bistaastha (Wed, 05 May 2021 17:29:23 GMT):
Also, I don't think I completely understand what "workload module migration" means at this point, so the project objectives in my doc are only half accurate from my own perspective at this point in time.

bistaastha (Wed, 05 May 2021 17:29:23 GMT):
I have been unable to set up the test network with fabric because of some system-level constraints that I will be working on resolving this weekend, so I am going by references at this point to understand everything. Also, I don't think I completely understand what "workload module migration" means at this point, so the project objectives in my doc are only half accurate from my own perspective at this point in time.

bistaastha (Wed, 05 May 2021 17:29:23 GMT):
~I have been unable to set up the test network with fabric because of some system-level constraints that I will be working on resolving this weekend, so I am going by references at this point to understand everything. Also, I don't think I completely understand what "workload module migration" means at this point, so the project objectives in my doc are only half accurate from my own perspective at this point in time.~ Both of the issues in this message are resolved now.

bistaastha (Wed, 05 May 2021 17:29:23 GMT):
~I have been unable to set up the test network with fabric because of some system-level constraints that I will be working on resolving this weekend, so I am going by references at this point to understand everything. Also, I don't think I completely understand what "workload module migration" means at this point, so the project objectives in my doc are only half accurate from my own perspective at this point in time. ~ Both of the issues in this message are resolved now.

bistaastha (Wed, 05 May 2021 17:29:23 GMT):
~ I have been unable to set up the test network with fabric because of some system-level constraints that I will be working on resolving this weekend, so I am going by references at this point to understand everything. Also, I don't think I completely understand what "workload module migration" means at this point, so the project objectives in my doc are only half accurate from my own perspective at this point in time. ~ Both of the issues in this message are resolved now.

klenik (Tue, 13 Jul 2021 15:20:32 GMT):
Hey everyone, it's been a while! @bistaastha has been working on the Caliper mentorship project for some weeks now. The project is about defining workload modules (benchmarks) in a declarative way (as YAML configuration), and provide a built-in workload module to automatically generate corresponding transactions. Two weeks from tomorrow, on the next-next Caliper call, we'd like to present an initial YAML example for describing operations and their parameters. I really hope we can get some valuable feedback from you experienced Caliper users :) @nkl199 @davidkel

klenik (Tue, 13 Jul 2021 15:21:26 GMT):
Spread the word, if you know someone who would be interested in YAML-based workload defs instead of code/script-based workloads

mianhammad (Mon, 13 Sep 2021 07:02:16 GMT):
Has joined the channel.

mianhammad (Mon, 13 Sep 2021 07:02:29 GMT):
Hi Everyone! How can I setup hyperledger caliper for ethereum? Can anyone guide me?

davidkel (Wed, 19 Jan 2022 11:00:44 GMT):
Hi @klenik I've been thinking about how the new fabric 2.4 gateway peer connector can be used in caliper and I think from an external point of view the way it could be done is 1. the new connector is selected when you bind to fabric:2.4 - if you want to benchmark using the legacy sdks you would bind to 2.2 or 1.4. 2. It will only use the new network config format, the old config format cannot be used 3. To determine the peers that will be used for gateway connections a) a connection profile can still be used, but the discover option will have no meaning here b) The peer information can be embedded into the network configuration file under the organisation tree, the format will be ``` peers: - endpoint: localhost:port tlsCACerts: pem: | pem-data path: .... grpcOptions: ssl-target-name-override: peer0.org.example.com hostnameOverride: peer0.org.example.com ``` Note that you can define more than 1 peer, doing so will allow you to create a round robin load balance against the listed peers for requests (A future enhancement, may not get into the first release) Also this format could be used for the legacy connectors as a list of peers to discover from making the connection profile redundant (A future enhancement) Would appreciate your thoughts on this

davidkel (Wed, 19 Jan 2022 11:00:44 GMT):
Hi @klenik I've been thinking about how the new fabric 2.4 gateway peer connector can be used in caliper and I think from an external point of view the way it could be done is 1. the new connector is selected when you bind to fabric:2.4 - if you want to benchmark using the legacy sdks you would bind to 2.2 or 1.4. 2. It will only use the new network config format, the old config format cannot be used 3. To determine the peers that will be used for gateway connections a) a connection profile can still be used, but the discover option will have no meaning here b) The peer information can be embedded into the network configuration file under the organisation tree, the format will be ``` peers: - endpoint: host:port tlsCACerts: pem: | pem-data path: .... grpcOptions: ssl-target-name-override: peer0.org.example.com hostnameOverride: peer0.org.example.com ``` Note that you can define more than 1 peer, doing so will allow you to create a round robin load balance against the listed peers for requests (A future enhancement, may not get into the first release) Also this format could be used for the legacy connectors as a list of peers to discover from making the connection profile redundant (A future enhancement) Would appreciate your thoughts on this

klenik (Wed, 19 Jan 2022 12:10:27 GMT):
Hi @davidkel, I completely agree with points 1-2. As for 3, introducing a third network config format might be confusing for the users, even if they just coexist temporarily. Plus the current (new) network config format has an extensive test coverage and maintainable OOP design (thanks to you :tada:) as I recall. So I'm wondering about the following things: 1. Does the current format have some limitations regarding the new SDK (beside logging a warning if the discover options is set to any value)? 2. Also, I get the feeling that the focus of the new connector will be "just" managing identity/signer and peer connection mappings. As I see from the sample codes, creating the gateway itself is a sync call, so I assume it's cheap. The connection, identity and signer object creation looks heavier, but it is done only once per peers and identities, independently. Does that mean that we can create a full (Identity X Peer) mapping with gateways (since we don't know what combinations the workloads require)? Or what is the envisioned usage pattern of the SDK? 3. Providing information to the SDK from the network configuration is one thing, we should also expose the functionalities to the users in a friendly manner. Like how we can specify smartContractIDs right now instead of always specifying the channel-chaincode combination, which is Fabric-specific. Same with exposing identities and target nodes (which would require a `name` attribute in your example config). Writing cross-platform workloads is hard (if not impossible at the current scene of DLTs), but I think we should strive towards transparent(ish) and flexible workloads as much as possible. 4. When you say "making the connection profile redundant", do you mean that the network configuration file won't reference a connection profile, since the SDK also dropped this part? So basically we'll have our own Fabric "connection profile", which is the network configuration file itself. So these are my hasty thoughts :sweat_smile: In conclusion, I'd try to avoid introducing too much change into the network configuration schema, unless it's for overcoming a limitation. Once we see the requirements of the different SDK versions (and drop the SUT init support), we can come up with a single clean schema for all versions (hopefully). Any thoughts on my thoughts? :)

davidkel (Wed, 19 Jan 2022 12:15:27 GMT):
ahh, sorry I think I confused you when I said new config format. As we have 2 config formats in caliper, I referred to the legacy one as the old one and the latest one as the new one. My plan would be to extend the format that is current documented one with this new entry under organisations eg ``` name: Calier test version: "2.0.0" caliper: blockchain: fabric channels: - channelName: mychannel contracts: - id: basic organizations: - mspid: Org1MSP identities: certificates: - name: 'User1' clientPrivateKey: path: '../fabric-samples/test-network/organizations/peerOrganizations/org1.example.com/users/User1@org1.example.com/msp/keystore/priv_sk' clientSignedCert: path: '../fabric-samples/test-network/organizations/peerOrganizations/org1.example.com/users/User1@org1.example.com/msp/signcerts/User1@org1.example.com-cert.pem' peers: - endpoint: host:port tlsCACerts: pem: | pem-data path: .... grpcOptions: ssl-target-name-override: peer0.org.example.com hostnameOverride: peer0.org.example.com ```

davidkel (Wed, 19 Jan 2022 12:17:57 GMT):
In the above we drop the connection profile reference and declare the peer directly. I would still like to support connection profiles because that is what test-network still generates and it's convenient, so if you don't want to copy stuff into the network config and you have a connection profile then that connection profile can still be used with the new connector

davidkel (Wed, 19 Jan 2022 12:25:12 GMT):
2. is a good point. The prototype kept the pattern currently used by current sdks to ensure everything was prepared upfront in the same way. As the pattern of the gateway is still generally the same I think this still applies but will take a look 3. I would love to remove the channels section and actually discover it, but that there is no api from the fabric team to actually do that and it definitely won't be in the new node connector. We could write one ourselves though to do this

davidkel (Wed, 19 Jan 2022 12:25:12 GMT):
2. is a good point. The prototype kept the pattern currently used by current sdks to ensure everything was prepared upfront in the same way. As the pattern of the gateway is still generally the same I think this still applies but will take a look 3. I would love to remove the channels section and actually discover it, but that there is no api from the fabric node-sdkto actually do that and it definitely won't be in the new node connector. We could write one ourselves though to do this

davidkel (Wed, 19 Jan 2022 12:25:12 GMT):
2. is a good point. The prototype kept the pattern currently used by current sdks to ensure everything was prepared upfront in the same way. As the pattern of the gateway is still generally the same I think this still applies but will take a look 3. I would love to remove the channels section and actually discover it, but that there is no api from the fabric node-sdk to actually do that and it definitely won't be in the new thin sdks. We could write one ourselves though to do this

klenik (Wed, 19 Jan 2022 13:13:24 GMT):
[ ](https://chat.hyperledger.org/channel/caliper-contributors?msg=Pav5dXRMqcS4M9g5D) The "copy stuff" part sounds like a good feature candidate for the generator :) give it some paths (connection profiles, crypto-config dir, wallet dir, docker-compose yaml, etc), and it outputs a network config file

davidkel (Wed, 19 Jan 2022 13:15:49 GMT):
I think we may see connection profiles being phased out completely, the intention of them was right, but the implementation even across versions of the node-sdk made them into a mess

klenik (Wed, 19 Jan 2022 13:15:55 GMT):
[ ](https://chat.hyperledger.org/channel/caliper-contributors?msg=gtMdbcZi8fGFaYvaz) For 3: well, we could just say, that the user specifies the channel-chaincode pair for TXs, and we let the SDK/peer detect errors (peer not in channel, chaincode not in channel, etc). These are errors that needs to be fixed once, not like unexpected errors during the TX life-cycle that need to be handled gracefully

davidkel (Wed, 19 Jan 2022 13:18:38 GMT):
if we want to have contract instances created up front we have to know the channel and chaincode otherwise we would need to build the network and contract instance during the first workload call and I don't know how expensive those calls are at the moment I would have to look at the thin client implementation

klenik (Wed, 19 Jan 2022 13:20:55 GMT):
I assumed that they're just decorators to add the chaincode and channel name to the proposal, in which case lazy creation wouldn't be costly. At least I hope that's the case :)

davidkel (Wed, 19 Jan 2022 13:24:44 GMT):
I've had a quick look and those calls don't look expensive, so we could drop the channel/contractid from the network config in the new connector

klenik (Wed, 19 Jan 2022 13:30:22 GMT):
That's good news. And if we pass wrong parameters, then it'll be caught only on the peer side?

davidkel (Wed, 19 Jan 2022 13:32:12 GMT):
I'd have to try it to see what sort of potentially horrible error message you might get. It would also mean that workloads would now have to declare the channel as well though as at the moment it works it out from the contractid declaration in the network config

klenik (Wed, 19 Jan 2022 14:17:10 GMT):
Yeah, but maybe it's not the worst idea to push the channel/chaincode data to the workload, it should be a workload parameter anyway. The same workload can target any network (given by the network config), it just needs some additional metadata (like channel and cc name). So I'm fine with this trade-off. Plus this way we could clean up the benchmark repo a bit, no more excessive network-cc paired configs.

klenik (Wed, 19 Jan 2022 14:22:49 GMT):
[ ](https://chat.hyperledger.org/channel/caliper-contributors?msg=dq4DykeFXaZuQp6aB) Now that you mention error messages. We really need to make it clearer where error messages originated from. Even by tagging them as [Caliper/Connector Error] or [Fabric SDK Error] or something like that. This way we can delegate error-related questions easier to the proper channels.

davidkel (Wed, 19 Jan 2022 14:23:58 GMT):
That sounds like a good idea to determine where error messages come from

davidkel (Wed, 19 Jan 2022 14:25:05 GMT):
We can push the channel to the workload, do we want to do that for all the fabric connectors or just the new peer gateway connector ?

davidkel (Wed, 19 Jan 2022 14:25:05 GMT):
We can push the channel to the workload, do we want to do that for all the fabric connectors or just the new peer gateway connector ? Happy to do it for all the connectors and simplify them

davidkel (Wed, 19 Jan 2022 14:25:05 GMT):
We can push the channel to the workload, do we want to do that for all the fabric connectors or just the new peer gateway connector ? Not sure about the current connectors as these are expensive operations in the legacy sdks

davidkel (Wed, 19 Jan 2022 14:31:30 GMT):
ie in the legacy connectors there would be a delay in the initial workload to create the network and contract objects

klenik (Wed, 19 Jan 2022 14:33:22 GMT):
I think we can leave the current connectors as is, until we fully understand the impact of this change. Can we query channel and chaincode information with the previous v2 SDK? I know the v1 SDKs allow it, but not sure what kind of functionalities were removed from the v2 SDK

davidkel (Wed, 19 Jan 2022 14:43:52 GMT):
v2 sdk doesn't have explicit apis but it should be possible to make discovery apis that are available and there would be similar in V1 to determine what's available. Discovery would be the way to do it

davidkel (Wed, 19 Jan 2022 14:43:52 GMT):
v2 sdk doesn't have explicit apis but it should be possible to make discovery apis that are available in the fabric-common module and there would be similar in V1 to determine what's available. Discovery would be the way to do it

klenik (Wed, 19 Jan 2022 14:45:40 GMT):
Ah yeah, I forgot about discovery :sweat_smile: Then I think it's possible to gather the available channels/chaincodes (based on some seed peers) and construct the gateways for them in the init phase, right?

davidkel (Wed, 19 Jan 2022 14:49:38 GMT):
Yes I think so, it was on my list of things I wanted to do for the connectors in the future.

davidkel (Wed, 19 Jan 2022 14:50:30 GMT):
The only downside I can see if that it will discover everything even if you aren't interested in it (eg old channels, old chaincodes, ones not under test etc)

davidkel (Wed, 19 Jan 2022 14:50:30 GMT):
The only downside I can see is that it will discover everything even if you aren't interested in it (eg old channels, old chaincodes, ones not under test etc)

klenik (Wed, 19 Jan 2022 14:53:56 GMT):
Well, that's true. But ideally performance testing is done on a clean SUT in development time. So those extra unused connections shouldn't be an issue, even if there are channel/chaincode remnants from previous tests. We can aim for the chaos engineering-style testing of running networks later :D

davidkel (Wed, 19 Jan 2022 14:55:59 GMT):
Excellent, so we will move the channel definition to the workload for the new peer gateway connector and as a later piece of work remove it from the v1 and v2 connectors using discovery

klenik (Wed, 19 Jan 2022 14:56:26 GMT):
Sounds like a plan :thumbsup:

davidkel (Wed, 19 Jan 2022 15:00:17 GMT):
So the network config for the new connector would now be ``` name: Calier test version: "2.0.0" caliper: blockchain: fabric organizations: - mspid: Org1MSP identities: certificates: - name: 'User1' clientPrivateKey: path: '../fabric-samples/test-network/organizations/peerOrganizations/org1.example.com/users/User1@org1.example.com/msp/keystore/priv_sk' clientSignedCert: path: '../fabric-samples/test-network/organizations/peerOrganizations/org1.example.com/users/User1@org1.example.com/msp/signcerts/User1@org1.example.com-cert.pem' peers: - endpoint: host:port tlsCACerts: pem: | pem-data path: .... grpcOptions: ssl-target-name-override: peer0.org.example.com hostnameOverride: peer0.org.example.com ``` or instead of peers you can still use a connection profile ``` path: './Org2ConnectionProfile.json' discover: true ```

davidkel (Wed, 19 Jan 2022 15:00:17 GMT):
So the network config for the new connector would now be ``` name: Calier test version: "2.0.0" caliper: blockchain: fabric organizations: - mspid: Org1MSP identities: certificates: - name: 'User1' clientPrivateKey: path: '../fabric-samples/test-network/organizations/peerOrganizations/org1.example.com/users/User1@org1.example.com/msp/keystore/priv_sk' clientSignedCert: path: '../fabric-samples/test-network/organizations/peerOrganizations/org1.example.com/users/User1@org1.example.com/msp/signcerts/User1@org1.example.com-cert.pem' peers: - endpoint: host:port tlsCACerts: pem: | pem-data path: .... grpcOptions: ssl-target-name-override: peer0.org.example.com hostnameOverride: peer0.org.example.com ``` or instead of peers you can still use a connection profile ``` connectionprofile: path: './Org2ConnectionProfile.json' discover: true ```

davidkel (Wed, 19 Jan 2022 15:00:17 GMT):
So the network config for the new connector would now be ``` name: Calier test version: "2.0.0" caliper: blockchain: fabric organizations: - mspid: Org1MSP identities: certificates: - name: 'User1' clientPrivateKey: path: '../fabric-samples/test-network/organizations/peerOrganizations/org1.example.com/users/User1@org1.example.com/msp/keystore/priv_sk' clientSignedCert: path: '../fabric-samples/test-network/organizations/peerOrganizations/org1.example.com/users/User1@org1.example.com/msp/signcerts/User1@org1.example.com-cert.pem' peers: - endpoint: host:port tlsCACerts: pem: | pem-data path: .... grpcOptions: ssl-target-name-override: peer0.org.example.com hostnameOverride: peer0.org.example.com ``` or instead of peers you can still use a connection profile ``` connectionprofile: path: './Org2ConnectionProfile.json' ```

klenik (Wed, 19 Jan 2022 15:06:06 GMT):
1) Is it (or will it be) possible to target a specific peer from the workload module? If yes, how? 2) I thought you said that `discover` is meaningless (or nonexistent) in this version. Did I misunderstood something?

davidkel (Wed, 19 Jan 2022 15:09:05 GMT):
1) For the new connector do you think there would be a benefit to target a specific peer if more than one was listed ? 2) yep you are right, discover would be meaningless :-) I will update the example

klenik (Wed, 19 Jan 2022 15:14:15 GMT):
1) Hmm, probably not, since the gateway peer will manage the actual life-cycle, right?

klenik (Wed, 19 Jan 2022 15:14:27 GMT):
Including the automatic load balancing, etc

davidkel (Wed, 19 Jan 2022 15:15:40 GMT):
It manages getting the right number of endorsements and should balance that across peers (as I raised the issue that during development it didn't do that :-) )

vlaca (Wed, 19 Jan 2022 16:57:04 GMT):
Has joined the channel.

klenik (Thu, 20 Jan 2022 09:46:14 GMT):
@davidkel I've submitted the quarterly project report (with some delay :sweat_smile:), feel free to extend it however you deem necessary. https://wiki.hyperledger.org/display/TSC/2021+Q4+Hyperledger+Caliper

davidkel (Thu, 20 Jan 2022 13:09:28 GMT):
That looked good to me, thanks @klenik

davidkel (Thu, 20 Jan 2022 15:24:56 GMT):
Hi @klenik I've created 2 github issues on caliper-benchmarks https://github.com/hyperledger/caliper-benchmarks/issues/152 (fabric networks) https://github.com/hyperledger/caliper-benchmarks/issues/153 (node chaincode assets) to try to rationalise the current benchmark repo, would be great to get your thoughts. I know we have some published reports there as well and I was thinking we leave them as is. I know that I won't be adding any further reports to this and that they refer to old and non-lts released versions but there may still be some value there to leave them. What do you think ?

klenik (Thu, 20 Jan 2022 21:02:37 GMT):
@davidkel This are just quick initial thoughts about the above issues, I'll meditate on them a bit more during the weekend. 1) Most of it should be removed. The reports could stay, it's a good write-up and guide for others. But the underlying artifacts can go too. I'd keep a docker-compose network for v1-latest and v2-lts, so users can easily experiment with them within a single repository (I'd vote for a 2org1peer1raft_goleveldb network, just to be lightweight, or even 1org). Other than this, I'd provide network configs for the official Fabric example networks, and workloads for the example CCs. The integration of 3rd party deploy tooling is a good question. Minifab looks promising and active. 2) Can't we extract the business logic (or the contract class itself) into a common "package"? Although not sure the packaging/install step would pick it up using `file:///` style dependencies. What if we provide 2 scripts that rewrite the package.json as needed (and we keep a single implementation, if it's really the same for both versions)? Or maybe we could only showcase v2 Fabric examples in the benchmark repo :man_shrugging_light_skin_tone: Caliper itself is backward compatible, but the benchmark repo is there to help new users start a test quickly. And new users probably want to start with a new Fabric version :) This choice could decide a lot of your question :)

klenik (Thu, 20 Jan 2022 21:02:37 GMT):
@davidkel These are just quick initial thoughts about the above issues, I'll meditate on them a bit more during the weekend. 1) Most of it should be removed. The reports could stay, it's a good write-up and guide for others. But the underlying artifacts can go too. I'd keep a docker-compose network for v1-latest and v2-lts, so users can easily experiment with them within a single repository (I'd vote for a 2org1peer1raft_goleveldb network, just to be lightweight, or even 1org). Other than this, I'd provide network configs for the official Fabric example networks, and workloads for the example CCs. The integration of 3rd party deploy tooling is a good question. Minifab looks promising and active. 2) Can't we extract the business logic (or the contract class itself) into a common "package"? Although not sure the packaging/install step would pick it up using `file:///` style dependencies. What if we provide 2 scripts that rewrite the package.json as needed (and we keep a single implementation, if it's really the same for both versions)? Or maybe we could only showcase v2 Fabric examples in the benchmark repo :man_shrugging_light_skin_tone: Caliper itself is backward compatible, but the benchmark repo is there to help new users start a test quickly. And new users probably want to start with a new Fabric version :) This choice could decide a lot of your question :)

davidkel (Fri, 21 Jan 2022 10:34:41 GMT):
I would like to encourage the use of hyperledger fabric 2.x now as 1.4 is now out of LTS so no new releases for core fabric. My preference would be to showcase 2.x and remove 1.4 support. We could create a branch so that if people need assets that still work on 1.4 then at least there is a "use at your own risk" branch ?

klenik (Fri, 21 Jan 2022 12:24:27 GMT):
Well, the repo is tagged for each release, so we wouldn't lose older assets. So I'm all for keeping only v2 stuff there

davidkel (Fri, 21 Jan 2022 14:39:37 GMT):
Excellent, that great

nao (Wed, 26 Jan 2022 13:18:52 GMT):
Has joined the channel.

davidkel (Wed, 26 Jan 2022 14:30:19 GMT):
@klenik I've been diving into the networks in caliper-benchmarks more and I think I've come to the conclusion that we should get rid of all of them and reference in the first instance test-network from fabric-samples with a suitable network config and then move onto something like mini-fabric for more custom networks in the future (as well as perhaps supporting test-network-nano-bash and test-network-k8s from fabric-samples). The problem is that the networks don't only construct a network but also create a channel and deploy chaincode. This means they can only work with the non-gateway 1.4 SUT. As we don't want to have the fabric connector in caliper doing admin type activities, if we leave any networks there then we would have to leave it as an exercise to the user on how to create a channel and deploy chaincode which is not a trivial thing to do (test-network proves that with all the scripting it does). What do you think ?

davidkel (Thu, 27 Jan 2022 14:06:45 GMT):
I've put the detail now in https://github.com/hyperledger/caliper-benchmarks/issues/152

klenik (Thu, 27 Jan 2022 16:46:42 GMT):
@davidkel Added my thoughts there

davidkel (Thu, 27 Jan 2022 18:15:57 GMT):
@klenik I looked in the issue but couldn't see a comment from you

klenik (Fri, 28 Jan 2022 23:21:39 GMT):
@davidkel Because I forgot to click the Comment button :man_facepalming_light_skin_tone: But now it's there :)

klenik (Tue, 01 Feb 2022 12:52:28 GMT):
The project proposal submission for this year's mentorship has started ([wiki](https://wiki.hyperledger.org/display/INTERN#HyperledgerMentorshipProgram-2022ProgramTimeline*)). It would be nice to gather and discuss some project ideas that would benefit the Caliper community, achievable in 3 months (not a lot of time when you need to get to know Caliper first, assuming full-time mentorship schedule), and more or less loosely coupled with the current development of Caliper (our current TODOs should not block the mentees work). Once we gather some ideas, we can discuss them in next week's call in more detail

klenik (Tue, 01 Feb 2022 13:03:10 GMT):
Idea#1: Hyperledger Caliper VS Code Extension Inspired by the help provided by the IBM Blockchain extension for Fabric, we could do something similar for Caliper to: 1. Automatically setup skeleton projects 2. Provide schemas for different configuration files (helpful when editing YAMLs directly) 3. Provide VS Code webviews (GUIs) for creating/editing configuration files, so users can operate on a higher abstraction level 4. Whatever we can make easier/automatic for users

davidkel (Tue, 01 Feb 2022 15:37:52 GMT):
@klenik anything to make caliper more easily consumable is definitely a good thing. If we do a VS Code extension then maybe that should be in a labs repo rather than the main caliper repo. My concern is that it runs the risk of going stale and writing VSCode extensions is a specific skill set that would require the contributor to maintain

klenik (Tue, 01 Feb 2022 22:27:20 GMT):
@davidkel Yes, it's something to think about. Although the benchmark repo suffers from the same staleness problem, even though (or maybe because) it's separate. The closer we are to the code base, the more incentive we have to keep things updated. I'm hopeful that we could engineer extensions in a way that keeps the vscode coupling part fairly separated. (Plus I wouldn't mind picking up some vscode extension skills, I think it's a great integration opportunity for Caliper on the long term, hence the project suggestion)

klenik (Tue, 01 Feb 2022 22:52:16 GMT):
My other project idea would be the complete overhaul of our CI tests, including migration to GA, sharding the bigger tests, conditional runs based on modifications, etc

klenik (Tue, 01 Feb 2022 22:53:25 GMT):
It wouldn't be a new feature, but an exciting and relevant devops technological challenge

davidkel (Tue, 01 Feb 2022 23:01:49 GMT):
That is a great idea for a project

sbohanlf (Mon, 07 Feb 2022 18:29:25 GMT):
Has joined the channel.

rjones (Sat, 12 Feb 2022 20:59:48 GMT):
@davidkel @klenik may I archive the caliper channels and export them? we have a category [over on discord](https://discord.com/channels/905194001349627914/941417722946924554)

davidkel (Mon, 14 Feb 2022 11:17:54 GMT):
@rjones I'm ok with that, just need the ok from @klenik as well

rjones (Mon, 14 Feb 2022 16:36:57 GMT):
OK.

rjones (Tue, 15 Feb 2022 01:14:55 GMT):
Has left the channel.

davidkel (Wed, 16 Feb 2022 17:25:48 GMT):
@rjones Just to confirm, spoken to @klenik we can archive the caliper channels and export them

rjones (Wed, 16 Feb 2022 18:09:35 GMT):
Has joined the channel.

rjones (Wed, 16 Feb 2022 18:09:35 GMT):
Please join us on Discord: https://discord.gg/hyperledger

rjones (Wed, 16 Feb 2022 18:10:01 GMT):
Has left the channel.

rjones (Wed, 23 Mar 2022 17:24:15 GMT):