baohua (Mon, 06 Feb 2017 06:10:37 GMT):
User User_1 added by baohua.
baohua (Mon, 06 Feb 2017 06:10:48 GMT):
cello project here :)
baohua (Mon, 06 Feb 2017 06:24:00 GMT):
User User_2 added by baohua.
Amjadnz (Mon, 06 Feb 2017 06:24:59 GMT):
Hi Yang
Amjadnz (Mon, 06 Feb 2017 06:25:10 GMT):
just restarting the server now
baohua (Mon, 06 Feb 2017 06:26:11 GMT):
:)
baohua (Mon, 06 Feb 2017 06:35:35 GMT):
cello
Amjadnz (Mon, 06 Feb 2017 06:35:41 GMT):
bash scripts/redeploy.sh dashboard
Remove the old image
Error response from daemon: No such image: cello-dashboard:latest
Rebuilding the cello-dashboard image
Building dashboard
Step 1/8 : FROM python:3.5
---> 0c95b8a497f3
Step 2/8 : MAINTAINER Baohua Yang <"baohyang@cn.ibm.com">
---> Using cache
---> 4e54f72afc99
Step 3/8 : ENV TZ Asia/Shanghai
---> Using cache
---> 45f16eadaf6b
Step 4/8 : WORKDIR /app
---> Using cache
---> ce77d0ddba7b
Step 5/8 : COPY ./requirements.txt /app
---> Using cache
---> b49b75e4dd9c
Step 6/8 : RUN pip install --no-cache-dir -i http://pypi.douban.com/simple/ --trusted-host pypi.douban.com -r requirements.txt
---> Using cache
---> 16f59cb41a3d
Step 7/8 : COPY . /app
---> d779ab975ddc
Removing intermediate container 34d75c220200
Step 8/8 : CMD python dashboard.py
---> Running in b3cb82a0a0ee
---> 8631871080d3
Removing intermediate container b3cb82a0a0ee
Successfully built 8631871080d3
Remove the old containers
WARNING: --all flag is obsolete. This is now the default behavior of `docker-compose rm`
No stopped containers
Redeploy the cello-dashboard container
Creating dashbard
baohua (Mon, 06 Feb 2017 06:35:45 GMT):
Cello project
baohua (Mon, 06 Feb 2017 06:36:33 GMT):
Then what's the log of the dashboard container?
Amjadnz (Mon, 06 Feb 2017 06:37:42 GMT):
dashbard | Traceback (most recent call last):
dashbard | File "dashboard.py", line 4, in
Amjadnz (Mon, 06 Feb 2017 06:37:46 GMT):
I guess same
baohua (Mon, 06 Feb 2017 06:39:09 GMT):
aha, seems in the image building, you're using cache.
baohua (Mon, 06 Feb 2017 06:39:28 GMT):
can u help remove all those related images and try redeploy again?
Amjadnz (Mon, 06 Feb 2017 06:40:08 GMT):
you mean use docker rmi
baohua (Mon, 06 Feb 2017 06:40:12 GMT):
yeap
Amjadnz (Mon, 06 Feb 2017 06:40:45 GMT):
ok
Amjadnz (Mon, 06 Feb 2017 06:41:56 GMT):
Removed all now
Amjadnz (Mon, 06 Feb 2017 06:42:05 GMT):
using sudo make setup again
baohua (Mon, 06 Feb 2017 06:42:52 GMT):
sure, let's see what happens
Amjadnz (Mon, 06 Feb 2017 06:52:07 GMT):
Its setup stage now. It would take some time.
Amjadnz (Mon, 06 Feb 2017 06:52:27 GMT):
Meanwile I did a `sudo rm -rf /var/lib/docker` and docker restart to start afresh
Amjadnz (Mon, 06 Feb 2017 06:53:09 GMT):
just to rule out any old images issue
Amjadnz (Mon, 06 Feb 2017 06:53:11 GMT):
cache
baohua (Mon, 06 Feb 2017 06:53:20 GMT):
can i know the docker version u're using?
Amjadnz (Mon, 06 Feb 2017 06:54:23 GMT):
`Client:
Version: 1.13.0
API version: 1.25
Go version: go1.7.3
Git commit: 49bf474
Built: Tue Jan 17 09:58:06 2017
OS/Arch: linux/amd64
Server:
Version: 1.13.0
API version: 1.25 (minimum version 1.12)
Go version: go1.7.3
Git commit: 49bf474
Built: Tue Jan 17 09:58:06 2017
OS/Arch: linux/amd64
Experimental: false`
baohua (Mon, 06 Feb 2017 06:55:34 GMT):
ok, i've never tested on 1.13, currently we are still using 1.12. But i guess the diff is minor.
Amjadnz (Mon, 06 Feb 2017 06:56:21 GMT):
very excited about cello - hope it all goes well
Amjadnz (Mon, 06 Feb 2017 06:56:52 GMT):
sudo make restart now
baohua (Mon, 06 Feb 2017 06:56:56 GMT):
aha, no worry, it's been deployed widely and should have no big issue :)
Amjadnz (Mon, 06 Feb 2017 07:00:08 GMT):
Seems the issue is still the same.
Amjadnz (Mon, 06 Feb 2017 07:00:19 GMT):
Putting the logs now - they are a bit big
Amjadnz (Mon, 06 Feb 2017 07:01:09 GMT):
dashbard | ImportError: cannot import name 'Client'
dashbard | Traceback (most recent call last):
dashbard | File "dashboard.py", line 4, in
Amjadnz (Mon, 06 Feb 2017 07:01:23 GMT):
all the modules are having the same message
Amjadnz (Mon, 06 Feb 2017 07:01:35 GMT):
cannot import name 'Client'
baohua (Mon, 06 Feb 2017 07:01:47 GMT):
can u edit the local src/requirements.txt and add a docker-py line manually? Then rebuild the images and restart
Amjadnz (Mon, 06 Feb 2017 07:01:56 GMT):
sure
Amjadnz (Mon, 06 Feb 2017 07:02:06 GMT):
which version of docker-py shall I try
baohua (Mon, 06 Feb 2017 07:02:37 GMT):
let's just ignore the version number first.
baohua (Mon, 06 Feb 2017 07:02:47 GMT):
which will try pulling the latest one.
Amjadnz (Mon, 06 Feb 2017 07:02:59 GMT):
so just put docker-py in the end
Amjadnz (Mon, 06 Feb 2017 07:03:10 GMT):
docker-compose>=1.7.0
Flask>=0.11.0
greenlet>=0.4.5
gunicorn>=19.0.0
pymongo>=3.2.0
requests>=2.0.0
docker-py
baohua (Mon, 06 Feb 2017 07:03:30 GMT):
yeap, pls paste the log of redeploy
Amjadnz (Mon, 06 Feb 2017 07:03:35 GMT):
sure
Amjadnz (Mon, 06 Feb 2017 07:04:33 GMT):
shall I try with `sudo rm -rf /var/lib/docker`
Amjadnz (Mon, 06 Feb 2017 07:04:37 GMT):
or just rebuild
Amjadnz (Mon, 06 Feb 2017 07:04:40 GMT):
dadhboard
baohua (Mon, 06 Feb 2017 07:04:41 GMT):
just rebuild
Amjadnz (Mon, 06 Feb 2017 07:04:44 GMT):
ok
Amjadnz (Mon, 06 Feb 2017 07:05:31 GMT):
make redeploy service=dashboard
Amjadnz (Mon, 06 Feb 2017 07:08:11 GMT):
now a new messages comes up
Amjadnz (Mon, 06 Feb 2017 07:08:14 GMT):
dashbard | Traceback (most recent call last):
dashbard | File "dashboard.py", line 4, in
Amjadnz (Mon, 06 Feb 2017 07:08:38 GMT):
it is using python3.5
baohua (Mon, 06 Feb 2017 07:10:22 GMT):
can u try get the version of compose inside the container?
baohua (Mon, 06 Feb 2017 07:10:31 GMT):
seems it has issues
Amjadnz (Mon, 06 Feb 2017 07:11:21 GMT):
I cannot login to container itself it is in "Restarting" mode
Amjadnz (Mon, 06 Feb 2017 07:11:35 GMT):
also I guess I saw similar bug before
Amjadnz (Mon, 06 Feb 2017 07:11:37 GMT):
pip install docker-py
pip install docker-compose
Amjadnz (Mon, 06 Feb 2017 07:11:41 GMT):
this seems to fix it
Amjadnz (Mon, 06 Feb 2017 07:11:47 GMT):
shall I try this in the requirements.txt
Amjadnz (Mon, 06 Feb 2017 07:11:53 GMT):
docker-compose
baohua (Mon, 06 Feb 2017 07:12:01 GMT):
i know, u can just start one container with that image using docker run -it xxx_image bash
Amjadnz (Mon, 06 Feb 2017 07:12:25 GMT):
ok I would try now
baohua (Mon, 06 Feb 2017 07:12:36 GMT):
Here're the original requirements.txt content: ```docker-compose>=1.7.0
Flask>=0.11.0
greenlet>=0.4.5
gunicorn>=19.0.0
pymongo>=3.2.0
requests>=2.0.0```
baohua (Mon, 06 Feb 2017 07:12:50 GMT):
On the top line, we did install docker-compose
baohua (Mon, 06 Feb 2017 07:13:06 GMT):
i suppose whether the latest docker-compose changes sth
Amjadnz (Mon, 06 Feb 2017 07:15:02 GMT):
`docker-compose version` inside the container
Amjadnz (Mon, 06 Feb 2017 07:15:11 GMT):
ERROR: Dependency conflict: an older version of the 'docker-py' package is polluting the namespace. Run the following command to remedy the issue:
pip uninstall docker docker-py; pip install docker
baohua (Mon, 06 Feb 2017 07:16:48 GMT):
One quick test is to update the top line of the original requirements to 'docker-compose==1.7.0', then try rebuild. to see if it works
Amjadnz (Mon, 06 Feb 2017 07:18:19 GMT):
also I noticed this message
Amjadnz (Mon, 06 Feb 2017 07:18:21 GMT):
sudo: apt-get: command not found
Amjadnz (Mon, 06 Feb 2017 07:18:34 GMT):
I'm running on RH FEDORA
baohua (Mon, 06 Feb 2017 07:19:06 GMT):
aha, then u need to manually install using yum
Amjadnz (Mon, 06 Feb 2017 07:19:47 GMT):
yeah
Amjadnz (Mon, 06 Feb 2017 07:19:52 GMT):
lets try that
Amjadnz (Mon, 06 Feb 2017 07:24:41 GMT):
I'm changing the apt-get to yum and starting setup again
Amjadnz (Mon, 06 Feb 2017 07:25:45 GMT):
Package curl-7.51.0-4.fc25.x86_64 is already installed, skipping.
Package docker-engine-1.13.0-1.fc25.x86_64 is already installed, skipping.
Package python-pip-8.1.2-2.fc25.noarch is already installed, skipping.
Amjadnz (Mon, 06 Feb 2017 07:26:49 GMT):
I would change the docker compose version to check again
Amjadnz (Mon, 06 Feb 2017 07:35:34 GMT):
dashbard | Traceback (most recent call last):
dashbard | File "dashboard.py", line 4, in
Amjadnz (Mon, 06 Feb 2017 07:35:57 GMT):
LogConfig
Amjadnz (Mon, 06 Feb 2017 07:36:01 GMT):
issue
Amjadnz (Mon, 06 Feb 2017 07:40:45 GMT):
this is the `pip list` in the container
Amjadnz (Mon, 06 Feb 2017 07:40:48 GMT):
cached-property (1.3.0)
click (6.7)
docker (2.0.2)
docker-compose (1.7.0)
docker-py (1.10.6)
docker-pycreds (0.2.1)
dockerpty (0.4.1)
docopt (0.6.2)
Flask (0.12)
greenlet (0.4.12)
gunicorn (19.6.0)
itsdangerous (0.24)
Jinja2 (2.9.5)
jsonschema (2.6.0)
MarkupSafe (0.23)
pip (9.0.1)
pymongo (3.4.0)
PyYAML (3.12)
requests (2.13.0)
setuptools (28.8.0)
six (1.10.0)
texttable (0.8.7)
websocket-client (0.40.0)
Werkzeug (0.11.15)
baohua (Mon, 06 Feb 2017 07:42:35 GMT):
Ok, strange, the compose complains about the docker-py it installs. I will try to explore why those two do not match each other.
Amjadnz (Mon, 06 Feb 2017 07:48:17 GMT):
is there a particular docker-py version you are using?
Amjadnz (Mon, 06 Feb 2017 07:48:29 GMT):
like 1.7 or 1.8
Amjadnz (Mon, 06 Feb 2017 07:48:45 GMT):
with docker compose 1.7
Amjadnz (Mon, 06 Feb 2017 07:48:58 GMT):
docker is notorious in breaking with version numbers
Amjadnz (Mon, 06 Feb 2017 07:50:47 GMT):
currently the container is using 1.10.6 of docker-py
baohua (Mon, 06 Feb 2017 07:52:13 GMT):
the docker-py is a dependency by docker-compose, should be automatically matched. I'm trying to find a clean server to have a fresh installation. To see if meet similar issue
baohua (Mon, 06 Feb 2017 08:24:52 GMT):
@Amjadnz sorry for the waiting, i'm still downloading the docker images... bad network...
Amjadnz (Mon, 06 Feb 2017 08:27:05 GMT):
ok
Amjadnz (Mon, 06 Feb 2017 08:27:08 GMT):
sure
Amjadnz (Mon, 06 Feb 2017 08:46:52 GMT):
Dear Yang Any update?
baohua (Mon, 06 Feb 2017 08:52:16 GMT):
not finished yet :(
Amjadnz (Mon, 06 Feb 2017 08:56:50 GMT):
Ok
baohua (Mon, 06 Feb 2017 08:57:14 GMT):
can u try rebuilding with docker-compose==1.7.0?
baohua (Mon, 06 Feb 2017 09:00:27 GMT):
the latest docker-compose 1.10 seems build failed in its repo https://pypi.python.org/pypi/docker-compose/1.10.1
Amjadnz (Mon, 06 Feb 2017 09:01:05 GMT):
ok
Amjadnz (Mon, 06 Feb 2017 09:01:13 GMT):
my current stuff is as follows
Amjadnz (Mon, 06 Feb 2017 09:01:15 GMT):
docker-compose==1.7.0
Flask>=0.11.0
greenlet>=0.4.5
gunicorn>=19.0.0
pymongo>=3.2.0
requests>=2.0.0
docker-py
Amjadnz (Mon, 06 Feb 2017 09:01:30 GMT):
shall I retry
baohua (Mon, 06 Feb 2017 09:01:58 GMT):
i also met the similar problem with latest docker-compose, and i update the requirements with a new fresh installation, it works.
Amjadnz (Mon, 06 Feb 2017 09:02:03 GMT):
removed all images and containers and also the /var/lib/docker
Amjadnz (Mon, 06 Feb 2017 09:02:16 GMT):
ok
baohua (Mon, 06 Feb 2017 09:02:16 GMT):
docker-compose==1.7.0 works with clean installation.
Amjadnz (Mon, 06 Feb 2017 09:02:22 GMT):
ok lets try that
baohua (Mon, 06 Feb 2017 09:02:34 GMT):
docker-compose==1.9.0 also works
baohua (Mon, 06 Feb 2017 09:02:54 GMT):
oh, sorry, 1.9.0 not work
Amjadnz (Mon, 06 Feb 2017 09:03:34 GMT):
I'm trying with 1.7.0
Amjadnz (Mon, 06 Feb 2017 09:03:42 GMT):
woudl update you in 5 mins
Amjadnz (Mon, 06 Feb 2017 09:03:45 GMT):
once its up
Amjadnz (Mon, 06 Feb 2017 09:16:16 GMT):
Another item
Amjadnz (Mon, 06 Feb 2017 09:17:58 GMT):
sudo make redeploy=dashboard
tox
py35 installed: appdirs==1.4.0,cached-property==1.3.0,click==6.7,docker-compose==1.7.0,docker-py==1.10.6,docker-pycreds==0.2.1,dockerpty==0.4.1,docopt==0.6.2,Flask==0.12,greenlet==0.4.12,gunicorn==19.6.0,itsdangerous==0.24,Jinja2==2.9.5,jsonschema==2.6.0,MarkupSafe==0.23,mock==2.0.0,packaging==16.8,pbr==1.10.0,py==1.4.32,pymongo==3.4.0,pyparsing==2.1.10,pytest==3.0.6,PyYAML==3.12,requests==2.13.0,six==1.10.0,texttable==0.8.7,websocket-client==0.40.0,Werkzeug==0.11.15
py35 runtests: PYTHONHASHSEED='3370622082'
py35 runtests: commands[0] | py.test
============================================================================================================================ test session starts =============================================================================================================================
platform linux -- Python 3.5.2, pytest-3.0.6, py-1.4.32, pluggy-0.4.0
rootdir: /home/tts/cello/cello, inifile:
collected 0 items
======================================================================================================================== no tests ran in 0.02 seconds ========================================================================================================================
ERROR: InvocationError: '/home/tts/cello/cello/.tox/py35/bin/py.test'
flake8 installed: appdirs==1.4.0,cached-property==1.3.0,click==6.7,docker-compose==1.7.0,docker-py==1.10.6,docker-pycreds==0.2.1,dockerpty==0.4.1,docopt==0.6.2,flake8==3.2.1,Flask==0.12,greenlet==0.4.12,gunicorn==19.6.0,itsdangerous==0.24,Jinja2==2.9.5,jsonschema==2.6.0,MarkupSafe==0.23,mccabe==0.5.3,packaging==16.8,pycodestyle==2.2.0,pyflakes==1.3.0,pymongo==3.4.0,pyparsing==2.1.10,PyYAML==3.12,requests==2.13.0,six==1.10.0,texttable==0.8.7,websocket-client==0.40.0,Werkzeug==0.11.15
flake8 runtests: PYTHONHASHSEED='3370622082'
flake8 runtests: commands[0] | flake8 --ignore=E402,F401 /home/tts/cello/cello/src
__________________________________________________________________________________________________________________________________ summary ___________________________________________________________________________________________________________________________________
ERROR: py35: commands failed
flake8: commands succeeded
Makefile:14: recipe for target 'check' failed
make: *** [check] Error 1
Amjadnz (Mon, 06 Feb 2017 09:18:22 GMT):
my bad
Amjadnz (Mon, 06 Feb 2017 09:18:34 GMT):
actually it has to be make redeploy service=dashboard
Amjadnz (Mon, 06 Feb 2017 09:18:36 GMT):
doing that npw
Amjadnz (Mon, 06 Feb 2017 09:18:38 GMT):
now
baohua (Mon, 06 Feb 2017 09:19:14 GMT):
this is what i'm using: "make stop; docker rmi cello-dashbard cello-watchdog cello-restserver; make start"
baohua (Mon, 06 Feb 2017 09:19:34 GMT):
with 1.7.0~1.9.0, it works
Amjadnz (Mon, 06 Feb 2017 09:23:02 GMT):
retrying that again now
Amjadnz (Mon, 06 Feb 2017 09:23:11 GMT):
I mean your command
baohua (Mon, 06 Feb 2017 09:23:45 GMT):
with compose==1.10.0, it does not work
Amjadnz (Mon, 06 Feb 2017 09:25:47 GMT):
Now after this change I have the restserver up
Amjadnz (Mon, 06 Feb 2017 09:25:53 GMT):
watchdog up
Amjadnz (Mon, 06 Feb 2017 09:26:00 GMT):
but the dashboard is still in "Restarting mode"
Amjadnz (Mon, 06 Feb 2017 09:26:39 GMT):
dashbard | Traceback (most recent call last):
dashbard | File "dashboard.py", line 4, in
Amjadnz (Mon, 06 Feb 2017 09:30:26 GMT):
shall I get rid of .tox aswell?
Amjadnz (Mon, 06 Feb 2017 09:30:31 GMT):
like make clean
Amjadnz (Mon, 06 Feb 2017 09:41:44 GMT):
I'm trying to remove all images including python and nginx and trying again
Amjadnz (Mon, 06 Feb 2017 09:41:50 GMT):
hope it gives us some new insight
Amjadnz (Mon, 06 Feb 2017 09:42:08 GMT):
atleast with this exercise we could make watchdog and restserver working
Amjadnz (Mon, 06 Feb 2017 09:42:15 GMT):
only dashboard is pending now
david.peyronnin (Mon, 06 Feb 2017 09:47:23 GMT):
Has joined the channel.
Amjadnz (Mon, 06 Feb 2017 10:01:11 GMT):
go all running now
Amjadnz (Mon, 06 Feb 2017 10:01:13 GMT):
All fine
Amjadnz (Mon, 06 Feb 2017 10:02:37 GMT):
Can get to dashboard now
Amjadnz (Mon, 06 Feb 2017 10:02:42 GMT):
thanks man
Amjadnz (Mon, 06 Feb 2017 10:03:00 GMT):
we might have to update the requirements - if somebody needs it down the line
baohua (Mon, 06 Feb 2017 10:32:17 GMT):
yea
baohua (Mon, 06 Feb 2017 10:32:39 GMT):
need to update that, thanks!
baohua (Mon, 06 Feb 2017 10:40:33 GMT):
would u like to take the honor to submit a patchset :grinning:
Amjadnz (Mon, 06 Feb 2017 11:06:06 GMT):
Surely - but its such a small find. Your project is pretty fast.
Amjadnz (Mon, 06 Feb 2017 11:06:37 GMT):
vast and I should have really something substantial to take credit. thanks anyway.
Amjadnz (Mon, 06 Feb 2017 11:07:00 GMT):
but you did all the work. great work by the way.
baohua (Mon, 06 Feb 2017 11:11:00 GMT):
Thanks, it would be nice if you can help fix the problem, and do remember to add the "fix #CE-11" in the git commit msg. Also, I can also help do that if not convenient to u :)
Amjadnz (Mon, 06 Feb 2017 11:14:53 GMT):
Sure no problems
Amjadnz (Mon, 06 Feb 2017 11:27:39 GMT):
Yang - I've done the git commit
Amjadnz (Mon, 06 Feb 2017 11:27:59 GMT):
when I'm trying git review its saying "review" commad not find
Amjadnz (Mon, 06 Feb 2017 11:28:27 GMT):
`git review
git: 'review' is not a git command. See 'git --help'`
Amjadnz (Mon, 06 Feb 2017 11:28:51 GMT):
as per the documents - I have to do a git commit and then git review
baohua (Mon, 06 Feb 2017 11:35:14 GMT):
aha, u may need to install git-review manually, remember it's a python lib, too
baohua (Mon, 06 Feb 2017 11:35:56 GMT):
where's ur patch? cannot find it.
baohua (Mon, 06 Feb 2017 11:36:53 GMT):
i submit in https://gerrit.hyperledger.org/r/#/c/5561/, FYI.
baohua (Mon, 06 Feb 2017 11:37:15 GMT):
pls see the commit msg for reference, i can drop my submit when u've done :)
Amjadnz (Mon, 06 Feb 2017 11:48:21 GMT):
did you get it now
Amjadnz (Mon, 06 Feb 2017 11:48:24 GMT):
I did that change
Amjadnz (Mon, 06 Feb 2017 11:51:21 GMT):
https://gerrit.hyperledger.org/r/#/c/5563/
baohua (Mon, 06 Feb 2017 11:53:41 GMT):
aha, can u see my submit as reference, i suggest we give dependencies version in a range. https://gerrit.hyperledger.org/r/#/c/5561/
baohua (Mon, 06 Feb 2017 11:54:59 GMT):
and the commit log width should be <79 chars :)
Amjadnz (Mon, 06 Feb 2017 12:11:09 GMT):
Shall I do the commit again
Amjadnz (Mon, 06 Feb 2017 12:17:32 GMT):
but you said 1.9.0 is not fine right?
Amjadnz (Mon, 06 Feb 2017 12:18:44 GMT):
so its 1.7 or 1.8
Amjadnz (Mon, 06 Feb 2017 12:18:55 GMT):
i need to check with 1.8 too
baohua (Mon, 06 Feb 2017 12:27:47 GMT):
actually, 1.7.0~1.9.0 works. while 1.10.0 does not work. Pls see my submit as an example. U can just revise the submission with `git commit -a --amend`, after that, `git review` again. See docs on how to contribute.
Amjadnz (Mon, 06 Feb 2017 12:42:03 GMT):
Yea Done. Sorry it took couple of time as this is the first time I was committing to hyperledger
Amjadnz (Mon, 06 Feb 2017 12:42:07 GMT):
starting woes
Amjadnz (Mon, 06 Feb 2017 12:42:15 GMT):
please confirm
Amjadnz (Mon, 06 Feb 2017 12:42:48 GMT):
https://gerrit.hyperledger.org/r/#/c/5565/
Amjadnz (Mon, 06 Feb 2017 12:43:00 GMT):
I needed to do it again with a new number
marcusvcs (Mon, 06 Feb 2017 12:57:59 GMT):
Has joined the channel.
baohua (Mon, 06 Feb 2017 13:23:41 GMT):
sure, thanks, i would like to merge this :)
ericmvaughn (Mon, 06 Feb 2017 21:09:34 GMT):
Has joined the channel.
mgutala (Mon, 06 Feb 2017 21:37:47 GMT):
Has joined the channel.
VolodymyrLebediev (Mon, 06 Feb 2017 21:58:09 GMT):
Has joined the channel.
liutc (Tue, 07 Feb 2017 01:01:34 GMT):
Has joined the channel.
baohua (Tue, 07 Feb 2017 01:33:20 GMT):
@Amjadnz https://gerrit.hyperledger.org/r/#/c/5557/. This patchset help add version range to other dependencies :)
fz (Tue, 07 Feb 2017 01:36:56 GMT):
Has joined the channel.
ray (Tue, 07 Feb 2017 02:16:26 GMT):
Has joined the channel.
genggjh (Tue, 07 Feb 2017 09:22:23 GMT):
Has joined the channel.
baohua (Tue, 07 Feb 2017 09:23:53 GMT):
https://gerrit.hyperledger.org/r/#/c/5633/
MKabanau (Tue, 07 Feb 2017 14:08:24 GMT):
Has joined the channel.
reachk.raj (Tue, 07 Feb 2017 14:12:42 GMT):
Has joined the channel.
cbf (Tue, 07 Feb 2017 15:13:31 GMT):
Has joined the channel.
cyrax (Tue, 07 Feb 2017 16:07:03 GMT):
Has joined the channel.
tbrooke (Tue, 07 Feb 2017 20:20:43 GMT):
Has joined the channel.
beauson45 (Tue, 07 Feb 2017 20:54:35 GMT):
Has joined the channel.
SivaKannan (Tue, 07 Feb 2017 21:06:21 GMT):
Has joined the channel.
mgk (Tue, 07 Feb 2017 22:03:35 GMT):
Has joined the channel.
Adil.B (Tue, 07 Feb 2017 22:10:16 GMT):
Has joined the channel.
mdevilliers (Wed, 08 Feb 2017 01:33:58 GMT):
Has joined the channel.
baohua (Wed, 08 Feb 2017 01:53:18 GMT):
https://gerrit.hyperledger.org/r/#/c/5659/
qiang0723 (Wed, 08 Feb 2017 02:01:12 GMT):
Has joined the channel.
lafenicecc (Wed, 08 Feb 2017 02:10:04 GMT):
Has joined the channel.
baohua (Wed, 08 Feb 2017 02:12:56 GMT):
User User_3 added by baohua.
Rokkinchic (Wed, 08 Feb 2017 03:18:13 GMT):
Has joined the channel.
SzaboZoltan (Wed, 08 Feb 2017 07:34:06 GMT):
Has joined the channel.
SzaboZoltan (Wed, 08 Feb 2017 07:35:57 GMT):
Could you please suggest me the steps with which I will have a working Cello dashboard? So, at first "_deployment.md_", then which patches do I have to apply?
alexliu (Wed, 08 Feb 2017 08:59:39 GMT):
Has joined the channel.
dolanort (Wed, 08 Feb 2017 10:25:39 GMT):
Has joined the channel.
dolanort (Wed, 08 Feb 2017 10:27:40 GMT):
Hi, I ran cello with the docker-compose file in the git repo (without changing anything).
I connected to the dashboard via the default login/pass. But I get a jinja2 exception about index.html template not found
jadhavajay (Wed, 08 Feb 2017 12:55:43 GMT):
Has joined the channel.
jsalvachua (Wed, 08 Feb 2017 13:49:07 GMT):
Has joined the channel.
SzaboZoltan (Wed, 08 Feb 2017 14:18:37 GMT):
^ The same for me. In the "make log": nginx | [error] 10#10: *1 connect() failed (113: No route to host) while connecting to upstream, client: xx.yyy.zz.vv, server: , request: "GET / HTTP/1.1", upstream: "http://172.18.0.5:8080/", host: "34.248.75.220:8080"
o_drugan (Wed, 08 Feb 2017 14:26:57 GMT):
Has joined the channel.
haidong (Wed, 08 Feb 2017 14:52:34 GMT):
Has joined the channel.
jkirke (Wed, 08 Feb 2017 16:34:30 GMT):
Has joined the channel.
angel-ibm (Wed, 08 Feb 2017 16:45:12 GMT):
Has joined the channel.
mikezaccardo (Wed, 08 Feb 2017 21:01:00 GMT):
Has joined the channel.
baohua (Thu, 09 Feb 2017 01:57:58 GMT):
@hightall , can u have a look?
hightall (Thu, 09 Feb 2017 02:23:12 GMT):
@dolanort do you pull the newest code? I just try it in a new env, it's fine.
zhoupeiwen (Thu, 09 Feb 2017 03:20:09 GMT):
Has joined the channel.
xiejunxi (Thu, 09 Feb 2017 03:23:36 GMT):
Has joined the channel.
Honglei (Thu, 09 Feb 2017 03:23:51 GMT):
Has joined the channel.
zemtsov (Thu, 09 Feb 2017 09:10:30 GMT):
Has joined the channel.
danieldewar (Thu, 09 Feb 2017 10:23:59 GMT):
Has joined the channel.
jonreid (Thu, 09 Feb 2017 11:42:10 GMT):
Has joined the channel.
adc (Thu, 09 Feb 2017 12:22:59 GMT):
Has joined the channel.
Ch3ck (Thu, 09 Feb 2017 12:45:54 GMT):
Has joined the channel.
SzaboZoltan (Thu, 09 Feb 2017 13:47:23 GMT):
If I follow https://github.com/hyperledger/cello/blob/master/docs/deployment.md I have the same error. Maybe the problem is that "git clone https://github.com/yeasy/cello" is not perfect? There is "dasbard" (without o) again in cello/docker-compose.yml. The error remained the same: I can not log in to dashboard, it receives login and pwd, but then: Not found.
SzaboZoltan (Thu, 09 Feb 2017 13:48:57 GMT):
(I meant: "dashbard"). A made a screencast while I installed the stuff to a scrach fresh EC2 image. Maybe the user permissons are not perfect also? (If I install everything as root...)
SzaboZoltan (Thu, 09 Feb 2017 14:02:23 GMT):
Now I see that "git clone http://gerrit.hyperledger.org/r/cello" is the better choice. PLEASE CORRECT it in the deployment.md, because it's very frustrating installing again-and-again, with the promise that it should work (and, in fact, it should not, with the "old" cello).
SzaboZoltan (Thu, 09 Feb 2017 14:11:23 GMT):
Now it really works. :-)
zhipengh (Thu, 09 Feb 2017 14:44:13 GMT):
Has joined the channel.
SzaboZoltan (Thu, 09 Feb 2017 19:21:54 GMT):
Please help to start at least one compute node. If I try to start it via 127.0.0.1:2375, it writes in log: The daemon_url is inactive:tcp://127.0.0.1:2375. (If I try docker -H 127.0.0.1:2375 version, it shows itself nicely.)
brett09 (Fri, 10 Feb 2017 01:00:39 GMT):
Has joined the channel.
brett09 (Fri, 10 Feb 2017 01:02:04 GMT):
Hi, just like to say nice work with cello - been playing with it for a couple of days
brett09 (Fri, 10 Feb 2017 01:02:53 GMT):
Some items I have come across - like SzaboZoltan the git reference in the deployment doc is out of date - either http://gerrit.hyperledger.org/r/cello or https://github.com/hyperledger/cello seem to be the primary now
brett09 (Fri, 10 Feb 2017 01:03:09 GMT):
also I had issues deploying chains - compose_up()] - Exception when compose start=.FileNotFoundError: [Errno 2] No such file or directory: './_compose_files/local/cluster-4.yml'
brett09 (Fri, 10 Feb 2017 01:03:43 GMT):
I changed cello/src/_compose_files/syslog to cello/src/_compose_files/local and it worked
baohua (Fri, 10 Feb 2017 01:20:34 GMT):
Thanks @SzaboZoltan would u like to submit a patchset on the new link addr?
baohua (Fri, 10 Feb 2017 01:20:56 GMT):
@brett09 Thanks, will fix this. Seems we miss some files when importing the code to gerrit :(
baohua (Fri, 10 Feb 2017 01:27:12 GMT):
http://gerrit.hyperledger.org/r/5819
Donald Liu (Fri, 10 Feb 2017 04:43:46 GMT):
Has joined the channel.
NIKESHGOGIA (Fri, 10 Feb 2017 07:13:26 GMT):
Has joined the channel.
toddinpal (Fri, 10 Feb 2017 14:11:18 GMT):
Has joined the channel.
toddinpal (Fri, 10 Feb 2017 14:57:21 GMT):
So I'm trying to get cello running in my environment and essentially all the servers die with this error:
toddinpal (Fri, 10 Feb 2017 14:57:23 GMT):
restserver | File "/app/agent/docker_swarm.py", line 10, in
toddinpal (Fri, 10 Feb 2017 14:57:40 GMT):
Not sure what I'm doing wrong... any suggestions would be appreciated
SzaboZoltan (Fri, 10 Feb 2017 17:55:22 GMT):
@baohua Do I have write permissions? :-) [For now, it would be easier if You could do it instead of me.]
SzaboZoltan (Fri, 10 Feb 2017 17:57:51 GMT):
@toddinpal Did you try NOT to use the officially documented "git clone ...cello" but instead of it: git clone http://gerrit.hyperledger.org/r/cello ? Then (or before) delete everything from /var/lib/docker/, pull the images again and it will (hopefully) work.
SzaboZoltan (Fri, 10 Feb 2017 18:00:38 GMT):
@brett09 Could you add more compute nodes also? Which way?
toddinpal (Fri, 10 Feb 2017 18:55:38 GMT):
I used the "documented" way...
baohua (Sat, 11 Feb 2017 00:22:46 GMT):
@SzaboZoltan aha, everyone can submit a patchset to review. it's open-source. Let me know if you want to take the honor to fix, as the bug reporter :)
SzaboZoltan (Sat, 11 Feb 2017 08:00:29 GMT):
Ok, thank you, I'll try it.
SzaboZoltan (Sat, 11 Feb 2017 08:00:29 GMT):
@baohua Ok, thank you, I'll try it.
SzaboZoltan (Sat, 11 Feb 2017 08:00:29 GMT):
@baohua Ok, thank you, I'll take the honor. :-)
baohua (Sat, 11 Feb 2017 13:39:48 GMT):
:grinning: sure, and pls feel free to add me into the reviewers.
toddinpal (Sat, 11 Feb 2017 18:49:23 GMT):
Are the "System Requirements" actually required and/or enforced? I mean for testing/development it seems 8c16g100g is pretty hefty
baohua (Sat, 11 Feb 2017 23:51:10 GMT):
This actually depends on the workload. And the requirement is a recommended one for most cases.
SzaboZoltan (Sun, 12 Feb 2017 06:27:38 GMT):
@baohua @toddinpal But until you get to a working one it's enough a cheap or a free tyer virtual machine also. (The "8c16g100g" costs a LOT on Amazon, I tried for a few hours.)
Amjadnz (Sun, 12 Feb 2017 12:12:25 GMT):
Yang - this is Amjad again. Got all working fine now.
Amjadnz (Sun, 12 Feb 2017 12:12:46 GMT):
Question : we have to deploy a chaincode - and if my understanding is correct
Amjadnz (Sun, 12 Feb 2017 12:12:53 GMT):
I follow these steps
Amjadnz (Sun, 12 Feb 2017 12:13:15 GMT):
1. 4 node setup (1 Master Cello and 3 Worker Nodes)
Amjadnz (Sun, 12 Feb 2017 12:13:31 GMT):
2. Startup Master, All Worker Nodes
Amjadnz (Sun, 12 Feb 2017 12:13:47 GMT):
3. Conenect to dashboard and add all nodes
Amjadnz (Sun, 12 Feb 2017 12:14:28 GMT):
4. On Dashboard add the Active Chains
Amjadnz (Sun, 12 Feb 2017 12:15:02 GMT):
5. Connect to NODE01 and deploy my chain code (REGULAR DEPLOY REST API)
Amjadnz (Sun, 12 Feb 2017 12:15:29 GMT):
6. I should be in a position to see the Nodes get auto populated and in working condition on all ndoes?
Amjadnz (Sun, 12 Feb 2017 12:15:38 GMT):
I'm in 2 minds on step 4
Amjadnz (Sun, 12 Feb 2017 12:15:56 GMT):
Should I create an active chain or it would auto populate
toddinpal (Mon, 13 Feb 2017 00:40:47 GMT):
Regardless of the workload, are the requirements enforced? Those requirements are very substantial. For experimentation it's hard to believe you need an 8 core 16 GB machine.
baohua (Mon, 13 Feb 2017 01:05:37 GMT):
@Amjadnz In step 4, you can find a auto-fill checkbox at the host property. if u check in that, then will fill up with chains automatically. otherwise, need to manually create chain on specific host :)
baohua (Mon, 13 Feb 2017 01:05:56 GMT):
@toddinpal aha, the requirements are not enforced if you just need some simple test :)
toddinpal (Mon, 13 Feb 2017 01:23:31 GMT):
@baohua Thanks! I'll look for other reasons I'm having problems. :-) I'm sure I'm suffering from network problems. I'm running one Virtualbox VM as the Cello master on my laptop and was hoping to add 4 CoreOS Virtualbox VMs as my worker nodes. My plan was to have each VM have a NAT network connection (to get to the outside world if needed) and a host only network connection for inter-node communication. I suspect that is doable, but I also suspect I've not got something configured correctly.
baohua (Mon, 13 Feb 2017 01:24:43 GMT):
yeap, it should work, and i also encourage u help update the doc if you make it work. I never tested at CoreOS, but seems it's interesting :grinning:
baohua (Mon, 13 Feb 2017 01:38:49 GMT):
https://gerrit.hyperledger.org/r/#/c/5905/
eheads (Mon, 13 Feb 2017 07:07:32 GMT):
Has joined the channel.
Amjadnz (Mon, 13 Feb 2017 09:53:32 GMT):
Hi Yang - Please note that there is no documentation of how to deploy a chaincode to the WORKER Node. and get it auto replicated in multiple nodes.
Amjadnz (Mon, 13 Feb 2017 09:53:55 GMT):
Seems there is some issue with my understanding and the available docs are not giving a clear picture.
Amjadnz (Mon, 13 Feb 2017 09:54:10 GMT):
So this is what I did
Amjadnz (Mon, 13 Feb 2017 09:54:28 GMT):
1. Setup the 4 node setup
Amjadnz (Mon, 13 Feb 2017 09:54:44 GMT):
2. Added them one by one to the dashboard - ALL FINE here no issues
Amjadnz (Mon, 13 Feb 2017 09:55:26 GMT):
3. this is starting the 4 containers at WORKER_NODE 01
Amjadnz (Mon, 13 Feb 2017 09:55:43 GMT):
the moment I create a ACTIVE_CHAIN
Amjadnz (Mon, 13 Feb 2017 09:55:58 GMT):
docker ps command output
Amjadnz (Mon, 13 Feb 2017 09:56:26 GMT):
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
99d77916979d hyperledger/fabric-peer:latest "peer node start" 14 minutes ago Up 14 minutes 7050-7059/tcp 58a17f46f113a9000a123ee3_vp2
77ae09fb8ebb hyperledger/fabric-peer:latest "peer node start" 14 minutes ago Up 14 minutes 7050-7059/tcp 58a17f46f113a9000a123ee3_vp3
2f1a66b216b7 hyperledger/fabric-peer:latest "peer node start" 14 minutes ago Up 14 minutes 7050-7059/tcp 58a17f46f113a9000a123ee3_vp1
f87bd0fd9cf3 hyperledger/fabric-peer:latest "peer node start" 14 minutes ago Up 14 minutes 0.0.0.0:7050-7051->7050-7051/tcp, 7052-7059/tcp 58a17f46f113a9000a123ee3_vp0
Amjadnz (Mon, 13 Feb 2017 10:01:28 GMT):
Now I would like to deploy the chaincode
Amjadnz (Mon, 13 Feb 2017 10:01:36 GMT):
so I login to bash the first container
Amjadnz (Mon, 13 Feb 2017 10:01:48 GMT):
sudo docker exec -it 58a17f46f113a9000a123ee3_vp0 bash
Amjadnz (Mon, 13 Feb 2017 10:01:54 GMT):
build my chaincode
Amjadnz (Mon, 13 Feb 2017 10:02:05 GMT):
and exit
Amjadnz (Mon, 13 Feb 2017 10:02:24 GMT):
from the mac I issue a deploy command
Amjadnz (Mon, 13 Feb 2017 10:13:12 GMT):
sorry from the WORKER_NODE 01 - I issue the command
Amjadnz (Mon, 13 Feb 2017 10:25:55 GMT):
peer chaincode deploy -p tts/evote_chaincode -c '{"args": ["init"]}'
Amjadnz (Mon, 13 Feb 2017 10:26:14 GMT):
Output as follows:
Amjadnz (Mon, 13 Feb 2017 10:26:16 GMT):
10:20:07.802 [logging] LoggingInit -> DEBU 001 Setting default logging level to DEBUG for command 'chaincode'
10:20:13.432 [chaincodeCmd] chaincodeDeploy -> INFO 002 Deploy result: type:GOLANG chaincodeID:
Amjadnz (Mon, 13 Feb 2017 10:26:31 GMT):
So far the chaincode is deployed
Amjadnz (Mon, 13 Feb 2017 10:26:44 GMT):
But I'm not able to query it
Amjadnz (Mon, 13 Feb 2017 10:27:48 GMT):
My Question is 2 fold and that we might have to add to the documentation as well
Amjadnz (Mon, 13 Feb 2017 10:28:20 GMT):
1. In traditional hyperledger scenario - I startup using docker-compose with 4 nodes with a yaml file
Amjadnz (Mon, 13 Feb 2017 10:28:43 GMT):
And that is listening on the port 2375 and all nodes are up
Amjadnz (Mon, 13 Feb 2017 10:29:17 GMT):
I use the deploy method to deploy the chaincode and the guest OS Creates a seperate container with the GUID of the vm.
Amjadnz (Mon, 13 Feb 2017 10:29:59 GMT):
So if I had 4 nodes - the guest os now has 5 nodes (4 hyperledger nodes + 1 node for new chaincode)
Amjadnz (Mon, 13 Feb 2017 10:31:18 GMT):
Now if I had 4 machines connected to the network - and they are communicating - the same VM is copied automatically to all the machines and all is fine and dusted. I did all these and there seems to be no confusion there on the HYPERLEDGER front.
Amjadnz (Mon, 13 Feb 2017 10:32:14 GMT):
Question 1: In CELLO the dashboard server is started up by using "sudo make restart" - all nodes are up *dashboard, *nginx, *mongodb, *watchdog , *restserver
Amjadnz (Mon, 13 Feb 2017 10:32:47 GMT):
So it does not involve in any chaincode deployment - correct?
Amjadnz (Mon, 13 Feb 2017 10:35:22 GMT):
Question 2: In CELLO the worker node is suppose to have the chaincode. But the worker node is actually creating 4 VMs with same ID 58a17f46f113a9000a123ee3_vp0 , vp1, vp2 and vp3 - where should the chaincode go
Amjadnz (Mon, 13 Feb 2017 10:36:00 GMT):
Sorry for my long post - but these show my excitement with cello I guess :grinning:
Amjadnz (Mon, 13 Feb 2017 10:39:12 GMT):
So when I do a CURL to query
Amjadnz (Mon, 13 Feb 2017 10:39:14 GMT):
curl -XPOST -d '{"jsonrpc":"2.0","method":"query","params":{ "type":1,"chaincodeID":{ "name":"7c870cd9b6ec0659f3ff809b73e70ce54c138b401caa53a264203eb3933ddcf462211aa38513bdede7838ab00cabad90b80f4054bafe583ae9b37a2b52eed4d5"},"ctorMsg":{ "args":[ "queryAccounts","a"]},"metadata":[ 97,115,115,105,103,110,101,114],"secureContext":"assigner"},"id": 301}' http://172.18.0.5:7050/chaincode
Amjadnz (Mon, 13 Feb 2017 10:39:35 GMT):
Output:
Amjadnz (Mon, 13 Feb 2017 10:39:38 GMT):
{"jsonrpc":"2.0","error":{"code":-32003,"message":"Query failure","data":"Error when querying chaincode: Error:Failed to launch chaincode spec(Could not get deployment transaction for 7c870cd9b6ec0659f3ff809b73e70ce54c138b401caa53a264203eb3933ddcf462211aa38513bdede7838ab00cabad90b80f4054bafe583ae9b37a2b52eed4d5 - LedgerError - ResourceNotFound: ledger: resource not found)"},"id":301}
Amjadnz (Mon, 13 Feb 2017 13:21:37 GMT):
[ ](https://chat.hyperledger.org/channel/cello?msg=LRxm77m5N52BHLdNP) @brett09 - can you pls help me understand how did you get chaincode - deployed in cello worker node?
Amjadnz (Mon, 13 Feb 2017 13:33:44 GMT):
Guys any one :slight_smile: active?
baohua (Mon, 13 Feb 2017 14:12:08 GMT):
@Amjadnz aha, seems lots of question. In brief, we do not provide a ui to support chaincode deploy/management. We currently only have the admin ui to help manage chains. And certainly any such developer ui is welcome.
Amjadnz (Mon, 13 Feb 2017 15:06:46 GMT):
Sorry guys my bad - actually my vm was missing the baseimage
Amjadnz (Mon, 13 Feb 2017 15:07:18 GMT):
now I got it to download would test and let you know if I run in to issues.
Amjadnz (Mon, 13 Feb 2017 15:40:15 GMT):
[ ](https://chat.hyperledger.org/channel/cello?msg=igHQsc7wW94fDzHQg) @baohua Yeah Yang - got it to work now. I need to check if I can load into multiple instances and call value from there.
Amjadnz (Mon, 13 Feb 2017 15:40:29 GMT):
Should be straight forward - but its not done till its done :-)
Amjadnz (Mon, 13 Feb 2017 16:31:56 GMT):
[ ](https://chat.hyperledger.org/channel/cello?msg=igHQsc7wW94fDzHQg) @baohua another question if you may please
I have a chaincode now deployed on the node0 - how do I get the same chaincode in node1
Is it automated?
I can see 4 containers created by the Node01 once I deploy the chaincode
But Node02 has no references towards it
Chaincode instances containers are created on the SINGLE HOST Node only with no copies on the other nodes? Is my statement correct?
baohua (Tue, 14 Feb 2017 06:49:51 GMT):
correct! @Amjadnz
jamessheen (Tue, 14 Feb 2017 14:45:06 GMT):
Has joined the channel.
Amjadnz (Tue, 14 Feb 2017 15:50:15 GMT):
Thanks man.
Amjadnz (Tue, 14 Feb 2017 15:50:44 GMT):
I thought I did a configuration issue - and it was not mentioned anywhere too.
Amjadnz (Tue, 14 Feb 2017 15:51:08 GMT):
Do you have an idea if this might change in near future as this is a pretty needy feature
Amjadnz (Tue, 14 Feb 2017 15:51:52 GMT):
Or is it community driven
SzaboZoltan (Tue, 14 Feb 2017 17:44:51 GMT):
Can I get an example docker-compose.yml for setting up some (4) worker nodes just for test? With the appropriate ports open? :-)
baohua (Wed, 15 Feb 2017 01:39:29 GMT):
no, that's not the correct way. the worker nodes need to be setup manually before u can use it, as described in the documentation.
baohua (Wed, 15 Feb 2017 09:22:23 GMT):
@SzaboZoltan Why u need to remove the .gitignore file in this change?
baohua (Wed, 15 Feb 2017 09:22:24 GMT):
https://gerrit.hyperledger.org/r/#/c/6013
baohua (Wed, 15 Feb 2017 09:22:34 GMT):
otherwise, LGTM.
baohua (Wed, 15 Feb 2017 09:23:24 GMT):
to revise a patchset, u can update the change, then use `git commit -a --amend`; And then `git review` to submit the revised version.
baohua (Wed, 15 Feb 2017 09:23:37 GMT):
as describe in https://github.com/hyperledger/cello/blob/master/docs/CONTRIBUTING.md
baohua (Wed, 15 Feb 2017 09:23:37 GMT):
as described in https://github.com/hyperledger/cello/blob/master/docs/CONTRIBUTING.md
dolanor (Wed, 15 Feb 2017 14:55:57 GMT):
Has joined the channel.
knagware9 (Thu, 16 Feb 2017 09:28:03 GMT):
Has joined the channel.
wutongtree (Thu, 16 Feb 2017 09:43:52 GMT):
Has joined the channel.
vijaykodam (Thu, 16 Feb 2017 11:55:29 GMT):
Has joined the channel.
alviontaran (Thu, 16 Feb 2017 16:15:42 GMT):
Has joined the channel.
SzaboZoltan (Fri, 17 Feb 2017 14:46:47 GMT):
"...as described in the documentation." I did not find any sentence concerning this fact. In the documentation every steps can be read as: it can be executed on the same machine.
SzaboZoltan (Fri, 17 Feb 2017 14:46:47 GMT):
"...as described in the documentation." I did not find any sentence concerning this fact. In the documentation every steps can be read as: it can be executed on the same machine.
SzaboZoltan (Fri, 17 Feb 2017 14:46:47 GMT):
"...as described in the documentation." I did not find any sentence concerning this fact. In the documentation every steps can be read as: it can be executed on the same machine.I meant: "...the worker nodes need to be setup manually before u can use it, as described in the documentation."
SzaboZoltan (Fri, 17 Feb 2017 14:46:47 GMT):
"...as described in the documentation." I did not find any sentence concerning this fact. In the documentation every steps can be read as: it can be executed on the same machine.
I meant: "...the worker nodes need to be setup manually before u can use it, as described in the documentation."
baohua (Fri, 17 Feb 2017 23:57:33 GMT):
it's at the last part:
baohua (Fri, 17 Feb 2017 23:57:52 GMT):
Message Attachments
jsong1230 (Sat, 18 Feb 2017 08:22:12 GMT):
Has joined the channel.
jsong1230 (Sat, 18 Feb 2017 12:38:54 GMT):
Message Attachments
jsong1230 (Sat, 18 Feb 2017 12:38:56 GMT):
I cannot access cello
jsong1230 (Sat, 18 Feb 2017 12:39:37 GMT):
Message Attachments
sidiyer (Sat, 18 Feb 2017 20:01:24 GMT):
Has joined the channel.
HenryZhang (Sun, 19 Feb 2017 16:18:50 GMT):
Has joined the channel.
SzaboZoltan (Mon, 20 Feb 2017 20:26:37 GMT):
@jsong1230 : Try to use git clone http://gerrit.hyperledger.org/r/cello instead of git clone https://github.com/yeasy/cello.
baohua (Tue, 21 Feb 2017 00:18:17 GMT):
@SzaboZoltan i saw ur previous patch, and would u like to revise it to fix this link? Thanks!
baohua (Tue, 21 Feb 2017 00:25:32 GMT):
@SzaboZoltan aha, maybe u just submit a new patchset, only fixing that link. and abandon the old one :)
baohua (Tue, 21 Feb 2017 02:03:50 GMT):
@SzaboZoltan aha, i edit online for u. never mind.
paul.sitoh (Tue, 21 Feb 2017 14:29:25 GMT):
Has joined the channel.
agaragiola (Tue, 21 Feb 2017 20:54:00 GMT):
Has joined the channel.
LumirB (Tue, 21 Feb 2017 23:00:56 GMT):
Has joined the channel.
bart.cant@gmail.com (Wed, 22 Feb 2017 23:35:15 GMT):
Has joined the channel.
bartcant (Thu, 23 Feb 2017 02:11:56 GMT):
Has joined the channel.
nykolla (Thu, 23 Feb 2017 21:52:23 GMT):
Has joined the channel.
albrandt (Fri, 24 Feb 2017 14:09:57 GMT):
Has joined the channel.
rjones (Fri, 24 Feb 2017 23:42:07 GMT):
Has joined the channel.
rjones (Fri, 24 Feb 2017 23:46:38 GMT):
@baohua I set up a test project with a new JIRA workflow, could you try it out and let me know if you want something like it for cello? https://jira.hyperledger.org/projects/WFT/issues/?filter=allopenissues
baohua (Sat, 25 Feb 2017 02:00:28 GMT):
thanks @rjones pls help enable and let's try the workflow. I still often get confused by jira :(
rjones (Sat, 25 Feb 2017 02:07:32 GMT):
@baohua I just wanted to you try it in the test project to see if creating backlog items and bugs and the ways they change state works for you
baohua (Sat, 25 Feb 2017 02:09:24 GMT):
yeap, i'm trying, seems good till now
Gouravchawla (Sat, 25 Feb 2017 03:14:06 GMT):
Has joined the channel.
Michael1891 (Sat, 25 Feb 2017 20:02:08 GMT):
Has joined the channel.
mayerwa (Mon, 27 Feb 2017 13:57:58 GMT):
Has joined the channel.
FlyingTiger (Wed, 01 Mar 2017 02:37:47 GMT):
Has joined the channel.
awalias (Wed, 01 Mar 2017 16:05:00 GMT):
Has joined the channel.
saism (Sun, 05 Mar 2017 19:49:07 GMT):
Has joined the channel.
Suma (Wed, 08 Mar 2017 15:40:28 GMT):
Has joined the channel.
dknochen (Wed, 08 Mar 2017 16:12:33 GMT):
Has joined the channel.
baohua (Thu, 09 Mar 2017 01:35:56 GMT):
we are collecting adoption cases for cello, pls feel free to add ur info there if u tried and like cello 🙂 https://wiki.hyperledger.org/projects/cello.md
baohua (Thu, 09 Mar 2017 01:37:26 GMT):
@hightall
YE.Yaocheng (Thu, 09 Mar 2017 07:50:27 GMT):
Has joined the channel.
aberfou (Thu, 09 Mar 2017 13:47:08 GMT):
Has joined the channel.
JatinderBali (Fri, 10 Mar 2017 15:40:41 GMT):
Has joined the channel.
rjones (Sat, 11 Mar 2017 04:56:43 GMT):
baohua
akm4 (Sat, 11 Mar 2017 21:11:27 GMT):
Has joined the channel.
akm4 (Sat, 11 Mar 2017 21:14:06 GMT):
Hi! The whole Cello project looks very cool to my eyes, thanks for starting it and hope it will grow. I didn't use it much, but for spinning the network on the local vagrant it was very usefull (then we moved to bluemix to continue experiments). I believe it has great potential for those who start working with Fabric and in future consortiums based on it
baohua (Sat, 11 Mar 2017 23:30:03 GMT):
Thanks @akm4 , yes, that's why we start that project, to help save time/effort to provision and manage those blockchains. Welcome for more feedbacks, and any type of contribution including bug fix and documentation.
baohua (Sun, 12 Mar 2017 00:08:42 GMT):
Message Attachments
baohua (Sun, 12 Mar 2017 00:09:04 GMT):
Chris showed the 5 top projects in Hyperledger Hackathon yesterday.
camelbeck (Sun, 12 Mar 2017 01:10:50 GMT):
Has joined the channel.
akm4 (Sun, 12 Mar 2017 19:47:42 GMT):
:thumbsup:
akm4 (Mon, 13 Mar 2017 05:24:37 GMT):
By the way, is there any plans to collaborate with fabric-composer team? I think, the projects are close by the idea (with some key differences of course)
baohua (Mon, 13 Mar 2017 06:26:23 GMT):
yeap, cello will definitely collaborate with other hyperledger projects, welcome for any idea and discussions :)
akm4 (Mon, 13 Mar 2017 07:01:32 GMT):
the idea is simple - to deploy and manage chaincode installations, like fabric-composer allows to (though in a bit unusual way). Would be handy for the administrators of the "consortium"
baohua (Mon, 13 Mar 2017 14:40:05 GMT):
IMHO, the main function of composer is to help generate chaincode, then cello may help the usage of those chaincode
akm4 (Mon, 13 Mar 2017 14:53:51 GMT):
sure! i meant that it would be cool to manage the chaincode lifecycle in the network - deploy, instantiate etc. And of course to have an audit (in blockchain style perhaps!) of what has been done to the CC and by whom. That will please ops and security services of traditional companies who are not ready to trust blockchain technologies
baohua (Tue, 14 Mar 2017 01:56:58 GMT):
good suggestion, akm4, would u like to create a jira task on this?
akm4 (Tue, 14 Mar 2017 06:46:02 GMT):
done
akm4 (Tue, 14 Mar 2017 06:46:25 GMT):
i created two tasks: one for CC management and the other about network audit
baohua (Tue, 14 Mar 2017 07:39:53 GMT):
:thumbsup:
YE.Yaocheng (Tue, 14 Mar 2017 08:09:16 GMT):
Hi, all! I'm trying to explore the blockchain on the bluemix. I'm using the basic plan, four nodes and a ca. But only the ca is running, the other four nodes cannot be start. Any ideas, guys?
YE.Yaocheng (Tue, 14 Mar 2017 08:09:29 GMT):
Message Attachments
YE.Yaocheng (Tue, 14 Mar 2017 08:10:27 GMT):
Message Attachments
YE.Yaocheng (Tue, 14 Mar 2017 08:11:13 GMT):
Message Attachments
YE.Yaocheng (Tue, 14 Mar 2017 08:17:30 GMT):
I noticed on the peer's log that "grpc : time out when dialing". Also, I found this error frequently when I was trying to deploy the fabric on my own. I think it should be a problem with docker dns. What do you think and any solutions? @baohua
baohua (Tue, 14 Mar 2017 09:00:47 GMT):
strange, what cello and docker version are u using?
YE.Yaocheng (Tue, 14 Mar 2017 09:30:08 GMT):
docker 17.03.0-ce;docker-compose 1.11.2
YE.Yaocheng (Tue, 14 Mar 2017 09:33:39 GMT):
I started fabric with docker-compose, "four-peer-ca.yaml" the file at github.com/hyperledger/fabric/examples/docker-compose/four-peer-ca.yaml at branch v0.6
baohua (Tue, 14 Mar 2017 12:20:10 GMT):
no, this is the channel for cello project. would suggest u ask this question at fabric or twg-china channels.
daijianw (Tue, 14 Mar 2017 12:38:45 GMT):
Has joined the channel.
YE.Yaocheng (Tue, 14 Mar 2017 12:44:06 GMT):
ok, I'm sorry
baohua (Tue, 14 Mar 2017 14:16:41 GMT):
:grinning:
baohua (Tue, 14 Mar 2017 15:00:08 GMT):
@FlyingTiger welcome
baohua (Tue, 14 Mar 2017 15:00:09 GMT):
https://jira.hyperledger.org/browse/CE-7 assigned to u~~ i suggest we start it right now, not waiting till summer :)
schwentker2 (Tue, 14 Mar 2017 21:01:33 GMT):
Has joined the channel.
duncanjw (Tue, 14 Mar 2017 22:05:40 GMT):
Has joined the channel.
baohua (Wed, 15 Mar 2017 01:10:24 GMT):
https://gerrit.hyperledger.org/r/#/c/7197/
FlyingTiger (Wed, 15 Mar 2017 01:35:52 GMT):
Hi, everybody, nice to know you.:smiley:
AJ123 (Wed, 15 Mar 2017 05:38:05 GMT):
Has joined the channel.
DaVi (Wed, 15 Mar 2017 11:40:30 GMT):
Has joined the channel.
Lin-YiTang (Wed, 15 Mar 2017 16:14:18 GMT):
Has joined the channel.
baohua (Thu, 16 Mar 2017 01:23:47 GMT):
@FlyingTiger remember to update ur contact info at https://wiki.hyperledger.org/projects/cello
baohua (Thu, 16 Mar 2017 01:39:55 GMT):
Now we are using original wiki grammar for the project. pls feel free to help refine, thanks @here
FlyingTiger (Thu, 16 Mar 2017 02:42:40 GMT):
@baohua Thanks for reminding, I updated my contact info.
baohua (Thu, 16 Mar 2017 08:21:01 GMT):
More documentation are added: https://gerrit.hyperledger.org/r/#/c/7257/
vishwass (Thu, 16 Mar 2017 15:42:42 GMT):
Has joined the channel.
vishwass (Thu, 16 Mar 2017 15:45:52 GMT):
Hi Baohua ! Vishwas here. We had a chat on email.
vishwass (Thu, 16 Mar 2017 15:49:45 GMT):
I wanted to start working on CE-4 and CE-5
vishwass (Thu, 16 Mar 2017 15:55:25 GMT):
I was trying to run Cello.Installation was successful . Could you please tell me where I can run this command.(you have mentioned the master ) - docker -H Worker_Node_IP:2375 version- I am getting connection error on this .
baohua (Fri, 17 Mar 2017 00:57:13 GMT):
aha, which doc are u seeing for the cmd? This seems the doc is not good enough for new starters :(
vishwass (Fri, 17 Mar 2017 01:48:04 GMT):
i am seeing from docs in yeasy/cello from github
vishwass (Fri, 17 Mar 2017 01:48:53 GMT):
@baohua
baohua (Fri, 17 Mar 2017 01:55:21 GMT):
yeap, could u help indicate the exact location, then see maybe we need to refine to be more clear.
andyxf1029 (Fri, 17 Mar 2017 06:08:09 GMT):
Has joined the channel.
vishwass (Fri, 17 Mar 2017 11:48:10 GMT):
@baohua https://github.com/hyperledger/cello/blob/master/docs/install.md
obigass (Fri, 17 Mar 2017 14:42:00 GMT):
Has joined the channel.
baohua (Fri, 17 Mar 2017 15:22:55 GMT):
ok, in the above line, there is `run the follow test at Master node and get OK response`
vishwass (Fri, 17 Mar 2017 15:50:55 GMT):
@baohua Sorry, there was a problem with port and ip of worker nodes. sorry about that. Just started to learn cello
baohua (Sat, 18 Mar 2017 00:15:20 GMT):
yeap, the ip and port is from the worker node you set as the chain host. e.g., for a docker host, just use `ifconfig` to see the ip. and defaultly, set the port to `2375`
baohua (Sun, 19 Mar 2017 12:41:12 GMT):
https://chat.hyperledger.org/channel/twg-china
baohua (Sun, 19 Mar 2017 12:41:32 GMT):
https://gerrit.hyperledger.org/r/#/c/7331
madity (Sun, 19 Mar 2017 19:17:23 GMT):
Has joined the channel.
dannysun85 (Mon, 20 Mar 2017 12:18:00 GMT):
Has joined the channel.
vishwass (Mon, 20 Mar 2017 15:48:35 GMT):
Hi @baohua was able to successfully deploy Hyperledger Cello in AWS using 2 EC2 instances.
vishwass (Mon, 20 Mar 2017 15:48:50 GMT):
Message Attachments
vishwass (Mon, 20 Mar 2017 15:49:11 GMT):
thanks for the help @baohua
vishwass (Mon, 20 Mar 2017 15:51:49 GMT):
Message Attachments
Amanskywalker (Mon, 20 Mar 2017 17:57:02 GMT):
Has joined the channel.
baohua (Tue, 21 Mar 2017 01:01:20 GMT):
Cool, vishwass, it would be cool if u can add some doc on describing the setup steps on AWS! @vishwass
FlyingTiger (Tue, 21 Mar 2017 01:18:48 GMT):
:thumbsup:
sandroku63 (Tue, 21 Mar 2017 08:10:30 GMT):
Has joined the channel.
sandroku63 (Tue, 21 Mar 2017 08:14:05 GMT):
Message Attachments
sandroku63 (Tue, 21 Mar 2017 08:15:40 GMT):
Hi@baohua how can i fix chain fail issue ? Thanks~
sandroku63 (Tue, 21 Mar 2017 08:18:16 GMT):
My host 2 is active now, but chain always show fail .
sandroku63 (Tue, 21 Mar 2017 08:19:08 GMT):
Message Attachments
sandroku63 (Tue, 21 Mar 2017 08:19:45 GMT):
Message Attachments
baohua (Tue, 21 Mar 2017 08:36:15 GMT):
fail mean there are some peers that do not see other nodes in the network.
baohua (Tue, 21 Mar 2017 08:36:48 GMT):
pls check ur fabric network. and those fail chains will not be offered to the requesters.
sandroku63 (Tue, 21 Mar 2017 08:49:33 GMT):
Thank you , I will be check it.
vishwass (Tue, 21 Mar 2017 11:48:56 GMT):
@baohua sure I can do that
vishwass (Tue, 21 Mar 2017 11:49:02 GMT):
I would love to
saksham1115 (Thu, 23 Mar 2017 10:13:45 GMT):
Has joined the channel.
matanyahu (Thu, 23 Mar 2017 10:52:45 GMT):
Has joined the channel.
baohua (Fri, 24 Mar 2017 05:27:49 GMT):
https://gerrit.hyperledger.org/r/#/c/7409/
xuanyue202 (Mon, 27 Mar 2017 05:47:41 GMT):
Has joined the channel.
t-watana (Mon, 27 Mar 2017 09:51:57 GMT):
Has joined the channel.
MingxiaGuo (Mon, 27 Mar 2017 12:58:06 GMT):
Has joined the channel.
baohua (Tue, 28 Mar 2017 05:45:52 GMT):
https://gerrit.hyperledger.org/r/#/c/7465/
baohua (Tue, 28 Mar 2017 06:16:38 GMT):
hi, a new sub project of cello-analytics is just created, welcome for more volunteers if u think it's interesting :) https://wiki.hyperledger.org/projects/cello-analytics
baohua (Tue, 28 Mar 2017 06:17:10 GMT):
@here
baohua (Tue, 28 Mar 2017 06:34:25 GMT):
http://gerrit.hyperledger.org/r/7489
baohua (Tue, 28 Mar 2017 06:39:02 GMT):
https://gerrit.hyperledger.org/r/#/c/7491/
Vadim (Tue, 28 Mar 2017 07:01:09 GMT):
Has joined the channel.
rohitbordia (Tue, 28 Mar 2017 17:44:36 GMT):
Has joined the channel.
rohitbordia (Tue, 28 Mar 2017 17:45:33 GMT):
Hi , i have used vagrant and deployed cello in ubuntu 14 , I see docker containers are running but, cannot get to dashboard app
rohitbordia (Tue, 28 Mar 2017 17:46:57 GMT):
from the documentation docker -H Worker_Node_IP:2375 version .. does not work and say check if docker dameon is running
rohitbordia (Tue, 28 Mar 2017 17:47:10 GMT):
any help or advice is greatly appreciated
rohitbordia (Tue, 28 Mar 2017 18:24:23 GMT):
@baohua : I have also did what you recommended to vishwass ifconfig even that didnt worked
baohua (Wed, 29 Mar 2017 01:46:18 GMT):
that means u cannot access worker_node from the master_node. pls check ur docker daemon setup on worker_node and network connection, e.g., firewall rules to block 2375 port?
rohitbordia (Wed, 29 Mar 2017 02:31:19 GMT):
sure thanks
vishwass (Wed, 29 Mar 2017 14:14:41 GMT):
updated aws setup as you suggested. @baohua http://gerrit.hyperledger.org/r/7533
baohua (Wed, 29 Mar 2017 15:10:18 GMT):
https://gerrit.hyperledger.org/r/#/c/7533/
holocron (Wed, 29 Mar 2017 17:42:01 GMT):
Has joined the channel.
rohitbordia (Wed, 29 Mar 2017 19:36:41 GMT):
@baohua : I was able to set-up master correctly with 2375 .. But, in hashboard add host its not taking the ip address
rohitbordia (Wed, 29 Mar 2017 19:37:07 GMT):
this works correctly : sudo docker -H 0.0.0.0:2375 version
rohitbordia (Wed, 29 Mar 2017 20:50:41 GMT):
is it a problem due to my docker version Docker version 17.03.1-ce , I don;t see it comes with docker machine and have only 1 node
baohua (Thu, 30 Mar 2017 01:34:16 GMT):
would suggest u use a linux server as the node first. 1.10~1.13 are tested.
baohua (Thu, 30 Mar 2017 02:11:34 GMT):
https://gerrit.hyperledger.org/r/#/c/7547/
hightall (Thu, 30 Mar 2017 02:46:56 GMT):
https://gerrit.hyperledger.org/r/#/c/7535/
rohitbordia (Thu, 30 Mar 2017 02:55:51 GMT):
@baohua : I created a linux vm 16.04 with vagrant and installed docker on it
rohitbordia (Thu, 30 Mar 2017 03:06:04 GMT):
or if you can guide me where to read about master /worker node in docker
baohua (Thu, 30 Mar 2017 03:15:03 GMT):
rohitbordia, seems u are not that familiar with docker usage. No worry, but u need some time to get onboard. And hope these links are helpful to u, https://docs.docker.com/engine/userguide/, http://cello.readthedocs.io/en/latest/
rohitbordia (Thu, 30 Mar 2017 03:19:01 GMT):
thanks
rohitbordia (Thu, 30 Mar 2017 18:59:55 GMT):
@baohua : Thanks spent some time with docker and was able to successfully create host and chains
baohua (Fri, 31 Mar 2017 02:52:02 GMT):
u got it!
hightall (Fri, 31 Mar 2017 13:45:18 GMT):
https://gerrit.hyperledger.org/r/#/c/7609/
nnao (Fri, 31 Mar 2017 18:56:44 GMT):
Has joined the channel.
vishwass (Fri, 31 Mar 2017 23:52:51 GMT):
Hi all, I was trying to solve CE-5 - updating REST API yaml files - Looks like I have reached a deadend .The restserver_v2.md specifies multiple calls under a single path. But this link https://github.com/swagger-api/swagger-editor/issues/854 says that swagger cannot does not allow multiple operation(with different parameters) under a single path. Does anyone have any idea or a work around for this problem.
vishwass (Fri, 31 Mar 2017 23:53:18 GMT):
?
vishwass (Sat, 01 Apr 2017 00:22:32 GMT):
I found a work around for this /a/b/c:
post:
/a/b/c#merge:
post:
vishwass (Sat, 01 Apr 2017 00:22:32 GMT):
I found a work around for this /a/b/c:
post:
/a/b/c#merge:
post:
vishwass (Sat, 01 Apr 2017 00:22:32 GMT):
I found a work around for this /a/b/c:
post:
/a/b/c#merge:
post:
vishwass (Sat, 01 Apr 2017 00:25:08 GMT):
I found a work around for this in this link https://github.com/OAI/OpenAPI-Specification/issues/182
vishwass (Sat, 01 Apr 2017 00:26:02 GMT):
``` /a/b/c:
post:
/a/b/c#merge:
post: ```
vishwass (Sat, 01 Apr 2017 00:27:43 GMT):
basically use tags for the same path. @baohua @hightall .What would you suggest ?
baohua (Sat, 01 Apr 2017 04:38:29 GMT):
@vishwass i would suggest we doc base on the existing implementation for this version of API (v2), while leading the discussion on the new spec/format of next version (v3). How do u think? personally, i would prefer the openAPI's work around, as sometimes, it's more flexible to support various operations on single resource type.
ccen (Sat, 01 Apr 2017 20:45:22 GMT):
Has joined the channel.
vishwass (Sat, 01 Apr 2017 22:12:44 GMT):
ya currently i guess we should stick with the work around . As the swagger 3.0 is not released in which they would probably add the feature
vishwass (Sat, 01 Apr 2017 22:12:48 GMT):
@baohua
baohua (Sun, 02 Apr 2017 00:52:47 GMT):
exactly, and thanks for ur help!
hightall (Sun, 02 Apr 2017 12:10:14 GMT):
https://gerrit.hyperledger.org/r/#/c/7611/
choas (Sun, 02 Apr 2017 18:35:03 GMT):
Has joined the channel.
baohua (Mon, 03 Apr 2017 01:31:29 GMT):
+2 && merged
hightall (Tue, 04 Apr 2017 11:32:53 GMT):
http://gerrit.hyperledger.org/r/7673
hightall (Tue, 04 Apr 2017 13:06:23 GMT):
http://gerrit.hyperledger.org/r/7677
mikezaccardo (Tue, 04 Apr 2017 13:59:11 GMT):
Hi, I've successfully installed Cello and have deployed multiple active chains, but I'm wondering how to deploy a `membersrvc` container? I do not see one running anywhere.
hightall (Tue, 04 Apr 2017 14:23:58 GMT):
http://gerrit.hyperledger.org/r/7685
mikezaccardo (Tue, 04 Apr 2017 16:26:16 GMT):
Also, is there a way to add a Docker host via the REST API? I don't see any available methods but just want to be sure.
duncanjw (Tue, 04 Apr 2017 22:21:56 GMT):
@baohua looking forward to catching up Thurs. In the meantime if someone can address @mikezaccardo's questions that would be awesome
baohua (Wed, 05 Apr 2017 01:28:38 GMT):
@duncanjw exactly! would expect to see ur demo :) @FlyingTiger @grapebaba
baohua (Wed, 05 Apr 2017 01:29:08 GMT):
@mikezaccardo exactly, u can add through the web UI (recomended), or directly use the rest api.
mikezaccardo (Wed, 05 Apr 2017 01:30:12 GMT):
@baohua can you refer me to the REST API call that enables adding a Docker host?
mikezaccardo (Wed, 05 Apr 2017 01:30:12 GMT):
@baohua can you please refer me to the REST API call that enables adding a Docker host?
baohua (Wed, 05 Apr 2017 01:32:13 GMT):
sure, would like to suggest u have a look at the code at cello/src/resources/host_api.py, where should be the latest routing there.
mikezaccardo (Wed, 05 Apr 2017 01:39:08 GMT):
@baohua i see it here, thanks! https://github.com/hyperledger/cello/blob/master/src/resources/host_api.py#L49
mikezaccardo (Wed, 05 Apr 2017 04:09:24 GMT):
@baohua can you help me with deploying a `membersrvc` container along with the validating peers? Whenever I create a new chain, only vp0, vp1, ... are created.
mikezaccardo (Wed, 05 Apr 2017 07:22:25 GMT): The browser (or proxy) sent a request that this server could not understand. The browser (or proxy) sent a request that this server could not understand.
@here I'm having trouble invoking that REST API call. Can anyone help?
Attempt 1:
```
curl -X POST -u admin:pass \
-H "Content-Type: multipart/form-data" \
-F "name=docker_host_1" \
-F "daemon_url=Bad Request
Bad Request
duncanjw (Wed, 05 Apr 2017 08:29:31 GMT):
@mikezaccardo in case there are problems with GTM tomorrow can we record demo for @baohua?
murdoaird (Wed, 05 Apr 2017 09:25:27 GMT):
Has joined the channel.
hshimo (Wed, 05 Apr 2017 15:18:53 GMT):
Has joined the channel.
hshimo (Wed, 05 Apr 2017 15:20:38 GMT):
What version of Fabric does Cello support? (I couldn't find the answer on the repo README.)
What is the stage of development? (still active development? alpha? beta? stable?)
hshimo (Wed, 05 Apr 2017 15:20:38 GMT):
What version of Fabric does Cello support? v0.6 or v1.0a? (I couldn't find the answer on the repo README.)
What is the stage of development? (still active development? alpha? beta? stable?)
holocron (Wed, 05 Apr 2017 15:25:18 GMT):
@hshimo from my brief inspection this week, i would say it only has v.6 support currently. I would love to hear an answer on this question as well.
vishwass (Wed, 05 Apr 2017 15:39:05 GMT):
currently supports v0.6 still need to support v1.0
mikezaccardo (Wed, 05 Apr 2017 17:29:15 GMT):
Got it working! It needed to be:
```
curl -X POST -u admin:pass \
-H "Content-Type: application/x-www-form-urlencoded" \
-d "name=docker_host_1&daemon_url=
DrTES (Wed, 05 Apr 2017 22:30:31 GMT):
Has joined the channel.
baohua (Thu, 06 Apr 2017 03:46:20 GMT):
great! would welcome if you can add a section in the documentation on how to call those apis with curl, besides the web ui operations. @mikezaccardo
hshimo (Thu, 06 Apr 2017 03:53:37 GMT):
@holocron @vishwass thank you for the answer! Where should I check when I want to know the dev progress? gerrit?
baohua (Thu, 06 Apr 2017 04:56:09 GMT):
https://github.com/hyperledger/cello/blob/master/docs/CONTRIBUTING.md
mikezaccardo (Thu, 06 Apr 2017 14:49:59 GMT):
@baohua will do re: cURL docs
rohitbordia (Thu, 06 Apr 2017 21:54:29 GMT):
Do we have any example chain code to deploy on Cello worker node?
baohua (Fri, 07 Apr 2017 02:00:07 GMT):
Now cello only provides fabric clusters, then u can use the original ways to deploy chaincode on the fabric cluster.
rohitbordia (Fri, 07 Apr 2017 03:55:09 GMT):
once we deploy manually the chaincode on worker nodes
rohitbordia (Fri, 07 Apr 2017 03:55:44 GMT):
we just need to change the ip address in node js example to connect to worker node rt?
tmartin (Fri, 07 Apr 2017 19:48:29 GMT):
Has joined the channel.
mikezaccardo (Fri, 07 Apr 2017 22:27:56 GMT):
@baohua @duncanjw our open-source Apache Brooklyn blueprint for deploying Cello is now publicly available: https://github.com/cloudsoft/brooklyn-cello
duncanjw (Fri, 07 Apr 2017 22:28:50 GMT):
@mikezaccardo awesome - thanks for all your hard work pulling this together
duncanjw (Sat, 08 Apr 2017 15:49:43 GMT):
@baohua we've added Cloudsoft to https://wiki.hyperledger.org/projects/cello?adoptions
ray (Sat, 08 Apr 2017 16:18:26 GMT):
Has left the channel.
dante (Sat, 08 Apr 2017 22:56:06 GMT):
Has joined the channel.
grkvlt (Sun, 09 Apr 2017 14:48:37 GMT):
Has joined the channel.
baohua (Mon, 10 Apr 2017 01:59:29 GMT):
thanks @duncanjw @mikezaccardo
baohua (Tue, 11 Apr 2017 02:26:37 GMT):
https://gerrit.hyperledger.org/r/#/c/7915/
msoumeit (Wed, 12 Apr 2017 17:05:16 GMT):
Has joined the channel.
haiderny (Thu, 13 Apr 2017 14:11:02 GMT):
Has joined the channel.
MikeMayori (Thu, 13 Apr 2017 16:27:57 GMT):
Has joined the channel.
baohua (Fri, 14 Apr 2017 02:05:25 GMT):
https://gerrit.hyperledger.org/r/#/c/7935/
magg (Sun, 16 Apr 2017 17:15:21 GMT):
Has joined the channel.
magg (Sun, 16 Apr 2017 17:22:56 GMT):
is there any tutorial on how to make cello work with Docker Swarm
magg (Sun, 16 Apr 2017 17:22:59 GMT):
???
SzaboZoltan (Sun, 16 Apr 2017 20:49:13 GMT):
@mikezaccardo I tried to follow https://github.com/cloudsoft/brooklyn-cello (great work really!) and make a cluster on AWS. It creates all the entities in the cloud but then the deployment fails due to ssh issues. "Error in task: ssh: Getting machine details for: SshMachineLocation...". It's funny that the EC2 instances have some kind of "jclouds#brooklyn-ooirql-amp-auto-gnn8-hyperledgercello-n2..." key pairs which is generated by the AMP/Brooklyn framework. Why are they not working?! Any experience?
SzaboZoltan (Mon, 17 Apr 2017 06:34:25 GMT):
@magg - yes, here: https://github.com/hyperledger/cello/blob/master/docs/install.md
mikezaccardo (Mon, 17 Apr 2017 14:27:26 GMT):
@SzaboZoltan You are correct in that AMP is generating a key pair that it will use to SSH into the VMs that it creates. I'm not sure why you experienced that error. Can you please send me your location configuration (with identity + credential removed) and any additional output from the app inspector console that relates to the error?
SzaboZoltan (Tue, 18 Apr 2017 05:11:03 GMT):
I think it will be an sshj problem. The error message is here: https://pastebin.com/k0RgLKKm | The instances began to be run (but finally terminated due to failure of deployment: http://porgeto.hu/0/instances.png http://porgeto.hu/0/instances_.png | My location looks like this: http://porgeto.hu/0/zol1_location.png
SzaboZoltan (Tue, 18 Apr 2017 05:11:03 GMT):
@mikezaccardo I think it will be an sshj problem. The error message is here: https://pastebin.com/k0RgLKKm | The instances began to be run (but finally terminated due to failure of deployment: http://porgeto.hu/0/instances.png http://porgeto.hu/0/instances_.png | My location looks like this: http://porgeto.hu/0/zol1_location.png
SzaboZoltan (Tue, 18 Apr 2017 05:11:03 GMT):
@mikezaccardo I think it will be an sshj problem. The error message is here: https://pastebin.com/k0RgLKKm | The instances began to be run (but finally terminated due to failure of the deployment): http://porgeto.hu/0/instances.png http://porgeto.hu/0/instances_.png | My location looks like this: http://porgeto.hu/0/zol1_location.png
SzaboZoltan (Tue, 18 Apr 2017 05:11:03 GMT):
@mikezaccardo I think it will be an sshj (or user/loginUser definition) problem. The error message is here: https://pastebin.com/k0RgLKKm | The instances began to be run (but finally terminated due to failure of the deployment): http://porgeto.hu/0/instances.png http://porgeto.hu/0/instances_.png | My location looks like this: http://porgeto.hu/0/zol1_location.png
SzaboZoltan (Tue, 18 Apr 2017 05:11:03 GMT):
@mikezaccardo - Now it's working fine. I gave too much parameters, such few is enough: http://porgeto.hu/0/zol2.png
SzaboZoltan (Tue, 18 Apr 2017 05:11:03 GMT):
@mikezaccardo - Now it's working fine. I gave too much parameters, such few is enough: http://porgeto.hu/0/zol2.png | porgeto.hu/0/green.png
presingzs (Tue, 18 Apr 2017 08:35:12 GMT):
Has joined the channel.
magg (Tue, 18 Apr 2017 16:06:56 GMT):
thanks @SzaboZoltan but I don't understand some things: suposse I have a docker swarm cluster... am I supossed to configure the workers node like the tutorial says and the master node should have the restserver, dashboard, nginx, mongo and watchdog
magg (Tue, 18 Apr 2017 16:06:56 GMT):
thanks @SzaboZoltan but I don't understand some things: suposse I have a docker swarm cluster... am I supossed to configure the Swarm Workers nodes like the tutorial says and the Swarm Master node should have the restserver, dashboard, nginx, mongo and watchdog. is that it?
magg (Tue, 18 Apr 2017 16:06:56 GMT):
thanks @SzaboZoltan but I don't understand some things: suposse I have a docker swarm cluster... am I supossed to configure the Swarm Workers nodes like the tutorial says and the Swarm Master node should have the restserver, dashboard, nginx, mongo and watchdog containers?. is that it?
SzaboZoltan (Tue, 18 Apr 2017 17:22:59 GMT):
@magg I am not a professional in this theme. I suggest you to have a look at https://github.com/cloudsoft/brooklyn-cello/blob/master/catalog.bom - in this file there is a part below "id: hyperledger-cello-master-node" and another part "id: hyperledger-cello-worker-node". There is an "install.command:" part in both section; it is really useful to follow this. You should do the steps of the *master-node on your swarm master, and the worker(s) the "worker-node".
SzaboZoltan (Tue, 18 Apr 2017 17:22:59 GMT):
@magg I am not a professional in this theme. I suggest you to have a look at https://github.com/cloudsoft/brooklyn-cello/blob/master/catalog.bom - in this file there is a part below "id: hyperledger-cello-master-node" and another part "id: hyperledger-cello-worker-node". There is an "install.command:" part in both section; it is really useful to follow this. You should do the steps of the "master-node" on your swarm master, and on the worker(s) the "worker-node".
SzaboZoltan (Tue, 18 Apr 2017 17:22:59 GMT):
@magg I am not a professional in this theme. I suggest you to have a look at https://github.com/cloudsoft/brooklyn-cello/blob/master/catalog.bom - in this file there is a part below the line "id: hyperledger-cello-master-node" and another part of "id: hyperledger-cello-worker-node". There is an "install.command:" part in both sections; it is really useful to follow this. You should do the steps of the "master-node" on your swarm master, and on the worker(s) the "worker-node".
SzaboZoltan (Tue, 18 Apr 2017 18:16:56 GMT):
A question: if I successfully deploy a Cello system via https://github.com/cloudsoft/brooklyn-cello , is there a possibility to log in via ssh into one of the AMP/brooklyn created AWS machines? (Which has automatically generatid public key like "clouds#brooklyn-oom8n2-root-apr18-e2x7-hyperledgercello-ophu#23".)
mikezaccardo (Tue, 18 Apr 2017 18:35:25 GMT):
@SzaboZoltan Great to hear that you got it working! In order to manually SSH into the VMs that are created, you'll need to either configure the location to provide an SSH key from your machine or have an `~/.ssh/_id_rsa` key already there -- see this for more info: https://docs.cloudsoft.io/locations/reference/index.html
mikezaccardo (Tue, 18 Apr 2017 18:35:25 GMT):
@SzaboZoltan Great to hear that you got it working! In order to manually SSH into the VMs that are created, you'll need to either configure the location to provide an SSH key from your machine or have an `~/.ssh/id_rsa` key already there -- see this for more info: https://docs.cloudsoft.io/locations/reference/index.html
mikezaccardo (Tue, 18 Apr 2017 18:37:22 GMT):
For the configuration using a specific key on your machine, the location configuration would need a line like: `loginUser.privateKeyFile: /path/to/your/privatekey`
duncanjw (Tue, 18 Apr 2017 23:11:21 GMT):
In other news https://cloudsoft.io/blog/deploying-hyperledger-fabric-on-ibm-bluemix-kubernetes-cluster-using-amp // @cbf
duncanjw (Tue, 18 Apr 2017 23:12:44 GMT):
Good proof point that if someone has stood up a K8s cluster - in this case IBM Bluemix - we can derive an Apache Brooklyn location from the standard kubectl proxy config file
duncanjw (Tue, 18 Apr 2017 23:13:24 GMT):
We will be validating this against GCP and Azure that both offer managed Kubernetes clusters
cbf (Wed, 19 Apr 2017 13:42:40 GMT):
nice!
SzaboZoltan (Wed, 19 Apr 2017 13:58:15 GMT):
@mikezaccardo Thank you, I've successfully logged in to one of the created VMs.
SzaboZoltan (Wed, 19 Apr 2017 14:03:36 GMT):
2 questions: Iis it possible to put into the blueprint, that I would like to have "stopable" VMs? The default possibilities is now are only "reboot, terminate" due to the lack of EBS entity. When I try to change shutdown behavior: >>An error occurred while changing the shutdown behavior of this instance: The instance 'i-04aab97d9f4423c98' does not have an 'ebs' root device type.<< | And also I would like to have t2.micro VMs, not the expensive m3.large ones (due to minRam parameter). But if I eliminate minRam, it will be "...medium", not "micro".
SzaboZoltan (Wed, 19 Apr 2017 14:03:36 GMT):
2 questions: Iis it possible to put into the blueprint, that I would like to have "stopable" VMs? The default possibilities are now are only "reboot, terminate" due to the lack of EBS entity. When I try to change shutdown behavior: >>An error occurred while changing the shutdown behavior of this instance: The instance 'i-04aab97d9f4423c98' does not have an 'ebs' root device type.<< | And also I would like to have t2.micro VMs, not the expensive m3.large ones (due to minRam parameter). But if I eliminate minRam, it will be "...medium", not "micro".
mikezaccardo (Wed, 19 Apr 2017 19:17:23 GMT):
1. Do you mean a way to get the VMs to stop and then come back alive at a later time? I don't think that is possible just now. You can of course get the VMs to terminate through AMP by "expunging" the application. See screenshot:
mikezaccardo (Wed, 19 Apr 2017 19:17:42 GMT):
Message Attachments
mikezaccardo (Wed, 19 Apr 2017 19:19:43 GMT):
2. I recommend using a specific `imageId` value for a known AWS image with the specs that you want. You would replace `imageNameRegex` with `imageId` in the blueprint before you import it:
* https://github.com/cloudsoft/brooklyn-cello/blob/master/catalog.bom#L102
* https://github.com/cloudsoft/brooklyn-cello/blob/master/catalog.bom#L102
mikezaccardo (Wed, 19 Apr 2017 19:19:43 GMT):
2. I recommend using a specific `imageId` value for a known AWS image with the specs that you want. See here for more info: https://docs.cloudsoft.io/locations/reference/#common-configuration-options.
You would replace `imageNameRegex` with `imageId` in the blueprint before you import it at these lines:
* https://github.com/cloudsoft/brooklyn-cello/blob/master/catalog.bom#L102
* https://github.com/cloudsoft/brooklyn-cello/blob/master/catalog.bom#L102
mikezaccardo (Wed, 19 Apr 2017 19:19:43 GMT):
2. I recommend using a specific `imageId` value for a known AWS image with the specs that you want. See here for more info: https://docs.cloudsoft.io/locations/reference/#common-configuration-options.
You would replace `imageNameRegex` with `imageId` in the blueprint before you import it at these lines:
* https://github.com/cloudsoft/brooklyn-cello/blob/master/catalog.bom#L102
* https://github.com/cloudsoft/brooklyn-cello/blob/master/catalog.bom#L157
dklesev (Wed, 19 Apr 2017 22:15:17 GMT):
Has joined the channel.
ulysses (Thu, 20 Apr 2017 16:35:28 GMT):
Has joined the channel.
johnsacco (Thu, 20 Apr 2017 21:34:20 GMT):
Has joined the channel.
magg (Fri, 21 Apr 2017 09:44:06 GMT):
hello I made a successful deployment of Cello in Grid5000.fr but I have a question. Can Cello only deploy a Blockchain in a Swarm worker machine ( 4 or 6 VP nodes)? We require to scatter the VP nodes of one Blockchain across different Swarm worker machine
magg (Fri, 21 Apr 2017 09:45:36 GMT):
Can I tell Cello to deploy say 20 VP nodes across 3 Swarm worker machines, for example?
magg (Fri, 21 Apr 2017 09:45:36 GMT):
Can I tell Cello to deploy (one blockchain) say 20 VP nodes across 3 Swarm worker machines, for example?
SzaboZoltan (Fri, 21 Apr 2017 11:33:05 GMT):
@magg What do you mean a VP node?
magg (Fri, 21 Apr 2017 13:54:24 GMT):
@SzaboZoltan a hyperledger peer or validation peer
magg (Fri, 21 Apr 2017 13:58:52 GMT):
https://hub.docker.com/r/hyperledger/fabric-peer/ this one
SzaboZoltan (Fri, 21 Apr 2017 14:07:43 GMT):
@magg Well, I don't know the exact relationship between nodes and peers and chains, but I suppose node=peer in this context.
magg (Fri, 21 Apr 2017 14:13:53 GMT):
ok yeah i meant to say peers... sorry. So I want to have many peers spread across the swarm... not only one swarm worker
magg (Fri, 21 Apr 2017 14:14:20 GMT):
and I only see the limit to add 4 or 6 peers
Dhanya.M.Kashyap (Fri, 21 Apr 2017 18:01:41 GMT):
Has joined the channel.
Dhanya.M.Kashyap (Fri, 21 Apr 2017 18:01:49 GMT):
https://gerrit.hyperledger.org/r/#/c/8369/
Dhanya.M.Kashyap (Fri, 21 Apr 2017 18:03:23 GMT):
new logo created, need review....and why did the job builder reject it? I didn't understand the reason?!!
choas (Fri, 21 Apr 2017 22:01:01 GMT):
@Dhanya.M.Kashyap
17:47:05 Error: Trailing white spaces found in file:logo3.png
The reason for this is at the Configuration https://jenkins.hyperledger.org/job/cello-verify-x86_64/configure-readonly/ Build script, which verifies ALL commited files for trailing white spaces at the end of a line: egrep -l " +$" $filename
... but it's a png file and not a source file (shouldn't verify binary files)
Dhanya.M.Kashyap (Sat, 22 Apr 2017 16:35:20 GMT):
@choas thankyou, i will try removing the white space at the end of the file
baohua (Sun, 23 Apr 2017 01:28:41 GMT):
@Dhanya.M.Kashyap thanks for ur efforts, while to update a patchset, u actually do not need to create new one. pls follow the contribution doc with using `git commit -a --amend`
baohua (Sun, 23 Apr 2017 01:28:44 GMT):
thanks
indirajith (Sun, 23 Apr 2017 20:15:23 GMT):
Has joined the channel.
lujiang (Mon, 24 Apr 2017 02:06:31 GMT):
Has joined the channel.
BlurgenStein (Tue, 25 Apr 2017 16:32:07 GMT):
Has joined the channel.
BlurgenStein (Tue, 25 Apr 2017 16:32:17 GMT):
Hello
BlurgenStein (Tue, 25 Apr 2017 16:32:37 GMT):
anyone here?
SzaboZoltan (Tue, 25 Apr 2017 19:13:15 GMT):
I sEeee youuuu
BlurgenStein (Tue, 25 Apr 2017 20:01:40 GMT):
awesome
BlurgenStein (Tue, 25 Apr 2017 20:01:47 GMT):
I wonder if I can help out
BlurgenStein (Tue, 25 Apr 2017 20:01:55 GMT):
I saw this internship request here:
BlurgenStein (Tue, 25 Apr 2017 20:02:23 GMT):
https://wiki.hyperledger.org/internship/project_ideas
rjones (Tue, 25 Apr 2017 21:51:01 GMT):
@BlurgenStein the window for applications closed a while ago, sorry
rjones (Tue, 25 Apr 2017 21:52:30 GMT):
@BlurgenStein https://wiki.hyperledger.org/internship/schedule
William_weicong (Wed, 26 Apr 2017 01:59:30 GMT):
Has joined the channel.
baohua (Wed, 26 Apr 2017 03:04:27 GMT):
:grinning: pity
BlurgenStein (Wed, 26 Apr 2017 03:15:15 GMT):
Hi! Thanks for answering. Okay, no problem. I figure help is still needed
BlurgenStein (Wed, 26 Apr 2017 03:15:26 GMT):
@rjones
rjones (Wed, 26 Apr 2017 04:07:03 GMT):
@BlurgenStein grab a JIRA off the queue and jump in!
BlurgenStein (Wed, 26 Apr 2017 04:24:18 GMT):
okay :)
BlurgenStein (Wed, 26 Apr 2017 04:40:48 GMT):
First I will read some more on Cello
baohua (Wed, 26 Apr 2017 05:11:16 GMT):
and we welcome to pick up existing jira task or create new ones
kletkeman (Wed, 26 Apr 2017 05:16:07 GMT):
Has joined the channel.
kletkeman (Wed, 26 Apr 2017 05:25:41 GMT):
@here -- high level question. Concerto became Fabric Composer, but does not actually manage fabrics, rather concerning itself with participants, data models, transactions (micro chaincode from my perspective) and ledgers. Cello has retained its name and concerns itself with fabrics and chains (channels).
It seems to me that these two should merge. Both are mostly stuck on v0.6 right now, although there are patches for v1 waiting to go into composer as soon as someone can figure out how to do that without killing momentum completely.
So my question is whether Cello will be merged into composer (as it truly *is* a fabric composer), at least in name and style if not directly into the code base? And the second question is how the v1 port is going, as I find the differences to be rather staggering at the fabric and client levels (not so much at the chaincode level)?
baohua (Wed, 26 Apr 2017 05:34:42 GMT):
Thanks for the advice. Because the scopes are quite different (concerto to help design/compose chaincode, cello to help manage/operate chains and do analytics), i do not think it's good idea. However, we can first see how each other can cooperate together in real scenarios. The v1 support is in implementation. Also, there are on-going work to support more underly infrastructures like k8s.
baohua (Wed, 26 Apr 2017 05:34:42 GMT):
Thanks for the advice. 1) Because the scopes are quite different (concerto to help design/compose chaincode, cello to help manage/operate chains and do analytics) currently, i do not think it's good idea. However, we are glad first to see how each other can cooperate together in real scenarios. 2) The v1 support is in implementation. Also, there are on-going work to support more underly infrastructures like k8s.
baohua (Wed, 26 Apr 2017 05:35:50 GMT):
By the way, we do not pick the name of composer from the beginning, as thinking there are already lots of projects with similar names, like Docker Composer. :)
baohua (Wed, 26 Apr 2017 07:08:19 GMT):
https://gerrit.hyperledger.org/r/#/c/8533/
MikeMayori (Wed, 26 Apr 2017 12:47:09 GMT):
@baohua Currently Cello is abandoned, does not work with fabric alpha v1
MikeMayori (Wed, 26 Apr 2017 12:47:28 GMT):
@baohua Composer will only work in IBM Bluemix Premises
kletkeman (Wed, 26 Apr 2017 13:13:39 GMT):
@MikeMayori It is my understanding that Composer is going to work with all fabric deployments, and the v1 port is ready for merge according to discussion on rocketchat. Regarding Cello, not having the v1 port done yet is not quite the same as "abandoned". Could you specify more clearly why you think both technologies are effectively retired? Because that does not jive at all with my understanding. Thanks.
kletkeman (Wed, 26 Apr 2017 13:13:39 GMT):
@MikeMayori It is my understanding that Composer is going to work with all fabric deployments, and the v1 port is ready for merge according to discussion on rocketchat. Regarding Cello, not having the v1 port done yet is not quite the same as "abandoned". Could you specify more clearly why you think both technologies are effectively retired? Because that does not jive at all with my understanding. Thanks. Edit: The link provided by @baohua indicates that Cello for v1 is also ready for merge.
baohua (Wed, 26 Apr 2017 13:15:57 GMT):
@MikeMayori i'm sorry, but from what would u say it's abandoned?
baohua (Wed, 26 Apr 2017 13:16:48 GMT):
if you mean the hyperledger cello, no, it's ongoing with many new interesting features. you can find the information at https://wiki.hyperledger.org/projects/cello.
kletkeman (Wed, 26 Apr 2017 13:17:41 GMT):
@baohua I understand that the scopes are quite different, but they overlap considerably. For example, there is a great opportunity to link Participants in the Composer data model to organizations and channels in Cello. At the least, the two should work together seamlessly, and at best they should be one Fabric Composer tool with a complete set of abilities to define and deploy fabrics in various configurations.
kletkeman (Wed, 26 Apr 2017 13:17:41 GMT):
@baohua I understand that the scopes are quite different, but they overlap considerably. For example, there is a great opportunity to link Participants in the Composer data model to organizations and channels in Cello. At the least, the two should work together seamlessly, and at best they should be one Fabric Composer tool with a complete set of abilities to define and deploy fabrics in various configurations. Edit: Channels and chaincodes are tightly linked (many to many relationships of course) and that is another opportunity for integration. The composer runtime does implement JavaScript microservice transactions, but they are all linked in the service of one network definition (forgot the name they use). If we leave the concepts too widely separated, then we run the risk of slowing adoption when the two are used together. An **either / or** situation would be damaging I think.
baohua (Wed, 26 Apr 2017 13:19:08 GMT):
thanks kletkeman, it would be interesting to see how to adopt ur mentioned model into Cello. Currently we do not cover much on the chaincode management, although there're some jira issues proposed by some people want to implement.
baohua (Wed, 26 Apr 2017 13:19:48 GMT):
So as the first step, we still wanna let Cello to handle well at the deployment and operation work over those blockchains like fabric, stl and iroha.
kletkeman (Wed, 26 Apr 2017 13:21:12 GMT):
@baohua Ok, so the focus is swarms and traditional Go and Java chaincodes, correct?
baohua (Wed, 26 Apr 2017 13:22:06 GMT):
Then we would like to see if it can handle more onto the front end, at that time, it may be interesting to see if people would like to use Cello to design and write the js transactions. I think I understand the fabric-composer project well as it's still incubated internally, and its target is to help design chaincode with high-level programming language like javascript, by removing the effort to write golang code from the scratch.
baohua (Wed, 26 Apr 2017 13:22:47 GMT):
Similar discussions happen before in the community, actually.
baohua (Wed, 26 Apr 2017 13:23:59 GMT):
The conclusion is that hyperledger is a collaborative projects, that would encourage the collaborations among various projects, and no need to group into unique one. e.g., the fabric, stl, iroha, all are ledger platforms, but we should welcome all existing. How do u think?
baohua (Wed, 26 Apr 2017 13:23:59 GMT):
The conclusion is that hyperledger is a collaborative project, that would encourage the collaborations among various projects, and no need to group into unique one. e.g., the fabric, stl, iroha, all are ledger platforms, but we should welcome all existing. How do u think?
kletkeman (Wed, 26 Apr 2017 13:24:12 GMT):
Yes, I am working with composer right now. It works with fabric definitions that enable security for Participants etc. Cello would never be used to design and write Composer chaincode, as that is what Composer does. But the business network definition (perhaps an instance of a system on a larger fabric) would be a good integration point.
baohua (Wed, 26 Apr 2017 13:25:32 GMT):
Maybe, but pls know that we do not wanna limit to fabric itself, hence we need to evaluate whether it will block cello to manage other blockchains like stl and iroha, to import such a fabric business work definition.
kletkeman (Wed, 26 Apr 2017 13:26:04 GMT):
For example ... define a complete system that uses smart contracts (or the finer grained transactions in composer) and export the BNA file, translating that as input to Cello, which orchestrates the creation of a working network based on parameters defined in Cello.
kletkeman (Wed, 26 Apr 2017 13:26:14 GMT):
Something like that would be pretty useful.
baohua (Wed, 26 Apr 2017 13:26:24 GMT):
Recently there are also related discussions. But we are open to all possible ideas, and welcome for advices.
baohua (Wed, 26 Apr 2017 13:26:41 GMT):
BNA file seems like some template?
kletkeman (Wed, 26 Apr 2017 13:27:23 GMT):
Ok, if Cello wants to be all singing and all dancing then that could override detailed integrations to specific tooling etc. for some time to come.
kletkeman (Wed, 26 Apr 2017 13:28:03 GMT):
BNA is a Business Network Archive, defining everything about a Composer based definition. Data model, Participants, Transactions and their implementations.
baohua (Wed, 26 Apr 2017 13:28:20 GMT):
And would BNA support blockchain platform besides fabric? If so, i personally feel interested to see whether we can together design some common/abstract format on general blockchain platform.
baohua (Wed, 26 Apr 2017 13:28:20 GMT):
And would BNA support blockchain platform besides fabric? If so, i personally feel interested to see whether we can together design some common/abstract format on more general blockchain platform.
kletkeman (Wed, 26 Apr 2017 13:30:12 GMT):
Presumably, someone could support blockchains other than Fabric with BNA. But that requires significant development resources, and getting one to work well should take priority in my opinion. For example, while Fabric and Ethereum state similar goals, Fabric has quite a different view of how to meet those goals. Trying to stay at the 10,000 foot level just means that a tool cannot service either well ...
baohua (Wed, 26 Apr 2017 13:30:38 GMT):
And it's not about all singing/dancing, as we proposed from the beginning of the project, we wanna provide more opinions for those potential hyperledger users. You can see there're various solutions by using different hyperledger tools. There were some specific "tool-suite" discussions before, too.
kletkeman (Wed, 26 Apr 2017 13:32:57 GMT):
Ok, I was not around at that time. I'm more focused on getting to the point where a detailed deployment with a complete definition can be performed easily. That requires pushing forward on one set of technologies. I will follow Cello as a non-integrated tool for now. It looks like it could prove very useful even without tight integration to tools like composer. Thanks.
baohua (Wed, 26 Apr 2017 13:32:58 GMT):
yeap, that's exactly what we are working on now. We noticed that such an abstract might be too difficult to finish everything. So we plan to start by supporting several. And since there's long roadmap to take, it would be a little too earlier to say which is the "best-practice". Actually, best-practice is coming from practices. Would u agree that?
baohua (Wed, 26 Apr 2017 13:35:00 GMT):
However, i would welcome u help open a jira issue to track the idea of letting cello support your BNA format. We definitely would like to support/work with more hyperledger projects. https://jira.hyperledger.org/projects/CE/issues. Thanks!
kletkeman (Wed, 26 Apr 2017 13:35:07 GMT):
I think we are just now developing an understanding of the value of blockchains in certain scenarios, and we are moving towards the ability to deploy Fabric on multiple topologies using multiple container technologies etc. So best practices will change as the Fabric community gets experience with real deployements over time.
kletkeman (Wed, 26 Apr 2017 13:36:19 GMT):
I'm not in the Composer team, but I will certainly look over them both if I can find the time. I have been hoping to find a tool to ease the pain of deploying complex Fabrics. Docker-compose helps a lot, but I don't think it's the solution ... just one piece.
baohua (Wed, 26 Apr 2017 13:36:56 GMT):
yeap, it will change, we have tried many topologies also, and find the underly technique is important, while there are also many key values from business, management and security conditions. That's really helpful if we can give some guideline/spec. That's also why the TWGC is working on the guideline/spec.
baohua (Wed, 26 Apr 2017 13:38:06 GMT):
I think u're talking about the operation, not only the deployment. the deployment usually happens once, while operations are more persistent and complex.
baohua (Wed, 26 Apr 2017 13:38:38 GMT):
We wanna start from deployment, and then evolve with more operational supports
baohua (Wed, 26 Apr 2017 13:39:17 GMT):
That's what we talk about the "management of blockchain", open to the smart contracts, too, if we believe it's part of the blockchain.
kletkeman (Wed, 26 Apr 2017 13:39:20 GMT):
Yes, I see them as the same thing, because deployments evolve over time. Added Participants usually imply added endorsers / orderers / observers ... that kind of thing. The ability to deploy all that automatically after updating a business network definition would be magical.
kletkeman (Wed, 26 Apr 2017 13:39:47 GMT):
And added participants can join channels, so everything is interconnected.
baohua (Wed, 26 Apr 2017 13:40:28 GMT):
Agree, let's help contribute into those hyperledger projects/tools, and make this real someday soon.
kletkeman (Wed, 26 Apr 2017 13:41:14 GMT):
Sounds like a good idea ... I'll poke around a bit more. I also have a chaincode platform for IoT contracts that could benefit, and it is traditional Go language.
baohua (Wed, 26 Apr 2017 13:41:32 GMT):
And would like to thanks a lot for proposing so many nice advices~~
kletkeman (Wed, 26 Apr 2017 13:41:55 GMT):
You are very welcome. We all have a lot to gain if anything like this can be made to happen.
baohua (Wed, 26 Apr 2017 13:42:23 GMT):
Yeap, and the business value are more likely at the higher layers :)
kletkeman (Wed, 26 Apr 2017 13:43:16 GMT):
Even with limitations, orchestration of business networks (as defined by BNA format for example) is a big deal once extended to Fabric topologies.
baohua (Wed, 26 Apr 2017 13:44:31 GMT):
aha, say how much would u like to pay for using the fabric-composer (BNA)?
baohua (Wed, 26 Apr 2017 13:44:31 GMT):
aha, say how much would u like to pay for using the fabric-composer (BNA)?:grimacing:
baohua (Wed, 26 Apr 2017 13:44:51 GMT):
I would evaluate how much we lost by open-sourcing it :thinking:
baohua (Wed, 26 Apr 2017 13:44:51 GMT):
I would evaluate how much we have lost by open-sourcing it :thinking:
kletkeman (Wed, 26 Apr 2017 13:48:34 GMT):
An interesting question ...
marek.dapps (Thu, 27 Apr 2017 10:17:34 GMT):
Has joined the channel.
wsh_bob (Fri, 28 Apr 2017 08:13:54 GMT):
Has joined the channel.
krasi69 (Sat, 29 Apr 2017 14:33:10 GMT):
Has joined the channel.
sstone1 (Sat, 29 Apr 2017 14:33:57 GMT):
Has joined the channel.
sstone1 (Sat, 29 Apr 2017 14:46:21 GMT):
Feel like I've missed out on a good debate here ;-)
sstone1 (Sat, 29 Apr 2017 14:48:43 GMT):
@MikeMayori where did you get the impression that "Composer will only work in IBM Bluemix Premises"? We work against all Hyperledger Fabric v0.6 and v1.0 instances. We would not have been accepted as a top level project by the Hyperledger TSC if we only worked on IBM Bluemix.
sstone1 (Sat, 29 Apr 2017 14:49:49 GMT):
And following on from that, Composer today does support Hyperledger Fabric v1.0... the pending work is to update the playground to support connections to real Blockchain platforms (which _should_ go in this week).
sstone1 (Sat, 29 Apr 2017 14:52:20 GMT):
> For example, there is a great opportunity to link Participants in the Composer data model to organizations and channels in Cello
This is something the Composer team is already discussing - we are talking about a "binding" between the Composer data model and the Fabric topology.
sstone1 (Sat, 29 Apr 2017 14:55:57 GMT):
> And would BNA support blockchain platform besides fabric?
The Composer code base is 90-95% independent of the Blockchain platform. The remaining Blockchain platform specific code is completely pluggable, so adding support for the Sawtooth, Iroha, Burrow platforms is something that is totally possible. Indeed I had a good discussion in DC last week with the Burrow guys about how we could link up Burrow and Composer.
sstone1 (Sat, 29 Apr 2017 14:56:34 GMT):
And I agree that Cello should continue to support multiple deployment formats and Blockchain platforms :)
sstone1 (Sat, 29 Apr 2017 14:57:07 GMT):
I understand people still wanting to write chaincode, but also the possibility of someone building a new higher level framework on top of Composer.
sstone1 (Sat, 29 Apr 2017 14:58:01 GMT):
But the idea of being able to give a BNA to Cello and have Cello deploy a Fabric instance and the BNA + Composer chaincode to that instance is very interesting...
sstone1 (Sat, 29 Apr 2017 14:59:20 GMT):
If/when we add the binding between the Composer business network and the Fabric "physical" network then Cello could consume that binding information in order to determine what nodes/channels to lay out
sstone1 (Sat, 29 Apr 2017 14:59:42 GMT):
@baohua @kletkeman ping for the above ;)
baohua (Sun, 30 Apr 2017 13:45:13 GMT):
exactly, one interesting point is to use BNA model as the bridge. I would welcome if anyone wanna create a jira task on this.
vfrancoise (Sun, 30 Apr 2017 17:23:12 GMT):
Has joined the channel.
duncanjw (Mon, 01 May 2017 01:09:29 GMT):
@baohua at a high level this is our vision https://www.dropbox.com/s/v11v6jwre19zaqp/Screenshot%202017-04-30%2018.08.58.png?dl=0
duncanjw (Mon, 01 May 2017 01:12:06 GMT):
In terms of Framework Management we have so far focused our efforts on Hyperledger Fabric and we our contribution being a set of Apache Brooklyn blueprints that form part of Cello
duncanjw (Mon, 01 May 2017 01:12:36 GMT):
Most recently we have demonstrated that we can deploy the Hyperledger Fabric to an arbitrary Kubernetes cluster (in this case one created using IBM Bluemix but provided we have the Kube config file this should work with any cluster)
duncanjw (Mon, 01 May 2017 01:13:08 GMT):
We've blogged about this https://cloudsoft.io/blog/deploying-hyperledger-fabric-on-ibm-bluemix-kubernetes-cluster-using-amp
duncanjw (Mon, 01 May 2017 01:13:43 GMT):
BTW is there a simple way of enabling attachments to messages in this channel?
duncanjw (Mon, 01 May 2017 01:15:21 GMT):
We will be at the Hyperledger mini-summit May 22-23 in NYC where we will be able to provide an update on concrete steps being taken by @mikezaccardo @grkvlt to achieve this
rjones (Mon, 01 May 2017 18:33:34 GMT):
Has left the channel.
yahtoo (Tue, 02 May 2017 03:10:36 GMT):
Has joined the channel.
baohua (Tue, 02 May 2017 06:18:22 GMT):
Thanks duncanjw! And would like to suggest u help submit those kube config template files to the Cello project, and we can keep updating there to support the k8s as the underly platform.
lixucheng (Tue, 02 May 2017 07:15:42 GMT):
Has joined the channel.
lixucheng (Tue, 02 May 2017 07:17:51 GMT):
@baohua I have pushed some dashboard code to git, please review it. The url is: http://gerrit.hyperledger.org/r/8829
baohua (Tue, 02 May 2017 07:53:00 GMT):
ok, thanks
baohua (Tue, 02 May 2017 07:54:44 GMT):
pls help include the CE jira task number in the commit msg. See https://github.com/hyperledger/cello/blob/master/docs/CONTRIBUTING.md
baohua (Tue, 02 May 2017 08:05:03 GMT):
@rjones dear ry, could u help have a look at https://gerrit.hyperledger.org/r/#/c/8829/. Seems the error is that our ci cannot access the pypi.douban.com mirror server.
rjones (Tue, 02 May 2017 08:05:03 GMT):
Has joined the channel.
rjones (Tue, 02 May 2017 17:01:39 GMT):
@baohua it looks like the second attempt passed. I know there was a major internet issue last night - Telia was down, taking down a lot of big websites (reddit, cloudflare(!), parts of AWS). Also, please ask questions like that in #ci-pipeline so my colleague @jwagantall will see them
jwagantall (Tue, 02 May 2017 17:01:39 GMT):
Has joined the channel.
baohua (Wed, 03 May 2017 03:36:38 GMT):
thanks, ry!
lixucheng (Thu, 04 May 2017 01:43:15 GMT):
@baohua I pushed the code of user dashboard's home page, please review it. URL is: http://gerrit.hyperledger.org/r/8933
baohua (Thu, 04 May 2017 01:43:39 GMT):
sure thanks!
baohua (Thu, 04 May 2017 01:44:05 GMT):
btw, do we have the setup documentation on how to bootup the dashboard?
baohua (Thu, 04 May 2017 01:44:32 GMT):
And pls post the patchset url as comments to the jira task, thanks
lixucheng (Thu, 04 May 2017 01:44:52 GMT):
ok
baohua (Fri, 05 May 2017 01:47:04 GMT):
need more review here: https://gerrit.hyperledger.org/r/8933
lixucheng (Fri, 05 May 2017 08:41:01 GMT):
@baohua please review the patchset: http://gerrit.hyperledger.org/r/8999 for chain CRUD functions
baohua (Fri, 05 May 2017 08:42:48 GMT):
sure, thanks, pls add link to the jira task
lixucheng (Fri, 05 May 2017 08:43:29 GMT):
yes, the link is added to the task
silliman (Fri, 05 May 2017 14:07:46 GMT):
Has joined the channel.
himansri (Sun, 07 May 2017 21:28:20 GMT):
Has joined the channel.
lixucheng (Mon, 08 May 2017 03:45:43 GMT):
@baohua The smart contract functions for user dashboard have been pushed, please review the patch set: http://gerrit.hyperledger.org/r/9081
baohua (Mon, 08 May 2017 03:46:28 GMT):
ok thanks
lixucheng (Mon, 08 May 2017 03:46:29 GMT):
and the link has been added to jira task
chawlanikhil24 (Mon, 08 May 2017 11:59:40 GMT):
Has joined the channel.
chawlanikhil24 (Mon, 08 May 2017 12:02:23 GMT):
@baohua Hi
chawlanikhil24 (Mon, 08 May 2017 12:02:41 GMT):
can you guide me a bit about this commit,
https://gerrit.hyperledger.org/r/#/c/9085/
chawlanikhil24 (Mon, 08 May 2017 12:03:10 GMT):
I created an Issue on JIRA, CE-37
chawlanikhil24 (Mon, 08 May 2017 12:03:33 GMT):
But how to link commit, I wanna know
bmkor (Mon, 08 May 2017 14:06:42 GMT):
Has joined the channel.
baohua (Tue, 09 May 2017 03:13:12 GMT):
aha, just post the gerrit link as comments to the jira task @chawlanikhil24
siddhantgohil (Tue, 09 May 2017 03:14:41 GMT):
Has joined the channel.
ZoltanSzabo (Tue, 09 May 2017 07:20:49 GMT):
Has joined the channel.
jchenibm (Tue, 09 May 2017 13:39:40 GMT):
Has joined the channel.
theofilis (Tue, 09 May 2017 20:35:40 GMT):
Has joined the channel.
theofilis (Tue, 09 May 2017 20:43:30 GMT):
Has left the channel.
lixucheng (Wed, 10 May 2017 03:54:53 GMT):
@baohua The patch set url: http://gerrit.hyperledger.org/r/9171 for chain detail functions, please review it.
baohua (Wed, 10 May 2017 05:55:46 GMT):
thanks
chawlanikhil24 (Thu, 11 May 2017 07:56:12 GMT):
Hi people!. I think the "install.md" needs correction, the specified location for Docker service file in Ubuntu 16.04 is "/lib/systemd/system/docker.service" not "/etc/systemd/system/docker.service.d/override.conf"
baohua (Thu, 11 May 2017 07:58:24 GMT):
thanks for the reporting, @chawlanikhil24 , would u like to help make a fix to the doc?
chawlanikhil24 (Thu, 11 May 2017 07:58:38 GMT):
ya sure! :)
baohua (Thu, 11 May 2017 07:58:41 GMT):
it should be easy with several steps, https://github.com/hyperledger/cello/blob/master/docs/CONTRIBUTING.md :)
baohua (Thu, 11 May 2017 07:58:44 GMT):
thanks
baohua (Thu, 11 May 2017 08:28:10 GMT):
hi, @chawlanikhil24 thanks for the patchset, and i suppose the regeneration cmd will update the "/lib/systemd/system/docker.service"?
baohua (Thu, 11 May 2017 08:28:28 GMT):
it should be regenerated by "/etc/systemd/system/docker.service.d/override.conf"?
chawlanikhil24 (Thu, 11 May 2017 08:41:54 GMT):
@baohua Made a PR, you can review it
chawlanikhil24 (Thu, 11 May 2017 08:42:30 GMT):
and can you tell me what's the default username and password for cello dashboard running at localhost:8080
baohua (Thu, 11 May 2017 08:42:39 GMT):
yeap, it's admin:pass
baohua (Thu, 11 May 2017 08:42:51 GMT):
yes, have replied ur pr
chawlanikhil24 (Thu, 11 May 2017 08:42:55 GMT):
thank you very much
baohua (Thu, 11 May 2017 08:43:04 GMT):
np :)
chawlanikhil24 (Thu, 11 May 2017 08:59:02 GMT):
@baohua , When I tried to find this file on my machine, Ended up with nothing
chawlanikhil24 (Thu, 11 May 2017 08:59:53 GMT):
regeneration will only occur if "/etc/systemd/system/docker.service.d/override.conf" exists on system!
chawlanikhil24 (Thu, 11 May 2017 09:01:30 GMT):
still, you can clear it by simply "sudo cat /etc/systemd/system/docker.service.d/override.conf", if you are using Ubuntu 16.04
Jay (Thu, 11 May 2017 09:53:46 GMT):
Has joined the channel.
baohua (Thu, 11 May 2017 12:22:02 GMT):
thanks
baohua (Thu, 11 May 2017 12:23:37 GMT):
@hightall could u help verify it on ubuntu 16.04?
dhuseby (Thu, 11 May 2017 14:13:19 GMT):
Has joined the channel.
chawlanikhil24 (Thu, 11 May 2017 15:29:44 GMT):
@baohua fixed the ci error :)
baohua (Fri, 12 May 2017 01:21:46 GMT):
thanks
baohua (Fri, 12 May 2017 01:49:44 GMT):
bug reported, welcome to help fix: https://jira.hyperledger.org/browse/CE-43
baohua (Fri, 12 May 2017 01:49:44 GMT):
documentation link broken bug reported, welcome to help fix: https://jira.hyperledger.org/browse/CE-43
qiang0723 (Fri, 12 May 2017 02:13:03 GMT):
I'll fix it
baohua (Fri, 12 May 2017 02:27:34 GMT):
thanks
baohua (Fri, 12 May 2017 03:32:34 GMT):
https://gerrit.hyperledger.org/r/#/c/9269/
yury (Sat, 13 May 2017 09:11:51 GMT):
Has joined the channel.
chawlanikhil24 (Sun, 14 May 2017 17:40:30 GMT):
Hi people!..
chawlanikhil24 (Sun, 14 May 2017 17:41:37 GMT):
I've a suggestion that can be added to install.md
chawlanikhil24 (Sun, 14 May 2017 17:43:37 GMT):
as the tutorial says to add the host and port to Docker Service file, instead there is also an alternative without altering any service file
chawlanikhil24 (Sun, 14 May 2017 17:46:12 GMT):
```
sudo systemctl stop docker.service
dockerd -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --api-cors-header='*' --default-ulimit=nofile=8192:16384 --default-ulimit=nproc=8192:16384 -D &
```
This will run the docker daemon on the specified host in the background, I've tested and working very well.
Please share you opinion.
lixucheng (Mon, 15 May 2017 03:49:06 GMT):
@baohua Please review the patch set: http://gerrit.hyperledger.org/r/9351 for user dashboard analytics functions.
baohua (Mon, 15 May 2017 05:29:06 GMT):
@chawlanikhil24 would welcome to add the instructions in the doc as an optional way to run Docker, thanks!
baohua (Mon, 15 May 2017 05:29:19 GMT):
@lixucheng thanks for the patchset!
mcdullsin (Mon, 15 May 2017 06:07:55 GMT):
Has joined the channel.
chawlanikhil24 (Mon, 15 May 2017 06:24:06 GMT):
@baohua Sure ! :D
chawlanikhil24 (Mon, 15 May 2017 06:25:46 GMT):
and please guide me, how to use kubernetes on cello ?
chawlanikhil24 (Mon, 15 May 2017 08:52:01 GMT):
@here Any documentation for kubernetes on cello ?
chawlanikhil24 (Mon, 15 May 2017 08:52:16 GMT):
I don't know how to use :/
chawlanikhil24 (Mon, 15 May 2017 10:47:32 GMT):
@baohua please review : https://gerrit.hyperledger.org/r/#/c/9359/
clasoncjDA (Mon, 15 May 2017 14:47:17 GMT):
Has joined the channel.
baohua (Tue, 16 May 2017 02:33:27 GMT):
@chawlanikhil24 The k8s support was on working. welcome to discuss at https://jira.hyperledger.org/projects/CE/issues/CE-7?filter=allopenissues
zemtsov (Tue, 16 May 2017 08:49:19 GMT):
Has left the channel.
rjones (Tue, 16 May 2017 19:37:42 GMT):
Has left the channel.
lixucheng (Wed, 17 May 2017 03:58:19 GMT):
@baohua Please review the patch set: http://gerrit.hyperledger.org/r/9435 for user dashboard
baohua (Wed, 17 May 2017 05:45:22 GMT):
:grinning:
timblankers (Thu, 18 May 2017 12:07:56 GMT):
Has joined the channel.
DaVi (Sat, 20 May 2017 09:07:57 GMT):
Hello , could someone give me some help ? I'm trying to commit on gerrit for CE-47 .. I stricktly followed the procedure on the contributing page ..and here is the result :
DaVi (Sat, 20 May 2017 09:08:05 GMT):
[root@cello-dev cello]# git clone ssh://DaVi@gerrit.hyperledger.org:29418/cello && scp -p -P 29418 DaVi@gerrit.hyperledger.org:hooks/commit-msg cello/.git/hooks/
Cloning into 'cello'...
remote: Total 1063 (delta 0), reused 1063 (delta 0)
Receiving objects: 100% (1063/1063), 5.86 MiB | 1.42 MiB/s, done.
Resolving deltas: 100% (293/293), done.
commit-msg
[root@cello-dev ~]# cd cello
[root@cello-dev cello]# git config user.name Daniel Vielvoye
[root@cello-dev cello]# git config user.email dvielvoye@gmail.com
[root@cello-dev cello]# git checkout -b CE-47
Switched to a new branch 'CE-47'
[root@cello-dev cello]# rm scripts/setup.sh
rm : supprimer fichier « scripts/setup.sh » ? y
[root@cello-dev cello]# cd scripts/
[root@cello-dev scripts]# cp /root/setup.sh .
[root@cello-dev scripts]# nano setup.sh
[root@cello-dev scripts]#
[root@cello-dev scripts]# git add .
[root@cello-dev scripts]# git status
# On branch CE-47
# Changes to be committed:
# (use "git reset HEAD
baohua (Sat, 20 May 2017 09:58:00 GMT):
seems the pushing url (7915) is merged
baohua (Sat, 20 May 2017 09:58:14 GMT):
could u help try a new jira task number with new branch?
DaVi (Sat, 20 May 2017 17:40:24 GMT):
ok ... new jira is CE-49 . This time everything went ok !
baohua (Mon, 22 May 2017 01:32:17 GMT):
aha, seems ci does not passed, need to check its log from the ci's comment msg.
baohua (Mon, 22 May 2017 01:32:25 GMT):
thanks for ur help!
lixucheng (Mon, 22 May 2017 01:50:09 GMT):
@baohua I have pushed the patch set for user dashboard document. The url is: http://gerrit.hyperledger.org/r/9641. Please review it.
Calvin_Heo (Mon, 22 May 2017 02:53:00 GMT):
Has joined the channel.
Robtg4 (Mon, 22 May 2017 06:39:03 GMT):
Has joined the channel.
DaVi (Mon, 22 May 2017 19:08:32 GMT):
@baohua I have pushed the pach for CE-49(mutliple linux distros ) . URL is https://gerrit.hyperledger.org/r/9617 . I also pushed the patch for CE-41 (Swarm detection) ,the url is https://gerrit.hyperledger.org/r/9627 . Please review both ! Thanks in advance.
baohua (Tue, 23 May 2017 02:34:51 GMT):
@DaVi ,thanks david, however, the ci fails, could u help check the out log and help fix the error?
baohua (Tue, 23 May 2017 02:35:11 GMT):
@lixucheng some comments are put there for the patch, thanks!
DaVi (Tue, 23 May 2017 09:11:13 GMT):
@baohua I fixed the issues . For the CE-49 ,here is the new link : https://gerrit.hyperledger.org/r/#/c/9685/2 . I did an abandon on the first one .
baohua (Wed, 24 May 2017 01:41:18 GMT):
Thanks Davi!
ssaddem (Thu, 25 May 2017 15:54:14 GMT):
Has joined the channel.
hmchen (Thu, 25 May 2017 22:12:30 GMT):
Has joined the channel.
yahtoo (Fri, 26 May 2017 16:10:59 GMT):
@baohua I have pushed the pach for CE-4(fabric 1.0 support ) . URL is https://gerrit.hyperledger.org/r/9869 .
baohua (Sat, 27 May 2017 10:03:01 GMT):
ci fails... have u run 'make check' locally?
bmkor (Sat, 27 May 2017 13:06:52 GMT):
Hi folks. May I ask if what's the channel name for cello-analytic, if any?
indirajith (Sat, 27 May 2017 14:48:16 GMT):
Hi all, I am experiencing a problem while setting up Fabric v1.0. The orderer container exits immediately. I am not able to overcome this issue for last 4 or 5 days. Any pointers are welcome. Thanks in advance.
indirajith (Sat, 27 May 2017 14:51:25 GMT):
Message Attachments
bmalavan (Sat, 27 May 2017 15:29:29 GMT):
Has joined the channel.
baohua (Sun, 28 May 2017 13:06:32 GMT):
@bmkor currently it also use this channel for the discussion :)
baohua (Sun, 28 May 2017 13:06:32 GMT):
@bmkor currently it also uses this channel for the discussion :)
baohua (Sun, 28 May 2017 13:07:09 GMT):
@indirajith would u have a look at https://github.com/yeasy/docker-compose-files/tree/master/hyperledger/1.0? I guess the exit is because the running of orderer cmd fails
bmkor (Sun, 28 May 2017 13:07:56 GMT):
I see. Thanks.
baohua (Sun, 28 May 2017 13:09:33 GMT):
are u feel interested with the project? Welcome for any question of fix :)
bmkor (Sun, 28 May 2017 13:33:51 GMT):
Yes. I am particular interested in how data analysis being carried out.
baohua (Sun, 28 May 2017 13:35:04 GMT):
sure, bmkor, currently we collect logs and analyze using ELK. more features are adding, welcome for any idea
bmkor (Sun, 28 May 2017 13:56:52 GMT):
Thanks. Would there be any graph analysis as well? Sorry that I don't familiar with ELK.
compleatang (Sun, 28 May 2017 16:44:08 GMT):
Has joined the channel.
bmkor (Mon, 29 May 2017 00:37:37 GMT):
Perhaps we can have features facilitiating both supervised and unsupervised learnings, and graph analysis?
baohua (Mon, 29 May 2017 02:46:41 GMT):
aha, thanks for the comments, bmkor, i guess there's no limit on what we can do after collecting those data from the network. would u like to help create a cello jira task to see if there's more disucssion on this idea?
chawlanikhil24 (Mon, 29 May 2017 19:12:29 GMT):
Hi people!.. I want to understand the code of cello, as in terms of how it does work ? can anyone give some brief! Help appreciated
baohua (Tue, 30 May 2017 01:21:14 GMT):
@chawlanikhil24 currently, the best way is to read the doc: https://github.com/hyperledger/cello/. If you have further question, can post to hyperledger-cello@lists.hyperledger.org or ask here :)
chawlanikhil24 (Tue, 30 May 2017 06:51:35 GMT):
@baohua I've been through this but needed some more
chawlanikhil24 (Tue, 30 May 2017 06:51:51 GMT):
Still thanks !.. I'll raise the questions here
bmkor (Tue, 30 May 2017 10:32:44 GMT):
My pleasure. Let me have it done in a few days. [ ](https://chat.hyperledger.org/channel/cello?msg=D6uG4eSPecaRN9Le2) @baohua
SDChoi (Wed, 31 May 2017 01:04:25 GMT):
Has joined the channel.
lixucheng (Wed, 31 May 2017 05:34:38 GMT):
@baohua I have updated to use docker compose, please review it. The patch set is: http://gerrit.hyperledger.org/r/9641
kmohanar (Wed, 31 May 2017 09:19:20 GMT):
Has joined the channel.
holocron (Wed, 31 May 2017 21:04:19 GMT):
Has left the channel.
baohua (Thu, 01 Jun 2017 02:08:27 GMT):
@ dwakeman is also working on supporting fabric on top of k8s, any status to update? @FlyingTiger
baohua (Thu, 01 Jun 2017 02:11:24 GMT):
@lixucheng thanks, and seems network fails during the ci running. just retrigger it.
chawlanikhil24 (Thu, 01 Jun 2017 05:50:08 GMT):
Has there been any development on this issues ?
https://jira.hyperledger.org/projects/CE/issues/CE-7?filter=allopenissues
chawlanikhil24 (Thu, 01 Jun 2017 05:50:08 GMT):
Has there been any development on this issue ?
https://jira.hyperledger.org/projects/CE/issues/CE-7?filter=allopenissues
chawlanikhil24 (Thu, 01 Jun 2017 05:51:20 GMT):
for this, I wanna know how swarm supports orchestration here!. anyone help please
baohua (Thu, 01 Jun 2017 06:28:13 GMT):
basically, the swam provided similar restful api with docker. and it also support overlay networking across hosts.
chawlanikhil24 (Thu, 01 Jun 2017 06:38:17 GMT):
@baohua so we use any script to call those API of DOcker and Swarm ?
baohua (Fri, 02 Jun 2017 01:36:18 GMT):
we use compose files to call those APIs to help boot up containers, you can see the code under https://github.com/hyperledger/cello/blob/master/src/agent/docker
david_dornseifer (Fri, 02 Jun 2017 03:58:05 GMT):
Has joined the channel.
cautiousoptimist (Fri, 02 Jun 2017 12:07:12 GMT):
Has joined the channel.
saism (Fri, 02 Jun 2017 12:25:32 GMT):
Has left the channel.
chawlanikhil24 (Mon, 05 Jun 2017 05:23:42 GMT):
Thank you very much @baohua
chawlanikhil24 (Mon, 05 Jun 2017 11:50:37 GMT):
And I want to share that, I am the intern selected for Hyperledger/Cello, "Deploy fabric on Kubernetes using cello"
chawlanikhil24 (Mon, 05 Jun 2017 12:51:54 GMT):
So, I hope, with your guidance, I'll be able to Complete this project successfully
travin (Mon, 05 Jun 2017 19:53:16 GMT):
Has joined the channel.
travin (Mon, 05 Jun 2017 19:53:49 GMT):
Hi Hyperledger Cello team. I'm from the White Paper WG and we're looking for someone to write the overview section about Cello as well as perhaps include a highlighted feature for that section. If you're interested, or know someone I should contact, please send me a private message about it. Thank you for your time!
dorian000 (Tue, 06 Jun 2017 01:41:20 GMT):
Has joined the channel.
baohua (Tue, 06 Jun 2017 02:21:55 GMT):
sure, travin
baohua (Tue, 06 Jun 2017 02:22:32 GMT):
Besides, there are several design documentation work ongoing, welcome anyone for comments, thanks! https://wiki.hyperledger.org/projects/cello?documentation
sidrmsh (Tue, 06 Jun 2017 22:30:46 GMT):
Has joined the channel.
dstarina (Wed, 07 Jun 2017 14:32:15 GMT):
Has joined the channel.
dstarina (Thu, 08 Jun 2017 07:42:02 GMT):
Hi. I have installed Cello using the manual here: https://github.com/yeasy/cello/blob/master/docs/deployment.md. Had some problems with docker daemons seeing each other, so from the file /etc/systemd/system/docker.service.d/override.conf I removed "-H fd:// $DOCKER_OPTS" (this stackoverflow resolution: https://stackoverflow.com/questions/44411828/cannot-connect-to-the-docker-daemon-port-2375?noredirect=1#44412406), and added '{"hosts": ["tcp://0.0.0.0:2375", "unix:///var/run/docker.sock"]}' to file /etc/docker/daemon.json. I did everything else exactly per instructions, and the installation seemed go to through fine, without any errors. However, when opening the dashboard (hostname:8080), after entering credentials (admin/pass), I get "404 Not Found". Did I miss something? How can I debug this?
chawlanikhil24 (Thu, 08 Jun 2017 11:25:47 GMT):
have you confirmed that docker is running of port localhost:2375 ?
chawlanikhil24 (Thu, 08 Jun 2017 11:26:35 GMT):
netstat | grep "2375
chawlanikhil24 (Thu, 08 Jun 2017 11:26:35 GMT):
netstat | grep "2375"
dstarina (Thu, 08 Jun 2017 11:44:10 GMT):
the netstat command doesn't get me any results
dstarina (Thu, 08 Jun 2017 11:45:16 GMT):
however, "docker -H 10.101.35.60:2375 version" does connect successfully
baohua (Thu, 08 Jun 2017 11:49:45 GMT):
so u can access the dashboard at hostname:8080, but after the credential entering, the page show 404?
baohua (Thu, 08 Jun 2017 11:50:58 GMT):
can u check if there's any error in the logs? `docker-compose logs -f`
baohua (Thu, 08 Jun 2017 11:52:13 GMT):
and also `docker ps -a` to show the running containers
dstarina (Thu, 08 Jun 2017 11:54:30 GMT):
Yes, that is correct - i can access it (get prompted for username/password), but the get 404
dstarina (Thu, 08 Jun 2017 11:54:30 GMT):
Yes, that is correct - i can access it (get prompted for username/password), but then get 404 after entering credentials
dstarina (Thu, 08 Jun 2017 11:54:38 GMT):
There's this error in the logs:
dstarina (Thu, 08 Jun 2017 11:54:38 GMT):
ERROR:
Can't find a suitable configuration file in this directory or any
parent. Are you in the right directory?
Supported filenames: docker-compose.yml, docker-compose.yaml
dstarina (Thu, 08 Jun 2017 11:55:30 GMT):
These are the running containers:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6c70e0176049 yeasy/nginx "/bin/bash /tmp/do..." 21 hours ago Up 21 hours 0.0.0.0:80->80/tcp, 0.0.0.0:8080->8080/tcp, 443/tcp nginx
4784c8d0f9ea mongo:3.2 "docker-entrypoint..." 21 hours ago Up 21 hours 127.0.0.1:27017-27018->27017-27018/tcp mongo
6f364e9aee5f cello-dashbard "python dashboard.py" 21 hours ago Restarting (1) 6 hours ago dashbard
dc992dec5860 cello-watchdog "python watchdog.py" 21 hours ago Restarting (1) 6 hours ago watchdog
f6922c6f8a30 cello-restserver "python restserver.py" 21 hours ago Restarting (1) 6 hours ago
dstarina (Thu, 08 Jun 2017 12:14:22 GMT):
Sorry, didn't view logs from the right directory ...
dstarina (Thu, 08 Jun 2017 12:17:05 GMT):
Here are the logs ...
dstarina (Thu, 08 Jun 2017 12:17:05 GMT):
Here are the logs from docker ...
dstarina (Thu, 08 Jun 2017 12:17:45 GMT):
Message Attachments
baohua (Thu, 08 Jun 2017 12:25:15 GMT):
as the logs shows, cello-dashbard, watchdog, restserver containers are keeping restarting
baohua (Thu, 08 Jun 2017 12:27:04 GMT):
the logs keep complaining "[31mrestserver |[0m ImportError: cannot import name 'Client'"
baohua (Thu, 08 Jun 2017 12:28:13 GMT):
can u confirm ur docker-compose version? docker-compose>=1.7.0,<=1.9.0
baohua (Thu, 08 Jun 2017 12:28:41 GMT):
there's known problem with latest docker-compose version, which conflicts with docker-py
baohua (Thu, 08 Jun 2017 12:30:23 GMT):
by the way, can u follow the steps at https://github.com/hyperledger/cello/blob/master/docs/install.md for a fresh installation? it's strange to have this problem. we will need to see the cause, thanks!
dstarina (Thu, 08 Jun 2017 12:35:15 GMT):
docker-compose version 1.13.0
dstarina (Thu, 08 Jun 2017 12:35:25 GMT):
so I need to downgrade docker-compose?
dstarina (Thu, 08 Jun 2017 12:35:33 GMT):
to 1.9?
dstarina (Thu, 08 Jun 2017 12:37:37 GMT):
I followed the steps at that link exactly (I can of course do it again), except for this:
dstarina (Thu, 08 Jun 2017 12:37:39 GMT):
Had some problems with docker daemons seeing each other, so from the file /etc/systemd/system/docker.service.d/override.conf I removed "-H fd:// $DOCKER_OPTS" (this stackoverflow resolution: https://stackoverflow.com/questions/44411828/cannot-connect-to-the-docker-daemon-port-2375?noredirect=1#44412406), and added '{"hosts": ["tcp://0.0.0.0:2375", "unix:///var/run/docker.sock"]}' to file /etc/docker/daemon.json.
dstarina (Thu, 08 Jun 2017 12:38:30 GMT):
I don't know much about docker so I don't know whether these changes could be the problem?
chawlanikhil24 (Thu, 08 Jun 2017 12:49:41 GMT):
@baohua should this be an issue on JIRA ?
baohua (Thu, 08 Jun 2017 12:50:25 GMT):
sure, pls help create a jira issue first, let's explore that more...
baohua (Thu, 08 Jun 2017 12:50:52 GMT):
@dstarina usually as all codes are running inside container, the version is fixed inside those containers.
chawlanikhil24 (Thu, 08 Jun 2017 12:51:11 GMT):
I am using docker-compose 1.8 and it worked with the official documentation provided on github @dstarina
baohua (Thu, 08 Jun 2017 12:51:42 GMT):
can i know the os info on ur running host? @dstarina
baohua (Thu, 08 Jun 2017 12:53:17 GMT):
yeap, at least 1.7~1.9, we have tested that :)
dstarina (Thu, 08 Jun 2017 13:49:13 GMT):
I'm running Ubuntu 16.04
dstarina (Thu, 08 Jun 2017 13:49:48 GMT):
docker-compose was installed together with docker, what docker version should I install then?
baohua (Thu, 08 Jun 2017 13:56:13 GMT):
aha, i suppose both docker 1.12 and 1.13 should work
baohua (Thu, 08 Jun 2017 13:56:47 GMT):
and the config u made should have nothing related to the error log, i guess
baohua (Thu, 08 Jun 2017 13:56:47 GMT):
and the docker config change u made should have nothing related to the error log, i guess
chawlanikhil24 (Thu, 08 Jun 2017 13:58:14 GMT):
@dstarina I am using the exactly same environement
baohua (Thu, 08 Jun 2017 14:01:03 GMT):
https://gerrit.hyperledger.org/r/#/c/9641/
baohua (Thu, 08 Jun 2017 14:02:17 GMT):
@dstarina can u try `make clean && make restart`?
dstarina (Thu, 08 Jun 2017 14:28:23 GMT):
@baohua just did that (before reinstalling docker). Same problem remains
baohua (Thu, 08 Jun 2017 14:28:59 GMT):
ok, so u're sure to have run the latest code, right?
baohua (Thu, 08 Jun 2017 14:29:22 GMT):
i will find some fresh machine to test, sorry for the inconvenience.
dstarina (Thu, 08 Jun 2017 14:29:40 GMT):
it was downloading bunch of stuff, so, probably yes
dstarina (Thu, 08 Jun 2017 14:30:09 GMT):
hey, no problem, thanks for your help
dstarina (Thu, 08 Jun 2017 14:31:19 GMT):
does it help you if I send the exact set of commands I used to install this?
dstarina (Thu, 08 Jun 2017 14:31:55 GMT):
together with the docker install, can send you basically the whole set of commands I used to set this up from the fresh install
dstarina (Thu, 08 Jun 2017 14:34:39 GMT):
Message Attachments
silliman (Thu, 08 Jun 2017 14:57:06 GMT):
Has left the channel.
baohua (Thu, 08 Jun 2017 15:12:02 GMT):
could u help create a jira bug issue in cello project? I will keep that to confirm later. thanks!
chawlanikhil24 (Thu, 08 Jun 2017 16:18:33 GMT):
@baohua I've create an issue for this where I've attached the log provided by @dstarina
chawlanikhil24 (Thu, 08 Jun 2017 16:18:34 GMT):
https://jira.hyperledger.org/projects/CE/issues/CE-54?filter=allopenissues
chawlanikhil24 (Thu, 08 Jun 2017 16:19:05 GMT):
we can further continue the discussion here
bmkor (Fri, 09 Jun 2017 00:55:42 GMT):
@baohua sorry that I'll go back to this later as testing fabric v1.0 beta got to be done asap.:bow: [ ](https://chat.hyperledger.org/channel/cello?msg=dWBi7qjXYjJ6tn7ym) @bmkor
bmkor (Fri, 09 Jun 2017 00:55:42 GMT):
@baohua sorry that I'll go back to this later as testing fabric v1.0 beta got to be done asap. :bow: [ ](https://chat.hyperledger.org/channel/cello?msg=dWBi7qjXYjJ6tn7ym)
baohua (Fri, 09 Jun 2017 01:15:49 GMT):
thanks!
bmkor (Fri, 09 Jun 2017 02:03:55 GMT):
Wondering if we can connect IoT stuff to the fabric?
baohua (Fri, 09 Jun 2017 04:07:57 GMT):
i like IOT, but not sure ur idea, could u help provide more details?
chawlanikhil24 (Fri, 09 Jun 2017 06:19:18 GMT):
IOT in blockchain, I am curious to know how it can be achieved? @bmkor
bmkor (Fri, 09 Jun 2017 06:20:50 GMT):
I am just wondering. Say I got many weather temperature sensors, distributed all over a region. Can each sensor serve as a peer?
bmkor (Fri, 09 Jun 2017 06:23:24 GMT):
Precisely, I got those sensors distributed all over the counties (or provinces). Each county would serve like an organisation in fabric and some sensors located in a particular country would be the peers under that organisation. No sure this analogy is appropriate.
bmkor (Fri, 09 Jun 2017 06:23:55 GMT):
just a wild thought
bmkor (Fri, 09 Jun 2017 06:41:19 GMT):
Any ideas?
chawlanikhil24 (Fri, 09 Jun 2017 07:03:01 GMT):
That sounds only like peer to peer connectivity, and what about the transactions in Blockchains , whose ledger has to be maintained by Blockchains, what they can be related to ?
baohua (Fri, 09 Jun 2017 07:09:24 GMT):
one way may be to serve the IoT device renting transactions atop blockchain... self-auto-renting
chawlanikhil24 (Fri, 09 Jun 2017 07:15:37 GMT):
@baohua can you explain a bit, new to these words
chawlanikhil24 (Fri, 09 Jun 2017 07:40:39 GMT):
In this compose file, there are few environment variables
chawlanikhil24 (Fri, 09 Jun 2017 07:40:41 GMT):
https://github.com/hyperledger/cello/blob/master/src/agent/docker/_compose_files/fabric-0.6/local/cluster-4.yml
chawlanikhil24 (Fri, 09 Jun 2017 07:41:08 GMT):
how can i get the values of these environement variables
chawlanikhil24 (Fri, 09 Jun 2017 09:18:37 GMT):
got the query solved
baohua (Fri, 09 Jun 2017 14:02:21 GMT):
@chawlanikhil24 u mean the values of the environment in the docker-compose files? it was set in the environments section, and can read the value of env variables at the host platform.
PhDev (Fri, 09 Jun 2017 14:28:49 GMT):
Has joined the channel.
baohua (Mon, 12 Jun 2017 03:36:26 GMT):
Dear all, we need some one help add the license header. Pls let us know if anyone volunteer to do that, thanks! See https://jira.hyperledger.org/browse/CE-52
dstarina (Mon, 12 Jun 2017 07:07:17 GMT):
@baohua @chawlanikhil24 which Ubuntu version, docker engine version and docker-compose version do you use in your environments?
dstarina (Mon, 12 Jun 2017 07:12:37 GMT):
because docker engine 1.13 ships together with docker-compose 1.13, so I would love to know what you have installed
dstarina (Mon, 12 Jun 2017 07:12:48 GMT):
do you also use an older version of docker engine?
dstarina (Mon, 12 Jun 2017 12:19:41 GMT):
@baohua @chawlanikhil24 About the issue we were discussing (https://jira.hyperledger.org/projects/CE/issues/CE-54?filter=allopenissues). I have created a new environment from scratch. I have installed docker 1.12 and docker-compose 1.8.1. Even with this, I have exactly the same problem. I am attaching the logs, but I think the problem in the logs is the same as before.
dstarina (Mon, 12 Jun 2017 12:19:56 GMT):
Message Attachments
dstarina (Mon, 12 Jun 2017 13:22:08 GMT):
This is from the logs, do you know why this error would occur? "/etc/nginx/html/50x.html" failed (2: No such file or directory)
dstarina (Mon, 12 Jun 2017 13:22:20 GMT):
maybe this is the reason for 404?
dstarina (Mon, 12 Jun 2017 13:32:16 GMT):
I didn't set any host yet, I didn't set swarm cluster or anything ... I only have this one machine running for now, is this a problem?
dstarina (Mon, 12 Jun 2017 14:21:40 GMT):
So I only have the master right now
baohua (Mon, 12 Jun 2017 14:27:10 GMT):
aha, pls check ur local containers at the master node, if there's exited or restarting ones, then must sth wrong with the service in the container.
baohua (Mon, 12 Jun 2017 14:48:20 GMT):
@dstarina i just find a clean ubuntu 16.04 os, and run `make setup` and then `make restart`.... seems everything is fine. so strange why your built out container has wrong issues. Can u help clean all the docker images, and run the cmds i posted?
chawlanikhil24 (Mon, 12 Jun 2017 15:04:06 GMT):
Try docker container prune
chawlanikhil24 (Mon, 12 Jun 2017 15:04:11 GMT):
docker image prune
chawlanikhil24 (Mon, 12 Jun 2017 15:04:51 GMT):
then retry! Somethings definitely wrong with the environment on your machine
chawlanikhil24 (Mon, 12 Jun 2017 15:05:07 GMT):
@dstarina
baohua (Mon, 12 Jun 2017 15:17:37 GMT):
https://gerrit.hyperledger.org/r/#/c/10473/
chawlanikhil24 (Mon, 12 Jun 2017 15:40:27 GMT):
Cool !
dstarina (Mon, 12 Jun 2017 18:56:16 GMT):
Then it seems there is something wrong with my installation process ... Because I made a fresh installation of Ubuntu from the installation CD and then followed the procedure once again ... could someone take a peek at my installation process? It is the exact process that I followed from completely fresh Ubuntu 16.04 installation. It is supposed to be following the procedure exactly, but maybe I misunderstood some step ...
dstarina (Mon, 12 Jun 2017 18:56:29 GMT):
Message Attachments
baohua (Tue, 13 Jun 2017 01:46:50 GMT):
@dstarina dear dstarina, very sorry for the inconvenience, i check ur installation steps, and not find wrong issues inside. Can i know ur results at `make logs` after the boots up? Or can i remotely login in to ur vm to have checking? Thanks!
dstarina (Tue, 13 Jun 2017 08:58:41 GMT):
@baohua If you would be so kind to remotely login and check, I will be really grateful ... But the machine is on our internal network. I will send you access credentials for VPN access to your IBM email, is that ok?
chawlanikhil24 (Tue, 13 Jun 2017 10:35:47 GMT):
@dstarina I'm on the task to deploy a VM on AWS and copy paste your steps!
dstarina (Tue, 13 Jun 2017 10:39:44 GMT):
Thank you @chawlanikhil24 ! I really appreciate your help
baohua (Tue, 13 Jun 2017 10:40:53 GMT):
@dstarina i will have a look, beside, can i know what you will use cello for? It would be nice and helpful if we know what adoption scenarios the users are from :)
dstarina (Tue, 13 Jun 2017 10:44:37 GMT):
While Fabric on Bluemix is great, there are some companies that, even when considering using the cloud, don't want their data out of the country
dstarina (Tue, 13 Jun 2017 10:45:06 GMT):
So we're making a deal with external cloud provider with data centers inside the country
dstarina (Tue, 13 Jun 2017 10:46:22 GMT):
And we would like to use Cello so it helps us to quickly create blockchain networks to use blockchain as a service
dstarina (Tue, 13 Jun 2017 10:47:12 GMT):
In the future, there will probably be an option to create peers for one network on seperate machines?
dstarina (Tue, 13 Jun 2017 10:47:25 GMT):
And to maybe add another peer to an existing network?
baohua (Tue, 13 Jun 2017 10:55:20 GMT):
understand, dstarina. would u like to share the adoption info at https://wiki.hyperledger.org/projects/cello#adoptions? It would highly encourage the development team.
baohua (Tue, 13 Jun 2017 10:55:49 GMT):
And to ur question, the answer if yes for the fabric 1.0 blockchain type :) we are rushing to finish the support of it currently
dstarina (Tue, 13 Jun 2017 10:58:51 GMT):
I will gladly share the adoption info once we make the deal :) Can't right now because we are still early in the process
dstarina (Tue, 13 Jun 2017 11:00:36 GMT):
We appreciate the work you are doing, this is a great project and am glad to have found it
baohua (Tue, 13 Jun 2017 11:02:10 GMT):
aha, i login in, and find the reason...
baohua (Tue, 13 Jun 2017 11:02:20 GMT):
u are not using the latest code
dstarina (Tue, 13 Jun 2017 11:03:20 GMT):
I did this: git clone https://github.com/yeasy/cello
dstarina (Tue, 13 Jun 2017 11:03:29 GMT):
where can I find the latest code?
baohua (Tue, 13 Jun 2017 11:03:46 GMT):
the latest repo is at https://github.com/hyperledger/cello
baohua (Tue, 13 Jun 2017 11:03:52 GMT):
i move ur own version as cello-old
baohua (Tue, 13 Jun 2017 11:03:59 GMT):
and create a new cello dir with the latest code
baohua (Tue, 13 Jun 2017 11:04:03 GMT):
can u try with that?
dstarina (Tue, 13 Jun 2017 11:04:22 GMT):
ohhh, ok! thank you so much, I will give it a try and let you know!
dstarina (Tue, 13 Jun 2017 11:12:18 GMT):
yes, it works now, the dashboard has opened! thank you!
chawlanikhil24 (Tue, 13 Jun 2017 11:19:36 GMT):
Finally problem rectified!
chawlanikhil24 (Tue, 13 Jun 2017 11:20:09 GMT):
@baohua github/yeasy is that your repo ?
baohua (Tue, 13 Jun 2017 11:21:03 GMT):
great! before cello was accepted by hyperledger, i use that to develop the code. now it is frozen, as the latest code is at https://github.com/hyperledger/cello
dstarina (Tue, 13 Jun 2017 11:27:56 GMT):
One question: can I also use the master machine as a worker or is that a bad idea? :)
dstarina (Tue, 13 Jun 2017 11:37:26 GMT):
http://gerrit.hyperledger.org/r/cello and https://github.com/hyperledger/cello - are those two different repositories?
baohua (Tue, 13 Jun 2017 11:49:14 GMT):
no they are the same, the github one is a mirror of the gerrit one
baohua (Tue, 13 Jun 2017 11:49:46 GMT):
@dstarina i would suggest use another machine, to keep everything clean. unless u can handle potential issues :wink:
dstarina (Tue, 13 Jun 2017 12:07:25 GMT):
I'm just going to use another machine ;) I have issues with that, too. I get "failed to create host". Could it be a problem that I'm using docker 1.13 on the master machine and docker 1.12 on the worker?
dstarina (Tue, 13 Jun 2017 12:07:25 GMT):
I'm just going to use another machine ;)
dstarina (Tue, 13 Jun 2017 12:24:41 GMT):
Can I ask you a more "philosophical" question? Because I know I'm going to get asked this by a customer, and I don't even understand it myself. I understand the reasoning behind blockchain when each network participant (each company) has its own peer running. But on all the tutorial, and now even in Cello, we always install the network with multiple peers on one machine and I started doubting my understanding of this. Is this a viable production scenario, or is the reason behind it just creating a demo environment? Is there a scenario where having multiple peers on a single machine would make sense in production environment?
dstarina (Tue, 13 Jun 2017 12:25:15 GMT):
Is my understanding (having one peer per company) correct?
dstarina (Tue, 13 Jun 2017 12:33:40 GMT):
I'm asking because I only have an option of choosing either 4 or 6 peers when creating the chain :)
baohua (Wed, 14 Jun 2017 01:49:24 GMT):
david, if you let cello run atop a swarm cluster, then it will support multi-hosts, as the native Docker host only support single node :)
baohua (Wed, 14 Jun 2017 02:21:37 GMT):
Dear all, we plan to make a Cello meetup at the Hackfest in Beijing, pls feel free to join and share your thoughts, experience, and ideas. Thanks! https://wiki.hyperledger.org/groups/twgc/hackfest-beijing
chawlanikhil24 (Wed, 14 Jun 2017 06:59:59 GMT):
[ ](https://chat.hyperledger.org/channel/cello?msg=6J5E6Ti8D7t3MWhox) @baohua you mean, like each worker node in swarm running cello itself ?
baohua (Wed, 14 Jun 2017 07:01:36 GMT):
Cello treat each worker node as a pool of resource, and will deploy blockchain inside. While for the swarm mode, we treat a swam cluster as a "logic worker node". There cello will leverage swarm's overlay network to deploy among various hosts managed by the swarm. is this clear to u?
chawlanikhil24 (Wed, 14 Jun 2017 07:01:41 GMT):
[ ](https://chat.hyperledger.org/channel/cello?msg=fJNcKZqNBNhrMyaZ7) @baohua Will it be covered live as well, Since I am in India and would like to be a part
baohua (Wed, 14 Jun 2017 07:02:47 GMT):
aha, it's a face-face event for the hackfest. but at the same time, welcome to having thoughts/comments post inside this channel, too.
chawlanikhil24 (Wed, 14 Jun 2017 07:05:11 GMT):
[ ](https://chat.hyperledger.org/channel/cello?msg=mdk9xDyHn75P46rQS) @baohua Yupp absolutely clear, its like, multiple blockchain like, b1,b2,b3....bn are all running on cello and managed by Swarm overlay network ?
chawlanikhil24 (Wed, 14 Jun 2017 07:06:25 GMT):
In this case, I suppose if there are n blockchains, then n ip addresses too, to manage each worker blockchain cello dashboard ?
baohua (Wed, 14 Jun 2017 07:07:13 GMT):
sorry, not quite get ur question...
baohua (Wed, 14 Jun 2017 07:07:37 GMT):
in this case, 1 worker node === 1 swarm cluster (multi-hosts)
baohua (Wed, 14 Jun 2017 07:08:00 GMT):
the ip address is managed in the swarm cluster (overlay networking layer)
dstarina (Wed, 14 Jun 2017 07:08:17 GMT):
Guys, what is the reason the only options to choose when deploying a chain are 4 or 6 peers on HL 0.6 and 7 peers on HL 1.0? Is this because Cello is still in development - or am I wrong in assuming that in business network, each participant in a network (for example each company in a business network) should be responsible for one peer? Are peers supposed to be connected with network participants or are they only a way to achieve immutability and not really connected to business networks?
baohua (Wed, 14 Jun 2017 07:09:57 GMT):
david, as it's still under development, we provide some fixed number, to give some quick usage of a blockchain network. the dynamic joining feature will be supported soon.
baohua (Wed, 14 Jun 2017 07:10:14 GMT):
after we got the fabric 1.0 supporting feature done @chawlanikhil24
baohua (Wed, 14 Jun 2017 07:10:17 GMT):
@dstarina
dstarina (Wed, 14 Jun 2017 07:10:55 GMT):
ok, thank you for the answer, glad to hear
chawlanikhil24 (Wed, 14 Jun 2017 07:11:31 GMT):
cello will deprecate swarm for fabric 1.0 ?
chawlanikhil24 (Wed, 14 Jun 2017 07:11:51 GMT):
or wll it be a choice for the user to run on swarm or k8s ?
baohua (Wed, 14 Jun 2017 07:18:19 GMT):
both will be available
baohua (Wed, 14 Jun 2017 07:18:47 GMT):
that'd be better for the users to choice their own underly cluster type :)
zgalvin111 (Wed, 14 Jun 2017 17:30:33 GMT):
Has joined the channel.
j3ffyang (Thu, 15 Jun 2017 03:27:19 GMT):
Has joined the channel.
lujiang (Thu, 15 Jun 2017 08:38:16 GMT):
pls review http://gerrit.hyperledger.org/r/10625
baohua (Thu, 15 Jun 2017 08:40:21 GMT):
:thumbsup:
laxpio (Thu, 15 Jun 2017 12:39:06 GMT):
Has joined the channel.
baohua (Fri, 16 Jun 2017 01:47:29 GMT):
@lujiang got several comments there :)
lujiang (Fri, 16 Jun 2017 10:06:11 GMT):
Pls review http://gerrit.hyperledger.org/r/10701
lujiang (Fri, 16 Jun 2017 10:06:19 GMT):
http://gerrit.hyperledger.org/r/10703
lujiang (Fri, 16 Jun 2017 10:06:40 GMT):
as we discussed, the copyright will remain as is
chawlanikhil24 (Fri, 16 Jun 2017 10:19:32 GMT):
@baohua Hi, I've prepared some k8s manifests for deploying fabric over K8s and would like to get them reviewed !
baohua (Fri, 16 Jun 2017 13:56:55 GMT):
thanks
chawlanikhil24 (Fri, 16 Jun 2017 18:11:24 GMT):
Message Attachments
lujiang (Sat, 17 Jun 2017 01:56:19 GMT):
pls review http://gerrit.hyperledger.org/r/10737
baohua (Sat, 17 Jun 2017 15:07:01 GMT):
sure, thanks, and feel free to directly add/invite those contributors as reviewers in the gerrit page.
baohua (Mon, 19 Jun 2017 07:47:07 GMT):
here's the latest cello slides on beijing hackfest: https://github.com/yeasy/seminar-talk/blob/master/hyperledger-hackfest-20170619/Hyperledger%20Cello%20Overview-20170619.pdf
hightall (Mon, 19 Jun 2017 15:30:20 GMT):
https://gerrit.hyperledger.org/r/#/c/10779/
fengfengs (Tue, 20 Jun 2017 01:23:28 GMT):
Has joined the channel.
baohua (Tue, 20 Jun 2017 07:15:42 GMT):
https://gerrit.hyperledger.org/r/10819
LordGoodman (Wed, 21 Jun 2017 05:57:08 GMT):
Has joined the channel.
baohua (Wed, 21 Jun 2017 08:24:54 GMT):
https://gerrit.hyperledger.org/r/10899
baohua (Wed, 21 Jun 2017 09:13:03 GMT):
Anyone would like to help update the fabric 1.0 compose files? Feel free to pick the task at https://jira.hyperledger.org/browse/CE-63. Thanks!
lyriarte (Wed, 21 Jun 2017 12:13:48 GMT):
Has joined the channel.
baohua (Wed, 21 Jun 2017 12:20:02 GMT):
https://gerrit.hyperledger.org/r/#/c/10901/
rcyrus (Wed, 21 Jun 2017 14:11:09 GMT):
Has joined the channel.
lujiang (Thu, 22 Jun 2017 09:27:49 GMT):
https://gerrit.hyperledger.org/r/#/c/10997/
jmcnevin (Thu, 22 Jun 2017 18:23:50 GMT):
Has joined the channel.
lujiang (Fri, 23 Jun 2017 09:02:28 GMT):
@tkuhrt I have a question about license header for some special lib.
tkuhrt (Fri, 23 Jun 2017 09:02:29 GMT):
Has joined the channel.
lujiang (Fri, 23 Jun 2017 09:05:58 GMT):
User-dashboard/public/js/lib/CSS3DRenderer.js is from example of Three.js source code , but it does not include the same license header as Three.js
lujiang (Fri, 23 Jun 2017 09:06:23 GMT):
Can we directly use it in our code? As it is?
tkuhrt (Fri, 23 Jun 2017 20:48:56 GMT):
@lujiang : Without a license, we cannot tell what the author intended the license to be. We either need to get the author to provide the license to determine whether it is compatible with the Apache 2.0 license or whether we will require an exception for inclusion of this file in the Cello source base. If we are unable to determine the license, is there an alternative that can be used?
baohua (Sat, 24 Jun 2017 00:58:45 GMT):
Anyone wanna volunteer to help enable the fabric 1.0.0-rc1 images, just feel free to pick https://jira.hyperledger.org/browse/CE-67
ThePleasurable (Sat, 24 Jun 2017 12:23:33 GMT):
Has joined the channel.
seaSwan (Sat, 24 Jun 2017 19:05:47 GMT):
Has joined the channel.
seaSwan (Sat, 24 Jun 2017 19:06:51 GMT):
hey guys, dont know if this is the correct chat but I just signed up for a free month of microsofts Azure blockchain hosting... wondering what all the options are related to hyperledger I can deploy?
chawlanikhil24 (Sun, 25 Jun 2017 05:26:35 GMT):
https://jira.hyperledger.org/projects/CE/issues/CE-7?filter=allopenissues
hightall (Sun, 25 Jun 2017 09:50:18 GMT):
https://gerrit.hyperledger.org/r/11133
chawlanikhil24 (Mon, 26 Jun 2017 06:11:37 GMT):
https://gerrit.hyperledger.org/r/#/c/11139/
zuowang (Mon, 26 Jun 2017 07:50:06 GMT):
Has joined the channel.
xixuejia (Tue, 27 Jun 2017 01:49:31 GMT):
Has joined the channel.
baohua (Tue, 27 Jun 2017 05:26:16 GMT):
https://gerrit.hyperledger.org/r/#/c/11173/
baohua (Tue, 27 Jun 2017 12:16:54 GMT):
https://gerrit.hyperledger.org/r/#/c/11179/
chawlanikhil24 (Wed, 28 Jun 2017 08:50:20 GMT):
2017/06/28 08:48:25 grpc: Server.Serve failed to complete security handshake from "172.17.0.6:49632": tls: first record does not look like a TLS handshake
chawlanikhil24 (Wed, 28 Jun 2017 08:50:43 GMT):
how do i fix it ?
chawlanikhil24 (Wed, 28 Jun 2017 11:47:17 GMT):
https://jira.hyperledger.org/projects/CE/issues/CE-7?filter=allopenissues
hightall (Wed, 28 Jun 2017 15:19:52 GMT):
https://gerrit.hyperledger.org/r/11223
baohua (Thu, 29 Jun 2017 03:37:24 GMT):
@chawlanikhil24 we may disable the tls first to make it connectable, then look at the reason of tls handshakes failure.
chawlanikhil24 (Thu, 29 Jun 2017 08:18:54 GMT):
OK! ... I'll begin with TLS disabling
5igm4 (Thu, 29 Jun 2017 17:16:47 GMT):
Has joined the channel.
wajid.poernomo (Sat, 01 Jul 2017 05:12:10 GMT):
Has joined the channel.
wajid.poernomo (Sat, 01 Jul 2017 05:13:35 GMT):
Looking at hardware spec here for Cello: https://github.com/yeasy/cello/blob/master/docs/deployment.md : What is the EC2 Instance equivalent of 8c16g100g
chawlanikhil24 (Sat, 01 Jul 2017 07:32:06 GMT):
for development, A micro instance was enough for me
chawlanikhil24 (Sat, 01 Jul 2017 07:32:26 GMT):
I don't have idea about the production
wajid.poernomo (Sat, 01 Jul 2017 08:43:19 GMT):
Thanks I'll give this a go
baohua (Mon, 03 Jul 2017 01:47:03 GMT):
@wajid.poernomo the config is a recommended one, it might work with smaller resource like 2c4g30g.
ccalabia (Mon, 03 Jul 2017 14:42:38 GMT):
Has joined the channel.
ccalabia (Tue, 04 Jul 2017 02:28:03 GMT):
Hi, I'm using Docker version 17.06.0-ce, build 02c1d87 and I'm stuck with the error below from Master Node
WARNING [agent.docker.host] [host.py:62 create()] - The worker_api is active:tcp://10.148.0.4:2375
ERROR [agent.docker.docker_swarm] [docker_swarm.py:168 detect_daemon_type()] - 'Cluster'
WARNING [agent.docker.host] [host.py:71 create()] - Detected type=None is not docker or swarm
WARNING [modules.host] [host.py:93 create()] - aa cannot be setup
WARNING [resources.host_api] [host_api.py:92 host_create()] - Failed to create host a
baohua (Tue, 04 Jul 2017 03:21:37 GMT):
@ccalabia hi, the current supported docker version is 1.10--1.13. We may enable the support of manual specifying the cluster type in future, besides auto-detecting it... @hightall
ccalabia (Tue, 04 Jul 2017 03:38:19 GMT):
@baohua Thanks!!!!
hongquan.zhang (Tue, 04 Jul 2017 03:49:14 GMT):
Has joined the channel.
sccdhanshan (Tue, 04 Jul 2017 05:36:20 GMT):
Has joined the channel.
jun (Tue, 04 Jul 2017 06:52:28 GMT):
Has joined the channel.
sccdhanshan (Tue, 04 Jul 2017 06:56:18 GMT):
Problem: use cello dashboard "add host" is right, but add chain does not respond for a long time
Installation environment:
cello lastest version
docker-engine 1.130 docker-compose 1.8.1
Hyperledger image version v1.0.0-rc1
Make logs show:
File "/app/modules/host.py", line 225, in create_cluster_work
Watchdog | consensus_mode = consensus_mode, size = cluster_size)
Watchdog | TypeError: create () got an unexpected keyword argument 'consensus_plugin'
DannyWong (Tue, 04 Jul 2017 08:37:41 GMT):
Has joined the channel.
baohua (Wed, 05 Jul 2017 09:09:12 GMT):
https://gerrit.hyperledger.org/r/#/c/11379/
DavidHorne (Wed, 05 Jul 2017 17:11:42 GMT):
Has joined the channel.
DavidHorne (Wed, 05 Jul 2017 17:12:27 GMT):
For some reason I can't delete any chains or hosts on the dashboard. here are the errors I am getting.
DavidHorne (Wed, 05 Jul 2017 17:12:32 GMT):
dashboard | [2017-07-06 01:05:57,250] ERROR [agent.docker.docker_swarm] [docker_swarm.py:411 compose_clean()] - Error in stop compose project, will clean
dashboard | [2017-07-06 01:05:57,250] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:412 compose_clean()] -
dashboard | Can't find a suitable configuration file in this directory or any
dashboard | parent. Are you in the right directory?
dashboard |
dashboard | Supported filenames: docker-compose.yml, docker-compose.yaml
traviscox (Wed, 05 Jul 2017 17:17:27 GMT):
Has joined the channel.
baohua (Thu, 06 Jul 2017 01:40:42 GMT):
@DavidHorne we fix several bugs recently. could u try the latest code? it should work with fabric 1.0 now.
pandabcai (Thu, 06 Jul 2017 01:41:43 GMT):
Has joined the channel.
sccdhanshan (Thu, 06 Jul 2017 06:59:55 GMT):
with fabric 1.0 ,I can add hosts on cello Dashboard, but can't add chain( does not respond for a long time)
Manually refresh the page, can see the creation of the chain, but in fact, the creation of the chain did not succeed.
Installation environment: docker-engine 1.12 docker-compose 1.8.1
Hyperledger image version v1.0.0-rc1
Make logs trace, seems normal
sccdhanshan (Thu, 06 Jul 2017 07:07:34 GMT):
with fabric 1.0 ,I can add hosts on cello Dashboard, but can't add chain on host( does not respond for a long time)
Manually refresh Dashboard, can see the creation of the chain, but in fact, the creation of the chain did not succeed on host,and can't delete that chains(only in Mongo db,Failed to create on host).
Installation environment: docker-engine 1.12 docker-compose 1.8.1
Hyperledger image version v1.0.0-rc1
Make logs trace, seems normal
sccdhanshan (Thu, 06 Jul 2017 07:13:07 GMT):
with fabric 1.0 ,I can add hosts on cello Dashboard, but can't add chain on host( does not respond for a long time)
Manually refresh Dashboard, can see the creation of the chain, but in fact, Failed to create on host,only insert record to Mongo db(the creation of the chain did not succeed on host),and can't delete that chains.
Installation environment: docker-engine 1.12 docker-compose 1.8.1
Hyperledger image version v1.0.0-rc1
Make logs trace, seems normal
sccdhanshan (Thu, 06 Jul 2017 07:14:35 GMT):
with fabric 1.0 ,I can add hosts on cello Dashboard, but can't add chain on host( does not respond for a long time)
Manually refresh Dashboard, can see the creation of the chain, but in fact, Failed to create on host,only insert record to Mongo db(the creation of the chain did not succeed on host),and can't delete that chains.
Installation environment: docker-engine 1.12 docker-compose 1.8.1
Hyperledger image version v1.0.0-rc1
sccdhanshan (Thu, 06 Jul 2017 07:15:44 GMT):
with fabric 1.0 ,I can add hosts on cello Dashboard, but can't add chain on host( does not respond for a long time),Manually refresh Dashboard, can see the creation of the chain, but in fact, Failed to create on host,only insert record to Mongo db(the creation of the chain did not succeed on host),and can't delete that chains.
Make logs trace, seems normal
Installation environment: docker-engine 1.12 docker-compose 1.8.1
Hyperledger image version v1.0.0-rc1
baohua (Thu, 06 Jul 2017 08:29:34 GMT):
strange, have u setup the worker node as in the doc? we need the docker images and those artifacts folds setup manually in the worker node.
zhasni (Thu, 06 Jul 2017 12:42:05 GMT):
Has joined the channel.
bluecrayon52 (Thu, 06 Jul 2017 12:52:14 GMT):
Has joined the channel.
ashahn (Thu, 06 Jul 2017 13:38:48 GMT):
Has joined the channel.
ankursam (Thu, 06 Jul 2017 13:38:48 GMT):
Has joined the channel.
hightall (Thu, 06 Jul 2017 15:41:23 GMT):
https://gerrit.hyperledger.org/r/11411
ShahzebHasnain (Thu, 06 Jul 2017 16:21:58 GMT):
Has joined the channel.
ShahzebHasnain (Thu, 06 Jul 2017 16:29:53 GMT):
@sccdhanshan facing the exact same case here
jmcnevin (Thu, 06 Jul 2017 20:01:06 GMT):
Has left the channel.
baohua (Fri, 07 Jul 2017 02:52:09 GMT):
https://gerrit.hyperledger.org/r/#/c/11427/
chawlanikhil24 (Fri, 07 Jul 2017 06:48:28 GMT):
https://gerrit.hyperledger.org/r/#/c/11433/
baohua (Fri, 07 Jul 2017 07:53:38 GMT):
https://gerrit.hyperledger.org/r/#/c/11435/
Gaurav_Impro (Fri, 07 Jul 2017 11:27:33 GMT):
Has joined the channel.
chawlanikhil24 (Sat, 08 Jul 2017 06:10:04 GMT):
@baohua ! I can delete the old docker host , neither can reset or clean
chawlanikhil24 (Sat, 08 Jul 2017 06:10:11 GMT):
on cell dashboard
chawlanikhil24 (Sat, 08 Jul 2017 06:10:11 GMT):
on cello dashboard
Gaurav_Impro (Sun, 09 Jul 2017 15:40:49 GMT):
@baohua I am getting error while starting the restserver , dashboard and watchdog...error message is "ERROR: for restserver Cannot start service restserver: oci runtime error: container_linux.go:262: starting container process caused "exec: \"/usr/bin/python3.5\": stat /usr/bin/python3.5: no such file or directory" ... Can you please let me know how to resolve this error?
chawlanikhil24 (Mon, 10 Jul 2017 04:30:42 GMT):
https://jira.hyperledger.org/projects/CE/issues/CE-75?filter=allopenissues
baohua (Mon, 10 Jul 2017 05:15:01 GMT):
@Gaurav_Impro feel free to open a jira bug, and pls describe ur env and steps to repeat the problem. And also suggest u strictly follow the installation doc first.
baohua (Mon, 10 Jul 2017 09:13:24 GMT):
https://gerrit.hyperledger.org/r/11465
zhasni (Mon, 10 Jul 2017 09:32:56 GMT):
Hi I just ran cello following docs `Instalation.md` an `tutorial.md` with 1.0-rc1
and I don't get the utility of cello...
there is a bunch of config files **src/agent/docker/_compose_files/fabric-1.0/local/** never used anywhere ?!
It's just an administration panel for deployed blockchains (1 BC on 1 Host) ???
If I'm wrong can you enlighten me ?
thx !
baohua (Tue, 11 Jul 2017 03:46:33 GMT):
need review: https://gerrit.hyperledger.org/r/#/c/11435/
baohua (Tue, 11 Jul 2017 03:48:19 GMT):
@zhasni if u setup successfully, the dashboard will come out, you can feel free to start chain through the web dashboard. currently the user dashboard is not by default connected. you can use the admin dashboard.
zhasni (Tue, 11 Jul 2017 07:55:52 GMT):
Message Attachments
zhasni (Tue, 11 Jul 2017 07:55:52 GMT):
Message Attachments
zhasni (Tue, 11 Jul 2017 07:58:18 GMT):
I manage to add a host and a chain cf picture.
nothing happened on my host or others (swarm)
no docker container, no peers, no ca, no orderders...
baohua (Tue, 11 Jul 2017 08:08:37 GMT):
u need to follow the install doc to setup a worker node first.
baohua (Tue, 11 Jul 2017 08:08:39 GMT):
https://gerrit.hyperledger.org/r/#/c/11519/
bmkor (Tue, 11 Jul 2017 13:00:42 GMT):
Wondering if the analytics/modelling/learning can be done this way:
bmkor (Tue, 11 Jul 2017 13:02:51 GMT):
Wondering if the analytics/modelling/learning can be done through some tools like connecting `caffe` to the ledger of some nodes via chaincodes?
bmkor (Tue, 11 Jul 2017 13:02:51 GMT):
Wondering if the analytics/modelling/learning can be done through connecting some tools like `caffe` to the ledger of some nodes via chaincodes?
baohua (Tue, 11 Jul 2017 14:24:24 GMT):
that was discussed, and we welcome for the feature, but no quite recent plan to do it. any idea?
bmkor (Wed, 12 Jul 2017 02:30:19 GMT):
Apart from the historical world state can be retri
bmkor (Wed, 12 Jul 2017 02:31:28 GMT):
Apart from the historical world state can be retrieved (as far as I understand), do we have the timestamp for each put state in the ledger?
bmkor (Wed, 12 Jul 2017 02:31:57 GMT):
I am thinking how clusterings can be carried out.
baohua (Wed, 12 Jul 2017 14:56:49 GMT):
@indirajith Feel free to pick this one: https://jira.hyperledger.org/browse/CE-83
baohua (Wed, 12 Jul 2017 14:59:27 GMT):
@bmkor there's ts for each tx inside the block, but depends on how u wanna achieve.
bmkor (Thu, 13 Jul 2017 00:23:44 GMT):
Thx. I suppose chaincode could grep this ts, right? [ ](https://chat.hyperledger.org/channel/cello?msg=SqQh62nZu6vaTjzry) @baohua
baohua (Thu, 13 Jul 2017 02:06:39 GMT):
yeap, can use the `GetTxTimestamp` to inquery it. which means the creation time of the tx.
baohua (Thu, 13 Jul 2017 03:34:33 GMT):
http://cello.readthedocs.io/en/latest/installation/
liutc (Thu, 13 Jul 2017 07:04:32 GMT):
has cello officially support fabric 1.0.0?
baohua (Thu, 13 Jul 2017 10:35:22 GMT):
@liutc the feature was just finished, and welcome to help evaluate.
baohua (Thu, 13 Jul 2017 10:35:39 GMT):
@indirajith feel free to ref this to pull new images: https://github.com/hyperledger/fabric-sdk-py/blob/master/check.sh
indirajith (Thu, 13 Jul 2017 10:59:36 GMT):
@baohua so, instead of pulling the images from yeasy/ pull directly from hyperledger/ ?
baohua (Thu, 13 Jul 2017 11:02:14 GMT):
yeap, since now we have the official release, we can use that directly, how do u think?
indirajith (Thu, 13 Jul 2017 11:02:54 GMT):
Yeah, sure! I will change them now
baohua (Thu, 13 Jul 2017 11:11:55 GMT):
thanks, and pls help verify locally before submitting~
indirajith (Thu, 13 Jul 2017 12:34:34 GMT):
review needed: https://gerrit.hyperledger.org/r/#/c/11607/
indirajith (Thu, 13 Jul 2017 14:02:33 GMT):
Has anyone encounter problems in Cello dashboard and restserver dockers keep on restarting due to import errors?
indirajith (Thu, 13 Jul 2017 14:03:47 GMT):
especially from /app/resources/login.py and /app/resources/__init__.py
baohua (Thu, 13 Jul 2017 14:23:47 GMT):
can u show the log? `make logs`
indirajith (Thu, 13 Jul 2017 14:38:07 GMT):
Message Attachments
jmcnevin (Thu, 13 Jul 2017 19:29:29 GMT):
Has joined the channel.
jmcnevin (Thu, 13 Jul 2017 19:30:34 GMT):
@indirajith I'm seeing this as well
baohua (Fri, 14 Jul 2017 01:55:46 GMT):
@indirajith @jmcnevin pls remove all containers and images first, then `make restart`
baohua (Fri, 14 Jul 2017 03:50:27 GMT):
https://gerrit.hyperledger.org/r/#/c/11643/
liutc (Fri, 14 Jul 2017 07:14:54 GMT):
@indirajith I am experiencing similar -- only difference is my dashboard container keeps stable. My logs are blow
liutc (Fri, 14 Jul 2017 07:15:06 GMT):
ImportError: No module named 'flask_login'
Traceback (most recent call last):
File "restserver.py", line 10, in
indirajith (Fri, 14 Jul 2017 08:45:55 GMT):
@liutc and @jmcnevin it works now. Please remove all the containers and images then retry.
zws (Fri, 14 Jul 2017 09:02:05 GMT):
Has joined the channel.
baohua (Fri, 14 Jul 2017 11:08:14 GMT):
:wink:
jmcnevin (Fri, 14 Jul 2017 14:06:05 GMT):
Message Attachments
indirajith (Fri, 14 Jul 2017 15:14:18 GMT):
Yes, me too. the problem with containers have been resolved but has template issues and it does not load the basic theme.
chawlanikhil24 (Sat, 15 Jul 2017 05:26:05 GMT):
@indirajith @jmcnevin see container logs
chawlanikhil24 (Sat, 15 Jul 2017 05:26:10 GMT):
what do they say ?
liuwenliang0632 (Sat, 15 Jul 2017 09:12:40 GMT):
Has joined the channel.
baohua (Sun, 16 Jul 2017 06:35:11 GMT):
the problem is due to the missing of login template, i disable the login checking module for a temp fix. will enable till the module is done. https://gerrit.hyperledger.org/r/#/c/11669/
baohua (Sun, 16 Jul 2017 06:35:14 GMT):
@hightall
indirajith (Sun, 16 Jul 2017 20:40:04 GMT):
Message Attachments
indirajith (Sun, 16 Jul 2017 20:42:47 GMT):
Code review needed: https://gerrit.hyperledger.org/r/#/c/11675/
cre8bidio (Mon, 17 Jul 2017 00:44:06 GMT):
Has joined the channel.
liutc (Mon, 17 Jul 2017 03:20:28 GMT):
@indirajith @baohua just tried and things are working now. thanks~~~
baohua (Mon, 17 Jul 2017 04:46:26 GMT):
nice to know that!
indirajith (Mon, 17 Jul 2017 08:23:03 GMT):
@chawlanikhil24 logs show the same error as the dashboard.
colinGrahms (Mon, 17 Jul 2017 09:39:08 GMT):
Has joined the channel.
rohitrocket (Tue, 18 Jul 2017 07:22:34 GMT):
Has joined the channel.
rohitrocket (Tue, 18 Jul 2017 11:33:00 GMT):
Has left the channel.
baohua (Wed, 19 Jul 2017 06:09:00 GMT):
https://gerrit.hyperledger.org/r/#/c/11723/
baohua (Wed, 19 Jul 2017 06:09:09 GMT):
https://gerrit.hyperledger.org/r/#/c/11721/
chawlanikhil24 (Wed, 19 Jul 2017 08:21:32 GMT):
@baohua just created container from K8s pod using docker.sock
chawlanikhil24 (Wed, 19 Jul 2017 08:23:30 GMT):
Message Attachments
baohua (Wed, 19 Jul 2017 08:30:34 GMT):
nice to know that!
indirajith (Wed, 19 Jul 2017 11:58:49 GMT):
Does anyone encounter the same problem? Worker node's orderer and peer keep on restarting due to 'err stat /etc/hyperledger/fabric/msp/signcerts: no such file or directory' error?
baohua (Wed, 19 Jul 2017 13:36:04 GMT):
aha, thanks for finding the problem
baohua (Wed, 19 Jul 2017 13:37:38 GMT):
@indirajith
indirajith (Wed, 19 Jul 2017 14:37:17 GMT):
https://gerrit.hyperledger.org/r/#/c/11749/ code review!
baohua (Thu, 20 Jul 2017 02:16:57 GMT):
+2!
vu3mmg (Thu, 20 Jul 2017 03:00:28 GMT):
Has joined the channel.
vu3mmg (Thu, 20 Jul 2017 03:02:17 GMT):
@baohua do cello have a policy editor
baohua (Thu, 20 Jul 2017 03:31:53 GMT):
we do not have that feature yet. and is there any specific scenario requirement?
vu3mmg (Thu, 20 Jul 2017 04:06:15 GMT):
background I had completed some POCs with banks , now moving to live pilot .the major issue which i am facing is network management certificate management and policy enforcement
vu3mmg (Thu, 20 Jul 2017 04:06:16 GMT):
i dont know whether all eh above will be under the preview of cello
vu3mmg (Thu, 20 Jul 2017 04:06:44 GMT):
i thought of having a light weight tool for policy management ...
vu3mmg (Thu, 20 Jul 2017 04:07:01 GMT):
and certificate management
vu3mmg (Thu, 20 Jul 2017 04:07:16 GMT):
please let me know your thoughts
baohua (Thu, 20 Jul 2017 05:18:55 GMT):
got ur requirements, can i know which company? I think ur demand makes sense, as currently there's little support for the policy management. we haven't thought this too much before. but can certainly discuss whether this is a popular requirements for cello. if you would like to, can feel free to create a jira item at jira.hyperledger.org.
vu3mmg (Thu, 20 Jul 2017 06:00:36 GMT):
digiledge
vu3mmg (Thu, 20 Jul 2017 06:01:06 GMT):
I could allocate some resources .... ok will do
vu3mmg (Thu, 20 Jul 2017 06:13:16 GMT):
https://jira.hyperledger.org/browse/CE-91
duwenhui (Thu, 20 Jul 2017 06:47:15 GMT):
Has joined the channel.
duwenhui (Thu, 20 Jul 2017 07:03:30 GMT):
@baohua dashboard | File "/usr/local/lib/python3.5/site-packages/compose/config/config.py", line 159, in from_filename
dashboard | return cls(filename, load_yaml(filename))
dashboard | File "/usr/local/lib/python3.5/site-packages/compose/config/config.py", line 1178, in load_yaml
dashboard | raise ConfigurationError(u"{}: {}".format(error_name, e))
dashboard | compose.config.errors.ConfigurationError: .FileNotFoundError: [Errno 2] No such file or directory: './agent/docker/_compose_files/fabric-1.0/local/fabric-kafka-4.yaml'
dashboard | 172.19.0.4 - - [20/Jul/2017 14:51:16] "GET /DevMgmt/DiscoveryTree.xml HTTP/1.0" 404 -
duwenhui (Thu, 20 Jul 2017 07:05:33 GMT):
@baohua when I'm on pause, releasing chain operation,the error accoured.
baohua (Thu, 20 Jul 2017 08:30:34 GMT):
can u try the latest code from master branch? from the log, we do not support kafka yet. pls try the solo type
duwenhui (Thu, 20 Jul 2017 09:46:57 GMT):
@baohua thx, i get it.
duwenhui (Thu, 20 Jul 2017 09:56:21 GMT):
@baohua how to reinit cello master node?
indirajith (Thu, 20 Jul 2017 10:00:55 GMT):
@duwenhui make restart
indirajith (Thu, 20 Jul 2017 10:01:28 GMT):
`make restart`
duwenhui (Thu, 20 Jul 2017 10:03:00 GMT):
@indirajith It doesn't work and the error data inside the cello isn't clear
indirajith (Thu, 20 Jul 2017 10:05:09 GMT):
what error message do you get? You can also make use of `make logs` to have a look at logs
duwenhui (Thu, 20 Jul 2017 10:08:23 GMT):
@indirajith it means hosts data and chains data .
baohua (Thu, 20 Jul 2017 11:39:12 GMT):
@duwenhui you can run `make restart` or `make redeploy`. just see the Makefile, there's clean command :)
markparz (Thu, 20 Jul 2017 15:10:56 GMT):
Has joined the channel.
markparz (Thu, 20 Jul 2017 15:11:58 GMT):
Hi everyone, I went to https://github.com/hyperledger/cello/blob/master/docs/tutorial.md and clicked on the "installation" link and the link is broken (404).
markparz (Thu, 20 Jul 2017 15:37:24 GMT):
fixed a error in doc... broken link https://gerrit.hyperledger.org/r/#/c/11779/
ashahn (Thu, 20 Jul 2017 18:22:06 GMT):
Has left the channel.
g.vamsi (Thu, 20 Jul 2017 19:08:39 GMT):
Has joined the channel.
chawlanikhil24 (Thu, 20 Jul 2017 20:33:45 GMT):
Hi people
chawlanikhil24 (Thu, 20 Jul 2017 20:33:50 GMT):
```
2017-07-20 20:28:12.664 UTC [kvledger] NewProvider -> INFO 016 ledger provider Initialized
2017-07-20 20:28:12.664 UTC [kvledger] recoverUnderConstructionLedger -> DEBU 017 Recovering under construction ledger
2017-07-20 20:28:12.664 UTC [kvledger] recoverUnderConstructionLedger -> DEBU 018 No under construction ledger found. Quitting recovery
2017-07-20 20:28:12.664 UTC [ledgermgmt] initialize -> INFO 019 ledger mgmt initialized
2017-07-20 20:28:12.664 UTC [eventhub_producer] AddEventType -> DEBU 01a Registering BLOCK
2017-07-20 20:28:12.665 UTC [eventhub_producer] AddEventType -> DEBU 01b Registering CHAINCODE
2017-07-20 20:28:12.665 UTC [eventhub_producer] AddEventType -> DEBU 01c Registering REJECTION
2017-07-20 20:28:12.665 UTC [eventhub_producer] AddEventType -> DEBU 01d Registering REGISTER
2017-07-20 20:28:12.665 UTC [eventhub_producer] start -> INFO 01e Event processor started
panic: listen tcp 10.0.0.98:7052: bind: cannot assign requested address
goroutine 1 [running]:
github.com/hyperledger/fabric/peer/node.createChaincodeServer(0x144d880, 0xc4201b87e0, 0xc4201d9930, 0xc, 0x0, 0x0, 0x0)
/go/src/github.com/hyperledger/fabric/peer/node/start.go:294 +0x2af
github.com/hyperledger/fabric/peer/node.serve(0x1499f90, 0x0, 0x0, 0x0, 0x0)
/go/src/github.com/hyperledger/fabric/peer/node/start.go:136 +0x56f
github.com/hyperledger/fabric/peer/node.glob..func1(0x1430ac0, 0x1499f90, 0x0, 0x0, 0x0, 0x0)
/go/src/github.com/hyperledger/fabric/peer/node/start.go:75 +0x3f
github.com/hyperledger/fabric/vendor/github.com/spf13/cobra.(*Command).execute(0x1430ac0, 0x1499f90, 0x0, 0x0, 0x1430ac0, 0x1499f90)
/go/src/github.com/hyperledger/fabric/vendor/github.com/spf13/cobra/command.go:599 +0x3e9
github.com/hyperledger/fabric/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0x1431120, 0xf, 0xc42000c035, 0x7)
/go/src/github.com/hyperledger/fabric/vendor/github.com/spf13/cobra/command.go:689 +0x339
github.com/hyperledger/fabric/vendor/github.com/spf13/cobra.(*Command).Execute(0x1431120, 0x1b, 0xc42000c035)
/go/src/github.com/hyperledger/fabric/vendor/github.com/spf13/cobra/command.go:648 +0x2b
main.main()
/go/src/github.com/hyperledger/fabric/peer/main.go:118 +0x54d
Logs from 7/20/17 8:28 PM to 7/
```
chawlanikhil24 (Thu, 20 Jul 2017 20:34:03 GMT):
can anyone help me regarding this!
duwenhui (Fri, 21 Jul 2017 01:32:35 GMT):
@baohua after make restart and make redeploy , the host and chain data added brefore remaine exist.
baohua (Fri, 21 Jul 2017 01:33:39 GMT):
Thanks and congrats for ur 1st patchset! @markparz
baohua (Fri, 21 Jul 2017 01:34:23 GMT):
@duwenhui sorry cannot understand, u mean some new problem?
baohua (Fri, 21 Jul 2017 01:37:14 GMT):
@chawlanikhil24 which version of image are u using? Seems the address port is invalid? listen tcp 10.0.0.98:7052: bind: cannot assign requested address
baohua (Fri, 21 Jul 2017 01:37:43 GMT):
pls check whether some other process already occupy the address.
baohua (Fri, 21 Jul 2017 02:13:23 GMT):
@duwenhui here's the channel
duwenhui (Fri, 21 Jul 2017 02:15:00 GMT):
@baohua Ubuntu 16.04
Update /lib/systemd/system/docker.service like
[Service]
DOCKER_OPTS="$DOCKER_OPTS -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --api-cors-header='*' --default-ulimit=nofile=8192:16384 --default-ulimit=nproc=8192:16384"
EnvironmentFile=-/etc/default/docker
ExecStart=
ExecStart=/usr/bin/dockerd -H fd:// $DOCKER_OPTS
Regenerate the docker service script and restart the docker engine:
$ sudo systemctl daemon-reload
$ sudo systemctl restart docker.service
This operation does not seem to be successful?
baohua (Fri, 21 Jul 2017 02:17:54 GMT):
aha, i've tested in ubuntu 16.04, not sure if it was changed. anyone else meets similar issue?
baohua (Fri, 21 Jul 2017 02:18:15 GMT):
@chawlanikhil24 @hightall @indirajith
liutc (Fri, 21 Jul 2017 04:20:57 GMT):
@baohua I do not know what exact problem @duwenhui met. In my case, the DOCKER_OPTS does not work in ExecStart=/usr/... I have to replace that with the value, i.e. directly writing everything in the same line
chawlanikhil24 (Fri, 21 Jul 2017 04:24:13 GMT):
@baohua It was unsuccessfull on my ubuntu machine as far as I remember, that's why I added one alternative in a PR
duwenhui (Fri, 21 Jul 2017 05:08:43 GMT):
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target docker.socket firewalld.service
Wants=network-online.target
Requires=docker.socket
[Service]
#Type=notify
DOCKER_OPTS="$DOCKER_OPTS -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --api-cors-header='*' --default-ulimit=nofile=8192:16384 --default-ulimit=nproc=8192:16384"
EnvironmentFile=-/etc/default/docker
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
ExecStart=
ExecStart=/usr/bin/dockerd -H fd:// $DOCKER_OPTS
#ExecReload=/bin/kill -s HUP $MAINPID
#LimitNOFILE=1048576
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
#LimitNPROC=infinity
#LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
#TasksMax=infinity
#TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
#Delegate=yes
# kill only the docker process, not all processes in the cgroup
#KillMode=process
# restart the docker process if it exits prematurely
#Restart=on-failure
#StartLimitBurst=3
#StartLimitInterval=60s
[Install]
WantedBy=multi-user.target
duwenhui (Fri, 21 Jul 2017 05:09:32 GMT):
it 's my docker.service config ,but i can't found listened port 2375
duwenhui (Fri, 21 Jul 2017 05:10:42 GMT):
@baohua
duwenhui (Fri, 21 Jul 2017 05:11:19 GMT):
root@BlockChain:~/gocode/src/github.com/cello# ss -antl
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 50 *:139 *:*
LISTEN 0 5 192.168.122.1:53 *:*
LISTEN 0 128 *:22 *:*
LISTEN 0 50 *:445 *:*
LISTEN 0 128 :::2377 :::*
LISTEN 0 128 :::7946 :::*
LISTEN 0 50 :::139 :::*
LISTEN 0 128 :::22 :::*
LISTEN 0 50 :::445 :::*
indirajith (Fri, 21 Jul 2017 09:57:52 GMT):
Code review: https://gerrit.hyperledger.org/r/#/c/11809/
indirajith (Fri, 21 Jul 2017 10:16:46 GMT):
@duwenhui @baohua Yes I remember, I could not make `$ docker -H Worker_Node_IP:2375 info` work with the method provided for Ubuntu 16.04. Then I went with the all other linux distro option twice and it worked for me.
baohua (Mon, 24 Jul 2017 02:00:35 GMT):
in recent ubuntu after 16.04, the os uses systemd to manage the bootup service, which is more complicated than the upstart. Hope we collect more usage cases to find a compatible way to add those docker options.
chawlanikhil24 (Mon, 24 Jul 2017 05:56:02 GMT):
https://gerrit.hyperledger.org/r/#/c/11861/
chawlanikhil24 (Mon, 24 Jul 2017 05:56:14 GMT):
peer joining issue fixed!. :)
Luke_Chen (Mon, 24 Jul 2017 06:26:08 GMT):
Has joined the channel.
baohua (Mon, 24 Jul 2017 06:37:56 GMT):
nice work!
chawlanikhil24 (Mon, 24 Jul 2017 06:45:31 GMT):
Thanks :)
indirajith (Wed, 26 Jul 2017 12:54:33 GMT):
@hightall please review https://gerrit.hyperledger.org/r/#/c/11809/ , thanks!
baohua (Thu, 27 Jul 2017 02:32:57 GMT):
Message Attachments
qiang0723 (Thu, 27 Jul 2017 06:37:05 GMT):
https://gerrit.hyperledger.org/r/#/c/11965/
vu3mmg (Thu, 27 Jul 2017 12:53:05 GMT):
Could you please guide me , to setup a dev env for cello
albrandt (Thu, 27 Jul 2017 19:15:21 GMT):
Has left the channel.
duwenhui (Fri, 28 Jul 2017 01:51:09 GMT):
How to check out the unmerged code ownered by other developer in gerrit?
baohua (Fri, 28 Jul 2017 02:02:53 GMT):
Use the gerritweb link to find the tar.gz package.
baohua (Fri, 28 Jul 2017 02:03:00 GMT):
Message Attachments
baohua (Fri, 28 Jul 2017 02:03:05 GMT):
https://gerrit.hyperledger.org/r/#/c/11985
duwenhui (Fri, 28 Jul 2017 02:14:03 GMT):
@baohua I can't find the tar.gz package.
baohua (Fri, 28 Jul 2017 02:14:43 GMT):
open gitweb link, then you will see there's a snapshot link
duwenhui (Fri, 28 Jul 2017 02:19:09 GMT):
@baohua thx, i get it.
baohua (Fri, 28 Jul 2017 02:21:24 GMT):
sure :)
Andy Zhang (Fri, 28 Jul 2017 08:11:27 GMT):
Has joined the channel.
dijun (Fri, 28 Jul 2017 08:32:15 GMT):
Has joined the channel.
Andy Zhang (Fri, 28 Jul 2017 10:27:38 GMT):
`make check` now failed with the following error message:
flake8 create: /Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/.tox/flake8
flake8 installdeps: flake8, -r/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/requirements.txt
flake8 installed: bcrypt==3.1.0,cached-property==1.3.0,cffi==1.10.0,click==6.7,colorama==0.3.9,docker==2.4.2,docker-compose==1.12.0,docker-pycreds==0.2.1,dockerpty==0.4.1,docopt==0.6.2,flake8==3.4.0,Flask==0.12.2,Flask-Login==0.4.0,greenlet==0.4.12,gunicorn==19.6.0,itsdangerous==0.24,Jinja2==2.9.6,jsonschema==2.6.0,MarkupSafe==1.0,mccabe==0.6.1,mongoengine==0.13.0,pycodestyle==2.3.1,pycparser==2.18,pyflakes==1.5.0,pymongo==3.4.0,PyYAML==3.12,requests==2.13.0,six==1.10.0,texttable==0.8.8,websocket-client==0.44.0,Werkzeug==0.12.2
flake8 runtests: PYTHONHASHSEED='3777955195'
flake8 runtests: commands[0] | flake8 --ignore=E402,F401 /Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/config.py:12:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/__init__.py:6:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/cluster.py:54:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:277:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:384:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:393:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:502:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:538:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:572:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/cluster.py:93:5: C901 'ClusterHandler.create' is too complex (17)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/cluster.py:109:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/cluster.py:582:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/cluster.py:641:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/host.py:217:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/host.py:324:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/resources/cluster_api.py:18:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/resources/login.py:14:79: E501 line too long (79 > 78 characters)
ERROR: InvocationError: '/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/.tox/flake8/bin/flake8 --ignore=E402,F401 /Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src'
Andy Zhang (Fri, 28 Jul 2017 10:27:38 GMT):
`make check` now failed with the following error message:
flake8 create: /Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/.tox/flake8
flake8 installdeps: flake8, -r/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/requirements.txt
flake8 installed: bcrypt==3.1.0,cached-property==1.3.0,cffi==1.10.0,click==6.7,colorama==0.3.9,docker==2.4.2,docker-compose==1.12.0,docker-pycreds==0.2.1,dockerpty==0.4.1,docopt==0.6.2,flake8==3.4.0,Flask==0.12.2,Flask-Login==0.4.0,greenlet==0.4.12,gunicorn==19.6.0,itsdangerous==0.24,Jinja2==2.9.6,jsonschema==2.6.0,MarkupSafe==1.0,mccabe==0.6.1,mongoengine==0.13.0,pycodestyle==2.3.1,pycparser==2.18,pyflakes==1.5.0,pymongo==3.4.0,PyYAML==3.12,requests==2.13.0,six==1.10.0,texttable==0.8.8,websocket-client==0.44.0,Werkzeug==0.12.2
flake8 runtests: PYTHONHASHSEED='3777955195'
flake8 runtests: commands[0] | flake8 --ignore=E402,F401 /Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/config.py:12:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/__init__.py:6:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/cluster.py:54:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:277:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:384:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:393:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:502:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:538:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:572:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/cluster.py:93:5: C901 'ClusterHandler.create' is too complex (17)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/cluster.py:109:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/cluster.py:582:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/cluster.py:641:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/host.py:217:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/host.py:324:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/resources/cluster_api.py:18:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/resources/login.py:14:79: E501 line too long (79 > 78 characters)
ERROR: InvocationError: '/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/.tox/flake8/bin/flake8 --ignore=E402,F401 /Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src'
Andy Zhang (Fri, 28 Jul 2017 10:27:38 GMT):
`make check` now failed with the following error message:
flake8 create: /Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/.tox/flake8
flake8 installdeps: flake8, -r/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/requirements.txt
flake8 installed: bcrypt==3.1.0,cached-property==1.3.0,cffi==1.10.0,click==6.7,colorama==0.3.9,docker==2.4.2,docker-compose==1.12.0,docker-pycreds==0.2.1,dockerpty==0.4.1,docopt==0.6.2,flake8==3.4.0,Flask==0.12.2,Flask-Login==0.4.0,greenlet==0.4.12,gunicorn==19.6.0,itsdangerous==0.24,Jinja2==2.9.6,jsonschema==2.6.0,MarkupSafe==1.0,mccabe==0.6.1,mongoengine==0.13.0,pycodestyle==2.3.1,pycparser==2.18,pyflakes==1.5.0,pymongo==3.4.0,PyYAML==3.12,requests==2.13.0,six==1.10.0,texttable==0.8.8,websocket-client==0.44.0,Werkzeug==0.12.2
flake8 runtests: PYTHONHASHSEED='3777955195'
flake8 runtests: commands[0] | flake8 --ignore=E402,F401 /Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/config.py:12:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/__init__.py:6:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/cluster.py:54:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:277:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:384:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:393:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:502:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:538:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:572:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/cluster.py:93:5: C901 'ClusterHandler.create' is too complex (17)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/cluster.py:109:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/cluster.py:582:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/cluster.py:641:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/host.py:217:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/host.py:324:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/resources/cluster_api.py:18:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/resources/login.py:14:79: E501 line too long (79 > 78 characters)
ERROR: InvocationError: '/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/.tox/flake8/bin/flake8 --ignore=E402,F401 /Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src'
Andy Zhang (Fri, 28 Jul 2017 10:27:38 GMT):
`make check` now failed with the following error message:
flake8 create: /Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/.tox/flake8
flake8 installdeps: flake8, -r/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/requirements.txt
flake8 installed: bcrypt==3.1.0,cached-property==1.3.0,cffi==1.10.0,click==6.7,colorama==0.3.9,docker==2.4.2,docker-compose==1.12.0,docker-pycreds==0.2.1,dockerpty==0.4.1,docopt==0.6.2,flake8==3.4.0,Flask==0.12.2,Flask-Login==0.4.0,greenlet==0.4.12,gunicorn==19.6.0,itsdangerous==0.24,Jinja2==2.9.6,jsonschema==2.6.0,MarkupSafe==1.0,mccabe==0.6.1,mongoengine==0.13.0,pycodestyle==2.3.1,pycparser==2.18,pyflakes==1.5.0,pymongo==3.4.0,PyYAML==3.12,requests==2.13.0,six==1.10.0,texttable==0.8.8,websocket-client==0.44.0,Werkzeug==0.12.2
flake8 runtests: PYTHONHASHSEED='3777955195'
flake8 runtests: commands[0] | flake8 --ignore=E402,F401 /Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/config.py:12:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/__init__.py:6:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/cluster.py:54:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:277:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:384:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:393:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:502:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:538:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:572:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/cluster.py:93:5: C901 'ClusterHandler.create' is too complex (17)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/cluster.py:109:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/cluster.py:582:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/cluster.py:641:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/host.py:217:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/host.py:324:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/resources/cluster_api.py:18:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/resources/login.py:14:79: E501 line too long (79 > 78 characters)
ERROR: InvocationError: '/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/.tox/flake8/bin/flake8 --ignore=E402,F401 /Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src'
Andy Zhang (Fri, 28 Jul 2017 10:27:38 GMT):
`make check` now failed with the following error message:
```
Andy Zhang (Fri, 28 Jul 2017 10:27:38 GMT):
`make check` now failed with the following error message:
```
flake8 create: /Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/.tox/flake8
flake8 installdeps: flake8, -r/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/requirements.txt
flake8 installed: bcrypt==3.1.0,cached-property==1.3.0,cffi==1.10.0,click==6.7,colorama==0.3.9,docker==2.4.2,docker-compose==1.12.0,docker-pycreds==0.2.1,dockerpty==0.4.1,docopt==0.6.2,flake8==3.4.0,Flask==0.12.2,Flask-Login==0.4.0,greenlet==0.4.12,gunicorn==19.6.0,itsdangerous==0.24,Jinja2==2.9.6,jsonschema==2.6.0,MarkupSafe==1.0,mccabe==0.6.1,mongoengine==0.13.0,pycodestyle==2.3.1,pycparser==2.18,pyflakes==1.5.0,pymongo==3.4.0,PyYAML==3.12,requests==2.13.0,six==1.10.0,texttable==0.8.8,websocket-client==0.44.0,Werkzeug==0.12.2
flake8 runtests: PYTHONHASHSEED='3777955195'
flake8 runtests: commands[0] | flake8 --ignore=E402,F401 /Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/config.py:12:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/__init__.py:6:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/cluster.py:54:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:277:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:384:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:393:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:502:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:538:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/agent/docker/docker_swarm.py:572:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/cluster.py:93:5: C901 'ClusterHandler.create' is too complex (17)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/cluster.py:109:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/cluster.py:582:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/cluster.py:641:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/host.py:217:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/modules/host.py:324:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/resources/cluster_api.py:18:79: E501 line too long (79 > 78 characters)
/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src/resources/login.py:14:79: E501 line too long (79 > 78 characters)
ERROR: InvocationError: '/Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/.tox/flake8/bin/flake8 --ignore=E402,F401 /Volumes/VMDisk/dev/go/src/github.com/hyperledger/cello/src'
```
baohua (Fri, 28 Jul 2017 11:42:58 GMT):
strange, why the minimal width is 78 in ur env? i guess 79 should be ok... that should be related to ur flake8 default checking rule. can u verify?
CBTheis (Fri, 28 Jul 2017 20:32:46 GMT):
Has joined the channel.
Andy Zhang (Mon, 31 Jul 2017 04:48:00 GMT):
I think so. Looks like default max-line-length on Mac for flake8 is 78. Maybe we could configure this value and others in tox.ini file as said in http://flake8.pycqa.org/en/latest/user/configuration.html
baohua (Mon, 31 Jul 2017 05:28:15 GMT):
sure, feel free to help create a jira task and fix. i guess 79 is more common.
baohua (Mon, 31 Jul 2017 12:18:52 GMT):
tong has submitted a patch set to add all the fabric-deploy code onto cello. Here is the patch set, please take a look when you have time. https://gerrit.hyperledger.org/r/#/c/12059
harsha (Mon, 31 Jul 2017 12:23:24 GMT):
Has joined the channel.
tongli (Mon, 31 Jul 2017 12:26:33 GMT):
Has joined the channel.
baohua (Mon, 31 Jul 2017 12:42:46 GMT):
pls help assign it some jira task, feel free to reuse this one if u would like to, thanks! https://jira.hyperledger.org/browse/CE-95
baohua (Tue, 01 Aug 2017 03:21:22 GMT):
@chawlanikhil24 can u have a look at this patchset? https://gerrit.hyperledger.org/r/#/c/11139/
baohua (Tue, 01 Aug 2017 03:22:11 GMT):
@duwenhui hi, can u post the link here? :)
duwenhui (Tue, 01 Aug 2017 03:24:24 GMT):
Error: POST failure [Post http://localhost:7054/enroll: dial tcp [::1]:7056: getsockopt: connection refused]; not sending
duwenhui (Tue, 01 Aug 2017 03:24:59 GMT):
https://gerrit.hyperledger.org/r/#/c/10871/
duwenhui (Tue, 01 Aug 2017 03:25:18 GMT):
./network_setup.sh up
setting to default channel 'mychannel'
mychannel
#################################################################
####### Generating crypto material using Fabric CA ##########
#################################################################
Checking executables ...
Setting up organizations ...
Starting CA server in crypto-config/ordererOrganizations/example.com/ca/root on port 7054 ...
CA server is started in crypto-config/ordererOrganizations/example.com/ca/root and listening on port 7054
Enrolled crypto-config/ordererOrganizations/example.com/ca/root/tls with CA at http://admin:adminpw@localhost:7054
Enrolled crypto-config/ordererOrganizations/example.com/users/rootAdmin with CA at http://admin:adminpw@localhost:7054
Registered user nodeAdmin with intermediate CA as crypto-config/ordererOrganizations/example.com/users/rootAdmin
Enrolled crypto-config/ordererOrganizations/example.com/users/Admin@example.com with CA at http://nodeAdmin:secret@localhost:7054
Registered user User1@example.com with intermediate CA as crypto-config/ordererOrganizations/example.com/users/rootAdmin
Enrolled crypto-config/ordererOrganizations/example.com/users/User1@example.com with CA at http://User1@example.com:secret@localhost:7054
Enrolled crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/tls with CA at http://admin:adminpw@localhost:7054
Registered user orderer.example.com with intermediate CA as crypto-config/ordererOrganizations/example.com/users/rootAdmin
Enrolled crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com with CA at http://orderer.example.com:secret@localhost:7054
Loaded CA certificates into crypto-config/ordererOrganizations/example.com from CA at http://admin:adminpw@localhost:7054
Starting CA server in crypto-config/peerOrganizations/org1.example.com/ca/root on port 7056 ...
CA server is started in crypto-config/peerOrganizations/org1.example.com/ca/root and listening on port 7056
FATAL: Failed to enroll crypto-config/peerOrganizations/org1.example.com/ca/root/tls with CA at http://admin:adminpw@localhost:7056; see crypto-config/peerOrganizations/org1.example.com/ca/root/tls/enroll.log
duwenhui (Tue, 01 Aug 2017 03:25:26 GMT):
@baohua
duwenhui (Tue, 01 Aug 2017 03:25:26 GMT):
@baohua duwenhui 下午2点45分
@smithbk
I searched for the previous reply and still couldn't find the answer. I tried the ./fabric-ca-client enroll -u http://admin:adminpw@localhost:7055, which executed successfully. Can you tell a clearer solution?
smithbk 2017年7月27日晚上8点09分
@duwenhui For some reason, the fabric-ca-server failed to start on port 7055. There must be an earlier error message indicating why.
```
Error: POST failure [Post http://localhost:7055/enroll: dial tcp [::1]:7055: getsockopt: connection refused]; not sending
```
duwenhui (Tue, 01 Aug 2017 03:27:45 GMT):
In this case ,I think that whenever I start a second CA server with a command and register and enroll i will make this error
duwenhui (Tue, 01 Aug 2017 03:39:41 GMT):
i modefide INTERMEDIATE_CA=false in the file fabric-ca-cryptogen.sh
chawlanikhil24 (Tue, 01 Aug 2017 04:58:45 GMT):
@baohua YEAP! this task is on my list. For now, I m figuring out the chaincode container issue and I am very close to crack it. :)
baohua (Tue, 01 Aug 2017 06:18:32 GMT):
@duwenhui wenhui, i guess the port should be 7054.
baohua (Tue, 01 Aug 2017 06:18:49 GMT):
are u testing on a new chain started by cello?
duwenhui (Tue, 01 Aug 2017 06:23:07 GMT):
@baohua Not port, I started multiple CA server one port is 7054, and the other port is 7055.
I haven't tested the new chain yet, and I will test it. I added the solo mode chuain a few days ago, but it was wrong when I deleted it.
baohua (Tue, 01 Aug 2017 07:49:22 GMT):
ok, sorry for the instability. pls help report the steps and operations if you find any bug. jira.hyperledger.org
baohua (Tue, 01 Aug 2017 07:49:26 GMT):
@hightall pls help have a look at https://gerrit.hyperledger.org/r/11985, thanks!
duwenhui (Tue, 01 Aug 2017 09:37:09 GMT):
@baohua
I just tested the e Cello Add Chain and found that all the Peer nodes were not started properly because of the certificate problem.
block@BlockChain:~/gocode/src/github.com/cello$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
block@BlockChain:~/gocode/src/github.com/cello$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a4e5294c473b hyperledger/fabric-tools "bash -c 'sleep 2;..." 31 seconds ago Up 31 seconds 59804a73292d37001451689b_fabric-cli
74334a89b8a7 hyperledger/fabric-peer "peer node start" 40 seconds ago Restarting (1) 4 seconds ago 59804a73292d37001451689b_peer0-org2
b003c15319d5 hyperledger/fabric-peer "peer node start" 40 seconds ago Restarting (1) 6 seconds ago 59804a73292d37001451689b_peer0-org1
4a3706fcad12 hyperledger/fabric-orderer "orderer start" 40 seconds ago Restarting (1) 5 seconds ago 59804a73292d37001451689b_orderer
b3a683e5c95a hyperledger/fabric-ca "fabric-ca-server ..." 40 seconds ago Up 37 seconds 7054/tcp 59804a73292d37001451689b_fabric-ca
46c3da0c460d hyperledger/fabric-peer "peer node start" 40 seconds ago Restarting (1) 4 seconds ago 59804a73292d37001451689b_peer1-org1
9e2fa259c759 hyperledger/fabric-peer "peer node start" 40 seconds ago Restarting (1) 7 seconds ago 59804a73292d37001451689b_peer1-org2
block@BlockChain:~/gocode/src/github.com/cello$ docker logs 59804a73292d37001451689b_peer1-org2
2017-08-01 09:31:33.074 UTC [main] main -> ERRO 001 Cannot run peer because error when setting up MSP from directory /etc/hyperledger/fabric/msp: err Could not load a valid signer certificate from directory /etc/hyperledger/fabric/msp/signcerts, err stat /etc/hyperledger/fabric/msp/signcerts: no such file or directory
2017-08-01 09:31:34.934 UTC [main] main -> ERRO 001 Cannot run peer because error when setting up MSP from directory /etc/hyperledger/fabric/msp: err Could not load a valid signer certificate from directory /etc/hyperledger/fabric/msp/signcerts, err stat /etc/hyperledger/fabric/msp/signcerts: no such file or directory
2017-08-01 09:31:36.740 UTC [main] main -> ERRO 001 Cannot run peer because error when setting up MSP from directory /etc/hyperledger/fabric/msp: err Could not load a valid signer certificate from directory /etc/hyperledger/fabric/msp/signcerts, err stat /etc/hyperledger/fabric/msp/signcerts: no such file or directory
2017-08-01 09:31:38.170 UTC [main] main -> ERRO 001 Cannot run peer because error when setting up MSP from directory /etc/hyperledger/fabric/msp: err Could not load a valid signer certificate from directory /etc/hyperledger/fabric/msp/signcerts, err stat /etc/hyperledger/fabric/msp/signcerts: no such file or directory
2017-08-01 09:31:39.894 UTC [main] main -> ERRO 001 Cannot run peer because error when setting up MSP from directory /etc/hyperledger/fabric/msp: err Could not load a valid signer certificate from directory /etc/hyperledger/fabric/msp/signcerts, err stat /etc/hyperledger/fabric/msp/signcerts: no such file or directory
2017-08-01 09:31:41.954 UTC [main] main -> ERRO 001 Cannot run peer because error when setting up MSP from directory /etc/hyperledger/fabric/msp: err Could not load a valid signer certificate from directory /etc/hyperledger/fabric/msp/signcerts, err stat /etc/hyperledger/fabric/msp/signcerts: no such file or directory
2017-08-01 09:31:45.481 UTC [main] main -> ERRO 001 Cannot run peer because error when setting up MSP from directory /etc/hyperledger/fabric/msp: err Could not load a valid signer certificate from directory /etc/hyperledger/fabric/msp/signcerts, err stat /etc/hyperledger/fabric/msp/signcerts: no such file or directory
2017-08-01 09:31:52.154 UTC [main] main -> ERRO 001 Cannot run peer because error when setting up MSP from directory /etc/hyperledger/fabric/msp: err Could not load a valid signer certificate from directory /etc/hyperledger/fabric/msp/signcerts, err stat /etc/hyperledger/fabric/msp/signcerts: no such file or directory
2017-08-01 09:32:05.290 UTC [main] main -> ERRO 001 Cannot run peer because error when setting up MSP from directory /etc/hyperledger/fabric/msp: err Could not load a valid signer certificate from directory /etc/hyperledger/fabric/msp/signcerts, err stat /etc/hyperledger/fabric/msp/signcerts: no such file or directory
block@BlockChain:~/gocode/src/github.com/cello$
baohua (Tue, 01 Aug 2017 09:41:13 GMT):
have u mapped the crypto-config dir at the worker node, as the doc indicated?
Luke_Chen (Tue, 01 Aug 2017 09:51:55 GMT):
Hello, how could I use docker swarm to setup a cluster ? I can't find any doc
duwenhui (Tue, 01 Aug 2017 09:52:03 GMT):
@baohua I can't find the instructions of the document, please help to attach a specific connection.
tongli (Tue, 01 Aug 2017 12:52:47 GMT):
@Luke_Chen not sure what your goal is, but if you like, please take a look at this project. https://github.com/litong01/fabric-deploy
tongli (Tue, 01 Aug 2017 12:53:07 GMT):
use that project you can either setup docker swarm like env or k8s.
tongli (Tue, 01 Aug 2017 12:53:22 GMT):
the project then will deploy fabric on top of either.
tongli (Tue, 01 Aug 2017 12:54:26 GMT):
if you only are interested in how the docker swarm like env setup, you can look at this role. https://github.com/litong01/fabric-deploy/tree/master/roles/deploy_compose
tongli (Tue, 01 Aug 2017 12:54:57 GMT):
it uses docker-compose to deploy fabric onto a docker + overlay network.
tongli (Tue, 01 Aug 2017 12:55:09 GMT):
which is very similar to docker swarm env.
tongli (Tue, 01 Aug 2017 12:55:58 GMT):
but to truly look into how the overlay network gets setup, take a look at this role. https://github.com/litong01/fabric-deploy/tree/master/roles/env_flanneld/networksetup
baohua (Tue, 01 Aug 2017 14:00:07 GMT):
@Luke_Chen are u asking how to setup a swarm cluster of setup a fabric network on top of a docker swarm? For the first question, pls refer to the docker documentation. for the second one, it's quite easy to do that using cello. Just import the swarm cluster as a worker node in cello's dashboard.
Luke_Chen (Wed, 02 Aug 2017 02:57:47 GMT):
@baohua Thank you , but could you please tell me more details about swarm cluster ? It's that mean add docker swarm manager node ?
Luke_Chen (Wed, 02 Aug 2017 03:01:37 GMT):
@tongli Thank you for your response, I want to know how to trigger docker swarm mode in cello project.
baohua (Wed, 02 Aug 2017 03:02:49 GMT):
yes, to add existing swarm cluster into Cello, just input the manager API like a single Docker Host
baohua (Wed, 02 Aug 2017 03:03:03 GMT):
like x.x.x.x:2376
baohua (Wed, 02 Aug 2017 03:03:25 GMT):
remember to enable the overlay networking mode, hence u can have fabric over multi-hosts
Luke_Chen (Wed, 02 Aug 2017 03:03:47 GMT):
@baohua OK thank you very much
naohide (Wed, 02 Aug 2017 04:09:47 GMT):
Has joined the channel.
baohua (Wed, 02 Aug 2017 05:50:04 GMT):
https://gerrit.hyperledger.org/r/#/c/11985/
Andy Zhang (Wed, 02 Aug 2017 08:55:20 GMT):
I fixed some broken link in documents, please help review:
https://gerrit.hyperledger.org/r/#/c/12009/
andyz (Wed, 02 Aug 2017 09:54:11 GMT):
Has joined the channel.
tongli (Wed, 02 Aug 2017 16:49:27 GMT):
@baohua thanks for reviewing the patch. It is quite big since it does so much. I will address your comments when I get back. Going on vacation.
tongli (Wed, 02 Aug 2017 16:50:07 GMT):
For anyone else in the cello group, please take a look when you have few minutes https://gerrit.hyperledger.org/r/#/c/12059/
baohua (Thu, 03 Aug 2017 01:37:30 GMT):
Thanks! tong. I do believe this is a great enhance on CE-95. Everyone, pls help review the patchset, would like to make it a perfect one! @here
baohua (Thu, 03 Aug 2017 01:38:10 GMT):
CE-95
liuwenliang0632 (Thu, 03 Aug 2017 01:43:50 GMT):
:thumbsup:
baohua (Thu, 03 Aug 2017 13:32:48 GMT):
congrats to @Andy Zhang the 1st patchset got merged! https://gerrit.hyperledger.org/r/#/c/12009/
baohua (Thu, 03 Aug 2017 13:32:59 GMT):
CE-100
Stam 5 (Thu, 03 Aug 2017 21:29:19 GMT):
Has joined the channel.
kenstaker (Fri, 04 Aug 2017 20:24:27 GMT):
Has joined the channel.
hightall (Sun, 06 Aug 2017 02:42:52 GMT):
https://gerrit.hyperledger.org/r/#/c/12173/
baohua (Mon, 07 Aug 2017 01:58:01 GMT):
:thumbsup:
mavstronaut (Tue, 08 Aug 2017 04:03:08 GMT):
Has joined the channel.
GongZhang (Thu, 10 Aug 2017 07:11:44 GMT):
Has joined the channel.
baohua (Thu, 10 Aug 2017 14:58:33 GMT):
https://gerrit.hyperledger.org/r/#/c/12315/
hightall (Thu, 10 Aug 2017 15:09:45 GMT):
https://gerrit.hyperledger.org/r/#/c/12315/
smonfort (Fri, 11 Aug 2017 20:34:25 GMT):
Has joined the channel.
chawlanikhil24 (Sun, 13 Aug 2017 08:46:34 GMT):
https://gerrit.hyperledger.org/r/#/c/12389/
hightall (Sun, 13 Aug 2017 13:47:16 GMT):
https://gerrit.hyperledger.org/r/#/c/12391/
surajit-paul (Sun, 13 Aug 2017 21:09:19 GMT):
Has joined the channel.
baohua (Mon, 14 Aug 2017 14:40:14 GMT):
Welcome for comments: https://jira.hyperledger.org/projects/CE/issues/CE-106
Luke_Chen (Wed, 16 Aug 2017 08:31:32 GMT):
Hello everyone, we are going to start a meeting to discuss "Enhance Cello to support Fabric on top of vSphere" on Aug 16 at 3:00 PM(GMT+8) , please feel free to join us.
link: uberconference.com/jiahaoc
tel: 719-284-5152
pin: 14190
Luke_Chen (Wed, 16 Aug 2017 08:51:24 GMT):
Sorry guys, the date should be Aug 17 at 3:00 PM(GMT+8)
Luke_Chen (Wed, 16 Aug 2017 08:56:05 GMT):
And welcome for comments https://jira.hyperledger.org/browse/CE-106
jmcnevin (Wed, 16 Aug 2017 17:05:37 GMT):
Has left the channel.
Luke_Chen (Thu, 17 Aug 2017 03:48:22 GMT):
Meeting about Enhance Cello to support Fabric on top of vSphere will be started on 8/17 at 3:00 pm(GMT+8),please feel free to join us.
cello issue link: https://jira.hyperledger.org/browse/CE-106
uberconference link: uberconference.com/jiahaoc
tel: 719-284-5152
pin: 14190
baohua (Thu, 17 Aug 2017 03:51:53 GMT):
:grinning: will attend
Luke_Chen (Thu, 17 Aug 2017 03:59:20 GMT):
@baohua welcome !
Luke_Chen (Thu, 17 Aug 2017 09:35:39 GMT):
Hello everyone, to view meeting record please check this link https://wiki.hyperledger.org/projects/cello/meetings/20170817
tongli (Thu, 17 Aug 2017 14:28:22 GMT):
@here new patch set to address ansible fabric deploy patch comments is here. please review.
tongli (Thu, 17 Aug 2017 14:28:25 GMT):
https://gerrit.hyperledger.org/r/#/c/12059/
Vadim (Thu, 17 Aug 2017 14:28:35 GMT):
Has left the channel.
tongli (Thu, 17 Aug 2017 15:44:40 GMT):
@baohua your two comments both already addressed in the patch. I should have been more clear on that. Thanks.
baohua (Fri, 18 Aug 2017 02:54:21 GMT):
got, thanks!
baohua (Fri, 18 Aug 2017 08:49:25 GMT):
Welcome for comments: https://docs.google.com/document/d/1SRS8A6l-Wf64qVVoPkLuyOlPbZd5CckG-nGxHVnK6SM/edit?ts=5996a918#heading=h.gvixbdeqwdmv
tongli (Fri, 18 Aug 2017 13:36:11 GMT):
@baohua thanks for reviewing the patch set.
tongli (Fri, 18 Aug 2017 13:36:52 GMT):
@here, please review this patch set which added tremendous capabilities to cello. https://gerrit.hyperledger.org/r/#/c/12059
baohua (Fri, 18 Aug 2017 13:37:15 GMT):
my pleasure, let's see other's comments, thanks!
tongli (Fri, 18 Aug 2017 13:37:29 GMT):
sure.
hightall (Fri, 18 Aug 2017 13:38:33 GMT):
https://gerrit.hyperledger.org/r/#/c/12587/
kkpatel (Fri, 18 Aug 2017 18:36:56 GMT):
Has joined the channel.
hightall (Sat, 19 Aug 2017 08:42:26 GMT):
https://gerrit.hyperledger.org/r/#/c/12615/
indirajith (Mon, 21 Aug 2017 01:02:27 GMT):
https://gerrit.hyperledger.org/r/#/c/12627/
baohua (Mon, 21 Aug 2017 01:25:08 GMT):
@indirajith give some comments, thanks
DannyWong (Mon, 21 Aug 2017 02:49:10 GMT):
Hi, is current cello supports Fabric v1 (previously it mainly supports 0.6)
baohua (Mon, 21 Aug 2017 07:24:48 GMT):
yes, fabric v1 can be run with cello now, with underly infrastructures can be docker or docker swarm cluster.
chawlanikhil24 (Mon, 21 Aug 2017 11:42:09 GMT):
Manual deployment scripts with proposed changes is added in this PR
chawlanikhil24 (Mon, 21 Aug 2017 11:42:10 GMT):
https://gerrit.hyperledger.org/r/#/c/12637/
chawlanikhil24 (Mon, 21 Aug 2017 11:42:46 GMT):
I need to terminate this PR , https://gerrit.hyperledger.org/r/#/c/12389/
chawlanikhil24 (Mon, 21 Aug 2017 11:42:56 GMT):
how can I do it //
chawlanikhil24 (Mon, 21 Aug 2017 11:42:56 GMT):
?
baohua (Mon, 21 Aug 2017 12:10:52 GMT):
just abandon @chawlanikhil24 :)
baohua (Mon, 21 Aug 2017 12:11:48 GMT):
btw, would u like to demo how to support k8s sometime this week with the new patchset?
chawlanikhil24 (Mon, 21 Aug 2017 13:01:01 GMT):
Sure !. How can I do it ? Via some online meeting ?
baohua (Mon, 21 Aug 2017 13:33:11 GMT):
sure, how about use the uberconf to demo online? which tz are u at?
chawlanikhil24 (Mon, 21 Aug 2017 15:24:55 GMT):
My is , IST
chawlanikhil24 (Mon, 21 Aug 2017 15:25:28 GMT):
Or I upload a demo video on YouTube
chawlanikhil24 (Mon, 21 Aug 2017 15:25:33 GMT):
What say ?
indirajith (Mon, 21 Aug 2017 15:39:14 GMT):
@baohua Thanks, I will do that.
indirajith (Mon, 21 Aug 2017 15:42:24 GMT):
IST is UTC + 5.30 hours.
baohua (Tue, 22 Aug 2017 01:26:42 GMT):
great, i'm UTC+8. You can feel free to start an online conf or using youtube, both should be fine @chawlanikhil24 @indirajith
baohua (Tue, 22 Aug 2017 03:10:32 GMT):
@HenryZhang would u like to help review https://gerrit.hyperledger.org/r/#/c/12637? Which is k8s support.
chawlanikhil24 (Tue, 22 Aug 2017 04:37:50 GMT):
Ohk! Great, YouTube is way better for others as well, kind of a online tutorial for anyone who wants to deploy. I'll share the link here as soon as I am done with the Video
HenryZhang (Tue, 22 Aug 2017 05:47:18 GMT):
@baohua got it. will do.
tongli (Tue, 22 Aug 2017 12:40:06 GMT):
@here friends, good morning and evening, this patch got two +2s already, can someone please push that button to merge it? https://gerrit.hyperledger.org/r/#/c/12059/
baohua (Tue, 22 Aug 2017 12:42:11 GMT):
let me find xiaoxi, as he gave comments in previous reviews.
tongli (Tue, 22 Aug 2017 13:00:18 GMT):
@baohua do you mean my patch? I think that xiaoxi wanted to split it into smaller pieces for easier review.
baohua (Tue, 22 Aug 2017 13:37:46 GMT):
yes, and i guess u've answered him. Would like to see if he has other comments. Do you mind we collect the advice from him? He has very professional experience in ansible.
baohua (Tue, 22 Aug 2017 13:54:52 GMT):
Congrats @tongli for his 1st merged patchset on project Hyperledger Cello, hope this helps u join the team well! :grinning: https://gerrit.hyperledger.org/r/#/c/12059/
tongli (Tue, 22 Aug 2017 13:55:10 GMT):
haha. cool, cool.
tongli (Tue, 22 Aug 2017 13:56:01 GMT):
Thanks guys. this is to start the project with the battle tested tool for deployment.
tongli (Tue, 22 Aug 2017 13:56:17 GMT):
I am sure people will like to use ansible. we can use cello UI to drive it.
baohua (Tue, 22 Aug 2017 13:56:38 GMT):
Let's keep refining as the review comments, I believe this would be a good beginning!
tongli (Tue, 22 Aug 2017 13:56:45 GMT):
there is an ansible driver in python to integrate ansible with python program.
tongli (Tue, 22 Aug 2017 13:57:02 GMT):
yes, totally.
tongli (Tue, 22 Aug 2017 13:57:13 GMT):
I am sure there are so many places that we can improve it.
tongli (Tue, 22 Aug 2017 13:57:53 GMT):
I will be thrilled to see if others starting patches based on this one to make the deployment of fabric like a breeze.
hightall (Tue, 22 Aug 2017 15:24:27 GMT):
https://gerrit.hyperledger.org/r/#/c/12625/
hightall (Wed, 23 Aug 2017 02:04:53 GMT):
https://gerrit.hyperledger.org/r/#/c/12683/
waaagh (Wed, 23 Aug 2017 04:53:14 GMT):
Has joined the channel.
waaagh (Wed, 23 Aug 2017 04:54:45 GMT):
Hi, my network is behind a proxy, is there a way to setup http(s)_proxy or change npm/pip mirror when build the docker images?
codewookie (Wed, 23 Aug 2017 11:07:05 GMT):
Has joined the channel.
chawlanikhil24 (Wed, 23 Aug 2017 13:44:16 GMT):
@here, need your advice, exposing peer containers to external endpoints will be a good move ? I guess its not safe from security point of view !
tongli (Wed, 23 Aug 2017 13:45:09 GMT):
well, peer and orders are eventually need to be accessible from client.
tongli (Wed, 23 Aug 2017 13:45:43 GMT):
in a real world production setup, they should be exposed to outside of your network.
chawlanikhil24 (Wed, 23 Aug 2017 13:46:00 GMT):
I guess, cli container serves the purpose
tongli (Wed, 23 Aug 2017 13:46:07 GMT):
otherwise how sdks (which get used by client) will be able to use them?
tongli (Wed, 23 Aug 2017 13:46:19 GMT):
ah?
chawlanikhil24 (Wed, 23 Aug 2017 13:46:30 GMT):
since, it can be connected to another peers and orderers
tongli (Wed, 23 Aug 2017 13:46:54 GMT):
cli container is something an admin may use.
tongli (Wed, 23 Aug 2017 13:46:59 GMT):
client can not use that.
chawlanikhil24 (Wed, 23 Aug 2017 13:47:25 GMT):
cli has an endpoint ! in my case
chawlanikhil24 (Wed, 23 Aug 2017 13:47:46 GMT):
I make it deliberately external
chawlanikhil24 (Wed, 23 Aug 2017 13:48:01 GMT):
so that , there should be only one accessible point to the whole cluster
chawlanikhil24 (Wed, 23 Aug 2017 13:49:24 GMT):
cli -> peer
cli -> orderer
cli -> other PODS
using, ClusterIP
chawlanikhil24 (Wed, 23 Aug 2017 13:58:06 GMT):
[ ](https://chat.hyperledger.org/channel/cello?msg=NK7LuidFqi9gN7WpL) @tongli Hmm, this makes sense !
baohua (Thu, 24 Aug 2017 07:46:37 GMT):
https://gerrit.hyperledger.org/r/#/c/12683/
baohua (Thu, 24 Aug 2017 07:46:41 GMT):
@hightall
baohua (Thu, 24 Aug 2017 07:47:11 GMT):
https://gerrit.hyperledger.org/r/#/c/12637/ @chawlanikhil24
baohua (Thu, 24 Aug 2017 07:47:36 GMT):
https://gerrit.hyperledger.org/r/#/c/12627/ @indirajith pls help address the comments
baohua (Thu, 24 Aug 2017 07:47:38 GMT):
Thanks
waaagh (Fri, 25 Aug 2017 04:54:32 GMT):
@baohua hi, should i create cello_net_solo manually on the worker node?
i fellow the install guide and tutorial, but failed at chain create bcs of there is not that docker network
baohua (Fri, 25 Aug 2017 06:54:46 GMT):
no u needn't create manually. Could u help post ur steps and detailed logs with a jira bug under Cello project? jira.hyperledger.org, we will look at it, thanks!
waaagh (Fri, 25 Aug 2017 09:15:08 GMT):
https://jira.hyperledger.org/browse/CE-119 @baohua im a newbie to jira, let me know if you need more infomation or there's any mistakes i provide
FabricioMendes (Fri, 25 Aug 2017 16:51:51 GMT):
Has joined the channel.
deker1991 (Sat, 26 Aug 2017 12:44:27 GMT):
Has joined the channel.
indirajith (Mon, 28 Aug 2017 04:36:06 GMT):
Code Review - https://gerrit.hyperledger.org/r/#/c/12627/
baohua (Mon, 28 Aug 2017 05:35:22 GMT):
https://gerrit.hyperledger.org/r/#/c/12637/
baohua (Mon, 28 Aug 2017 05:35:53 GMT):
@waaagh thanks for ur report, @hightall is looking at ur problem.
waaagh (Mon, 28 Aug 2017 06:25:08 GMT):
glade to hear that :)
MarcelSchlatter (Mon, 28 Aug 2017 20:06:12 GMT):
Has joined the channel.
Luke_Chen (Tue, 29 Aug 2017 07:25:43 GMT):
please check the link below to see our design document and feel free to leave your comments.
https://docs.google.com/document/d/1ONnSkEzSA_JQ8LDuyvyjIEx20iNfGTgHRemYeC2c_o4/edit?usp=sharing
meanwhile, please refer cello issue link: https://jira.hyperledger.org/browse/CE-106.
Thank you
tongli (Tue, 29 Aug 2017 13:17:34 GMT):
@Luke_Chen We have recently added ansible modules to cello which provisions VM against aws, openstack, azuer, VirtualBox.
tongli (Tue, 29 Aug 2017 13:17:52 GMT):
if you like to support vSphere, (VMWare), it is very easy to do that.
tongli (Tue, 29 Aug 2017 13:18:16 GMT):
simply add a provision module (few playbooks) for the provisioning.
tongli (Tue, 29 Aug 2017 13:19:03 GMT):
IBM lab uses vSphere (a bit of customized vSphere).
tongli (Tue, 29 Aug 2017 13:19:20 GMT):
Ansible provides VMWare ansible module.
tongli (Tue, 29 Aug 2017 13:20:33 GMT):
I think you can probably think about that. Once VM(s) are provisioned, then the next step of setting up fabric network will be identical to do the same against other cloud as AWS, OpenStack.
tongli (Tue, 29 Aug 2017 13:20:44 GMT):
let me know if you have more questions.
tongli (Tue, 29 Aug 2017 16:44:59 GMT):
Does anybody know what the default login id and password for cello dashboard?
tongli (Tue, 29 Aug 2017 16:46:43 GMT):
never mind, admin/pass
chawlanikhil24 (Wed, 30 Aug 2017 04:08:15 GMT):
Hi People,
chawlanikhil24 (Wed, 30 Aug 2017 04:08:15 GMT):
https://gerrit.hyperledger.org/r/#/c/12637/
chawlanikhil24 (Wed, 30 Aug 2017 04:08:38 GMT):
This PR has been reviewed with +2 and can be merged
chawlanikhil24 (Wed, 30 Aug 2017 04:09:42 GMT):
Currently working on a better version for README and YouTube tutorial
tongli (Wed, 30 Aug 2017 13:30:51 GMT):
@baohua can you tell me how to configuration to cello is done? for example, if something needs to be set to a value, how the default is defined and how to specify a different value?
baohua (Wed, 30 Aug 2017 13:47:57 GMT):
@tongli hi tong, can i know what kind of configuration? for the cello service itself or for the fabric network?
tongli (Wed, 30 Aug 2017 13:48:15 GMT):
@baohua the docker port,
tongli (Wed, 30 Aug 2017 13:48:24 GMT):
it may not always run on 2375.
baohua (Wed, 30 Aug 2017 13:49:08 GMT):
yeap, when u specify a docker host as a "host" in cello?
tongli (Wed, 30 Aug 2017 13:49:15 GMT):
thinking making that configurable, default will be 2375, but people can make changes to the configuration file (if there is one) or env variable, so that the port will be different.
tongli (Wed, 30 Aug 2017 13:49:41 GMT):
I guess I am asking overall how configurable cello is.
tongli (Wed, 30 Aug 2017 13:49:54 GMT):
right.
baohua (Wed, 30 Aug 2017 13:50:56 GMT):
there are several places we using docker. If you mean importing a docker host, then it's simple. On the admin portal, you can specify the API url to whatever you want, e.g., 9.9.9.9:2376 (suppose there's a docker daemon listened)
tongli (Wed, 30 Aug 2017 13:53:28 GMT):
I can verify that. but I think I am asking what is the general mechanism to config cello? for example, what if someone wants the dashboard not being on 8080 but on 9090.
tongli (Wed, 30 Aug 2017 13:53:56 GMT):
is there a configuration file for cello so that once the installation is done, the dashboard goes to 9090 instead of 8080.
tongli (Wed, 30 Aug 2017 13:54:10 GMT):
also say I do not want to use admin:pass, I want to use something else,
tongli (Wed, 30 Aug 2017 13:54:19 GMT):
how is that getting done?
tongli (Wed, 30 Aug 2017 13:54:41 GMT):
I think I am asking for the general configuration mechanism of cello.
tongli (Wed, 30 Aug 2017 13:55:15 GMT):
if there is none, that is fine, we can add.
baohua (Wed, 30 Aug 2017 13:57:13 GMT):
yes, that's another question. and we now have a src/config.py file, which mainly configs web/mongo info. I guess we can feel free to extend from it. thoughts?
tongli (Wed, 30 Aug 2017 14:11:55 GMT):
cool. as long as there is a general way of configuration anything, that is good.
tongli (Wed, 30 Aug 2017 14:12:33 GMT):
the only drawback with python code is that people feel a bit intimidated changing code vs changing a yaml file or flat text file.
tongli (Wed, 30 Aug 2017 15:13:44 GMT):
Please review this patch. https://gerrit.hyperledger.org/r/#/c/12915/
tongli (Wed, 30 Aug 2017 15:14:13 GMT):
we need to add these cloud specific configuration files and these file also serve as templates.
baohua (Wed, 30 Aug 2017 15:31:05 GMT):
@chawlanikhil24 thanks, and let's add more docs and enable operate k8s based fabric using the dashboard.
tongli (Thu, 31 Aug 2017 00:45:41 GMT):
guys, please review this patch so that our testing team can start using the project at the CI.
tongli (Thu, 31 Aug 2017 00:45:51 GMT):
https://gerrit.hyperledger.org/r/#/c/12915/
baohua (Thu, 31 Aug 2017 08:11:41 GMT):
+2, thanks, and let's see other reviewers' comments
sk (Thu, 31 Aug 2017 08:44:28 GMT):
Has joined the channel.
chawlanikhil24 (Thu, 31 Aug 2017 09:32:31 GMT):
@baohua yea! I am on the job for better docs, and dashboard connection.
Luke_Chen (Thu, 31 Aug 2017 11:32:27 GMT):
Please use the follow link to comment on the design doc. Sorry for the inconvenient.
https://docs.google.com/document/d/19Ezh2bakm-drWFbiwwH0LLZz5f1Wi70QFPHWCVQgAeA/edit?usp=sharing
tongli (Thu, 31 Aug 2017 12:26:33 GMT):
@baohua thanks for reviewing the patch.
baohua (Thu, 31 Aug 2017 12:35:04 GMT):
my pleasure~
tongli (Thu, 31 Aug 2017 12:45:11 GMT):
@here, anybody else? please review https://gerrit.hyperledger.org/r/#/c/12915, this is really just adding the pre-can configuration files.
tongli (Thu, 31 Aug 2017 12:45:21 GMT):
pre-canned.
tongli (Thu, 31 Aug 2017 12:54:00 GMT):
@baohua, how do I find who has +2 power for cello? Thanks.
baohua (Thu, 31 Aug 2017 13:06:35 GMT):
tong, currently, you can feel free to ping @hightall. Seems we need more core reviewers, those who have made 3+ significant commits and show continuous participant for 3+ months, feel free to ping me :) Thanks!
baohua (Thu, 31 Aug 2017 13:06:35 GMT):
tong, currently, you can feel free to ping @hightall. Seems we need more core reviewers, those who have made 3+ significant commits and show continuous participant/contribution for 3+ months, feel free to ping me :) Thanks!
tongli (Thu, 31 Aug 2017 13:09:13 GMT):
cool. will ping him.
tongli (Thu, 31 Aug 2017 13:10:08 GMT):
@hightall can you please also look at the patch? Our testing team is waiting for the patch. https://gerrit.hyperledger.org/r/#/c/12915
tongli (Thu, 31 Aug 2017 13:10:25 GMT):
there are two more patch sets that I am working on.
tongli (Thu, 31 Aug 2017 13:12:51 GMT):
1. add tls enable/disable flag
tongli (Thu, 31 Aug 2017 15:40:23 GMT):
2. add ca container for generating certificates instead of using cryptogen
baohua (Fri, 01 Sep 2017 01:24:52 GMT):
cool, lft having those features. Believe they will enhance the project!
lmrln (Fri, 01 Sep 2017 10:00:00 GMT):
Has joined the channel.
tongli (Fri, 01 Sep 2017 15:33:32 GMT):
@here, I have created a jira issue and submitted a patch set to make tls enablement configurable for ansible deployment.
tongli (Fri, 01 Sep 2017 15:33:47 GMT):
please take a look at this jira issue and the patch set. Thanks.
tongli (Fri, 01 Sep 2017 15:34:04 GMT):
https://jira.hyperledger.org/browse/CE-120
tongli (Fri, 01 Sep 2017 15:34:18 GMT):
https://gerrit.hyperledger.org/r/#/c/13083/
baohua (Sat, 02 Sep 2017 01:05:46 GMT):
Thanks, @tongli . Would u like to post ur info to the developer list? https://wiki.hyperledger.org/projects/cello. And we also appreciate the contribution of sharing adoption cases.
JoseEdo (Sat, 02 Sep 2017 08:02:41 GMT):
Has joined the channel.
Cipezer (Sat, 02 Sep 2017 21:50:52 GMT):
Has joined the channel.
waaagh (Sun, 03 Sep 2017 01:34:58 GMT):
@baohua @hightall Seems https://jira.hyperledger.org/browse/CE-119 is still in TODO stage, i will try to create to pr to fix it :)
baohua (Sun, 03 Sep 2017 07:10:41 GMT):
thanks @waaagh, and you can feel free to ping @hightall , too
baohua (Sun, 03 Sep 2017 07:11:27 GMT):
the plan is that we want users to specify the host type when creating a new one.
DinhHo (Sun, 03 Sep 2017 14:30:11 GMT):
Has joined the channel.
waaagh (Mon, 04 Sep 2017 00:41:35 GMT):
Fine, is there a jira ticket for this plan?
waaagh (Mon, 04 Sep 2017 00:41:44 GMT):
@baohua
baohua (Mon, 04 Sep 2017 01:38:42 GMT):
can directly use https://jira.hyperledger.org/browse/CE-119
waaagh (Mon, 04 Sep 2017 15:56:32 GMT):
pushed a patch set for CE-119, plz review it @here
baohua (Tue, 05 Sep 2017 01:24:00 GMT):
thanks, and usually we also post the gerrit link here and add to jira task comments @waaagh :)
waaagh (Tue, 05 Sep 2017 01:24:55 GMT):
@baohua gotcha
waaagh (Tue, 05 Sep 2017 01:25:09 GMT):
https://gerrit.hyperledger.org/r/#/c/13127/
waaagh (Tue, 05 Sep 2017 01:34:39 GMT):
next will try to patch dashboard so that user can specify host_type through UI
waaagh (Tue, 05 Sep 2017 01:35:28 GMT):
@hightall plz let me know any of ur ideas
baohua (Tue, 05 Sep 2017 03:05:47 GMT):
exactly, we can add some pull-down box
baohua (Tue, 05 Sep 2017 03:09:03 GMT):
https://gerrit.hyperledger.org/r/#/c/13083/
hightall (Tue, 05 Sep 2017 15:18:05 GMT):
@waaagh ok,so you will add host_type into ui in the next patch. got it.I will review again after you add the test case.
SimonOberzan (Wed, 06 Sep 2017 10:13:17 GMT):
Has joined the channel.
george.hant (Wed, 06 Sep 2017 13:22:44 GMT):
Has joined the channel.
waaagh (Wed, 06 Sep 2017 13:39:28 GMT):
@hightall test case committed, plz take a few seconds to review.
waaagh (Wed, 06 Sep 2017 13:47:26 GMT):
seems CI job failed, try to fix
waaagh (Wed, 06 Sep 2017 14:38:51 GMT):
CI build passed
tongli (Wed, 06 Sep 2017 16:54:10 GMT):
@here guys, this patch https://gerrit.hyperledger.org/r/#/c/13199/ moves the default to 1.0.2
tongli (Wed, 06 Sep 2017 17:40:06 GMT):
Please review. Thanks a lot.
baohua (Thu, 07 Sep 2017 01:04:23 GMT):
@tongli pls follow the http://cello.readthedocs.io/en/latest/CONTRIBUTING/, and create a jira task for it. thanks.
tongli (Thu, 07 Sep 2017 01:37:47 GMT):
@baohua will do.
baohua (Thu, 07 Sep 2017 02:23:38 GMT):
@waaagh Currently there is no host-type specified from the dashboard, hence it may break the operation if we do not set at FE or use default value? https://gerrit.hyperledger.org/r/#/c/13127
waaagh (Thu, 07 Sep 2017 02:26:53 GMT):
@baohua if no host_type specified at FE, we will try to detect the type via docker api, not the default value
waaagh (Thu, 07 Sep 2017 02:27:46 GMT):
there's also a test case cover this case
baohua (Thu, 07 Sep 2017 02:28:18 GMT):
Thanks, i will recheck, and maybe we can add some host_type choice at FE? Then can make sure there's value from it. how do u think? in another pr?
baohua (Thu, 07 Sep 2017 02:28:53 GMT):
And could u help add the license header at https://gerrit.hyperledger.org/r/#/c/13127/4/test/cases/test_host_create.py
waaagh (Thu, 07 Sep 2017 02:28:57 GMT):
Yes, as u mentioned yesterday, a pull-down box is a good idea
baohua (Thu, 07 Sep 2017 02:29:01 GMT):
since u create it :)
waaagh (Thu, 07 Sep 2017 02:30:09 GMT):
find, will do it later today
waaagh (Thu, 07 Sep 2017 02:30:12 GMT):
find/fine
waaagh (Thu, 07 Sep 2017 13:21:50 GMT):
@baohua a license header already be added :)
tongli (Thu, 07 Sep 2017 15:51:18 GMT):
@here please review this patch so that we move to 1.0.2 by default
tongli (Thu, 07 Sep 2017 15:51:19 GMT):
https://gerrit.hyperledger.org/r/#/c/13199/
baohua (Fri, 08 Sep 2017 02:42:19 GMT):
done, and let's see other's review comments.
waaagh (Fri, 08 Sep 2017 03:19:45 GMT):
https://gerrit.hyperledger.org/r/#/c/13127/ @baohua pls take a few seconds to review, a license header already added
baohua (Fri, 08 Sep 2017 03:22:10 GMT):
cool, but i guess u need to specify ur company?
baohua (Fri, 08 Sep 2017 03:22:22 GMT):
Or just leave the field blank.
waaagh (Fri, 08 Sep 2017 03:24:39 GMT):
not fimilar with it, can u be more precise?
baohua (Fri, 08 Sep 2017 03:24:58 GMT):
# Copyright IBM Corp, All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#
baohua (Fri, 08 Sep 2017 03:25:07 GMT):
Copyright should be from ur team :)
baohua (Fri, 08 Sep 2017 03:25:42 GMT):
if u donot wanna keep copyright ,then may leave the line empty, i guess.
baohua (Fri, 08 Sep 2017 03:25:50 GMT):
Need confirm with @tkuhrt
baohua (Fri, 08 Sep 2017 03:26:23 GMT):
i mean the 1st line
baohua (Fri, 08 Sep 2017 03:26:31 GMT):
```# Copyright IBM Corp, All Rights Reserved.```
waaagh (Fri, 08 Sep 2017 03:27:39 GMT):
for now my development is independent, so the header will be:
```
#
#
# SPDX-License-Identifier: Apache-2.0
#
```
right?
baohua (Fri, 08 Sep 2017 03:27:59 GMT):
i guess this may work, pls confirm with @tkuhrt
waaagh (Fri, 08 Sep 2017 03:30:52 GMT):
ok, already pm him
waaagh (Fri, 08 Sep 2017 11:52:20 GMT):
added a new license header with @baohua, @tkuhrt gives the confirmation :)
VoR0220 (Fri, 08 Sep 2017 13:23:43 GMT):
Has joined the channel.
tongli (Fri, 08 Sep 2017 15:21:50 GMT):
@hightall can you take a look at this patch set , https://gerrit.hyperledger.org/r/#/c/13199/ thanks.
TCosley1 (Fri, 08 Sep 2017 21:36:18 GMT):
Has joined the channel.
baohua (Sat, 09 Sep 2017 00:51:49 GMT):
Congrats to @waaagh for his 1st merged patchset in Hyperleger Cello: https://jira.hyperledger.org/browse/CE-119
baohua (Sat, 09 Sep 2017 08:39:41 GMT):
https://gerrit.hyperledger.org/r/#/c/13299/
robert.feng (Sun, 10 Sep 2017 03:04:15 GMT):
Has joined the channel.
soccerhaotian (Sun, 10 Sep 2017 12:34:24 GMT):
Has joined the channel.
yoyokeen (Mon, 11 Sep 2017 01:12:51 GMT):
Has joined the channel.
AuHuR (Mon, 11 Sep 2017 06:52:29 GMT):
Has joined the channel.
sk (Mon, 11 Sep 2017 07:43:26 GMT):
@All, I have already set up Kubernets Cluster which is running Fabric 1.0. Now, I want to set up and run Cello on it...from where can I get Cello manifest file for Kubernets? Please help...I searched a lot but could not find it.
yoyokeen (Mon, 11 Sep 2017 08:46:58 GMT):
@here,I'm a new one on Cello.
yoyokeen (Mon, 11 Sep 2017 08:48:27 GMT):
now,I want to use Cello on my blockchain platform (fabric-e2e)
yoyokeen (Mon, 11 Sep 2017 08:49:26 GMT):
just use the monitor and operation function.
yoyokeen (Mon, 11 Sep 2017 08:50:01 GMT):
how to install and config the Cello?
sk (Mon, 11 Sep 2017 08:50:11 GMT):
Same question I have :)
sk (Mon, 11 Sep 2017 08:50:11 GMT):
Same question I have also :)
baohua (Mon, 11 Sep 2017 09:31:47 GMT):
@yoyokeen @sk it should be easy to find the doc at http://cello.readthedocs.io/en/latest/
yoyokeen (Mon, 11 Sep 2017 09:32:03 GMT):
seeing now
yoyokeen (Mon, 11 Sep 2017 09:32:22 GMT):
thx.@baohua
sk (Mon, 11 Sep 2017 09:42:16 GMT):
@baohua thanks. I looked it earlier but it does not say anything about how to set up Cello with Kubernets cluster
sk (Mon, 11 Sep 2017 09:42:16 GMT):
@baohua thanks. I looked it earlier and it doe not say anything about how to set up Cell with Kubernets cluster
sk (Mon, 11 Sep 2017 09:42:16 GMT):
@baohua thanks. I looked it earlier but it does not say anything about how to set up Cell with Kubernets cluster
sk (Mon, 11 Sep 2017 09:43:35 GMT):
While IBM is providing Kubernets + Fabric on Bluemix
sk (Mon, 11 Sep 2017 09:43:35 GMT):
While IBM is recommending Kubernets + Fabric on Bluemix
chawlanikhil24 (Mon, 11 Sep 2017 10:13:02 GMT):
@sk , For now, Cello project has manifests and scripts to setup fabric on Kubernetes
chawlanikhil24 (Mon, 11 Sep 2017 10:13:45 GMT):
but the Dashboard Control is still under development, there are mannually deployable manifests and Ansible scripts as welll
sk (Mon, 11 Sep 2017 10:15:58 GMT):
Thanks @chawlanikhil24, so can I install Cello manually using manifests on Bluemix Kubernets Cluster? Is this correct understanding?
chawlanikhil24 (Mon, 11 Sep 2017 10:16:37 GMT):
@sk, for now, you can't control Kuberenetes from cello
sk (Mon, 11 Sep 2017 10:17:22 GMT):
Oops! any other way to have dashboard/management of blockchain on Bluemix Kubernets Cluster?
sk (Mon, 11 Sep 2017 10:18:03 GMT):
I have already acquired Bluemix Kubernets Cluster
chawlanikhil24 (Mon, 11 Sep 2017 10:18:09 GMT):
for now!
chawlanikhil24 (Mon, 11 Sep 2017 10:18:20 GMT):
I am working on the dashboard connectionsd
sk (Mon, 11 Sep 2017 10:20:14 GMT):
ok, then what other options w/o using Bluemix Kubernets Cluster, to have dashboard/management of blockchain + Fabric?
sk (Mon, 11 Sep 2017 10:20:23 GMT):
on Cloud
chawlanikhil24 (Mon, 11 Sep 2017 11:31:07 GMT):
Sorry I don't know about that
chawlanikhil24 (Mon, 11 Sep 2017 11:31:31 GMT):
for now I am limited to Cello , Fabric and Kubernetes
sk (Mon, 11 Sep 2017 11:36:02 GMT):
humm...interesting...having Fabric + Container ready but now unable to do anything with it
erNail (Mon, 11 Sep 2017 11:53:16 GMT):
Has joined the channel.
erNail (Mon, 11 Sep 2017 11:54:25 GMT):
Hi, i've just installed cello and i'm trying to create a host via the dashboard. However, i get a "Could not create host" error message. Any idea how i can fix it ?
erNail (Mon, 11 Sep 2017 12:01:06 GMT):
Is it even possible to use the master_node as host ?
rubi_1432 (Mon, 11 Sep 2017 12:05:14 GMT):
Has joined the channel.
rubi_1432 (Mon, 11 Sep 2017 12:07:51 GMT):
Hi everyone, can i use cello with Hyperledger fabric to move from virtual to physical machines ?
tongli (Mon, 11 Sep 2017 12:22:56 GMT):
@rubi_1432 you want to do a relocation without losing the chain, I assume , is that right?
tongli (Mon, 11 Sep 2017 12:25:06 GMT):
In theory , you could add more peers from physical nodes, then run for awhile so that new nodes will establish the chains, once new nodes catch up, you could shutdown the virtual node
tongli (Mon, 11 Sep 2017 12:25:33 GMT):
I do not think you can do a direct copy sort of move.
indirajith (Mon, 11 Sep 2017 12:38:19 GMT):
Does anyone encounter a problem with rest server? Especially CE-115 commit seems to be causing this problem.
SimonOberzan (Mon, 11 Sep 2017 13:16:48 GMT):
Hi, I'm trying to create a cluster on a swarm host. I get the message: `The Docker Engine you're using is running in swarm mode.
Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.
To deploy your application across the swarm, use `docker stack deploy`.
ERROR: for peer0.org1.example.com Cannot start service peer0.org1.example.com: b'Could not attach to network cello_net_solo: rpc error: code = 7 desc = network cello_net_solo not manually attachable'`
SimonOberzan (Mon, 11 Sep 2017 13:16:48 GMT):
Hi, I'm trying to create a cluster on a swarm host. I get the message: `The Docker Engine you're using is running in swarm mode.
Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.
To deploy your application across the swarm, use `docker stack deploy`.
ERROR: for peer0.org1.example.com Cannot start service peer0.org1.example.com: b'Could not attach to network cello_net_solo: rpc error: code = 7 desc = network cello_net_solo not manually attachable'`
Help much appreciated
SimonOberzan (Mon, 11 Sep 2017 13:16:48 GMT):
Hi, I'm trying to create a cluster on a swarm host. I get the message: `The Docker Engine you're using is running in swarm mode.\
Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.
To deploy your application across the swarm, use `docker stack deploy`.
ERROR: for peer0.org1.example.com Cannot start service peer0.org1.example.com: b'Could not attach to network cello_net_solo: rpc error: code = 7 desc = network cello_net_solo not manually attachable'`
Help much appreciated
SimonOberzan (Mon, 11 Sep 2017 13:16:48 GMT):
Hi, I'm trying to create a cluster on a swarm host. I get the message: `The Docker Engine you're using is running in swarm mode.`
Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.
To deploy your application across the swarm, use `docker stack deploy`.
ERROR: for peer0.org1.example.com Cannot start service peer0.org1.example.com: b'Could not attach to network cello_net_solo: rpc error: code = 7 desc = network cello_net_solo not manually attachable'`
Help much appreciated
SimonOberzan (Mon, 11 Sep 2017 13:16:48 GMT):
Hi, I'm trying to create a cluster on a swarm host. I get the message: `The Docker Engine you're using is running in swarm mode.`
`Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.`
`To deploy your application across the swarm, use `docker stack deploy`.`
`ERROR: for peer0.org1.example.com Cannot start service peer0.org1.example.com: b'Could not attach to network cello_net_solo: rpc error: code = 7 desc = network cello_net_solo not manually attachable'``
Help much appreciated
SimonOberzan (Mon, 11 Sep 2017 13:16:48 GMT):
Hi, I'm trying to create a cluster on a swarm host. I get the message: `The Docker Engine you're using is running in swarm mode.`
`Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.`
`To deploy your application across the swarm, use `docker stack deploy`.`
`ERROR: for peer0.org1.example.com Cannot start service peer0.org1.example.com: b'Could not attach to network cello_net_solo: rpc error: code = 7 desc = network cello_net_solo not manually attachable'``
Any idea how to fix it?
SimonOberzan (Mon, 11 Sep 2017 13:16:48 GMT):
Hi, I'm trying to create a cluster on a swarm host. I get the message: `The Docker Engine you're using is running in swarm mode.`
`Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.`
`To deploy your application across the swarm, use `docker stack deploy`.`
`ERROR: for peer0.org1.example.com Cannot start service peer0.org1.example.com: b'Could not attach to network cello_net_solo: rpc error: code = 7 desc = network cello_net_solo not manually attachable'``
Any idea how to fix it?
baohua (Mon, 11 Sep 2017 13:50:50 GMT):
@SimonOberzan could u help create a jira bug at jira.hyperledger.org, and provide the env info (e.g., version) and steps to repeat? Thanks
baohua (Mon, 11 Sep 2017 13:52:38 GMT):
i guess it might be u're using very latest version of docker. we mostly recommend to use 1.13. The new version of docker have behaviors changed.
baohua (Mon, 11 Sep 2017 13:53:17 GMT):
And, pls use `docker network ls` to check if the cello_net_solo network exists.
yoyokeen (Mon, 11 Sep 2017 14:11:31 GMT):
http://112.126.87.91:8080/login
yoyokeen (Mon, 11 Sep 2017 14:12:21 GMT):
http://localhost:8080/login,what's the username&passwd
rubi_1432 (Mon, 11 Sep 2017 14:13:33 GMT):
@tongli Yes, i want to add peers from physical node. But i can't figure out how to do it. I am starting with connecting physical machines and then creating a channel over it using *Build your first network* tutorial. Is this the right direction ?
tongli (Mon, 11 Sep 2017 14:14:39 GMT):
@rubi_1432 I thought you already have a fabric network running, then you want to move to physical servers.
tongli (Mon, 11 Sep 2017 14:15:18 GMT):
if this is brand new env, then following the instruction should have worked since cello did not care if your node is virtual or physical
tongli (Mon, 11 Sep 2017 14:15:52 GMT):
or you can utilize what I recently contributed to setup multi-node fabric network by using ansible.
tongli (Mon, 11 Sep 2017 14:16:01 GMT):
which is at src/agent/ansible.
tongli (Mon, 11 Sep 2017 14:16:28 GMT):
look into a doc at src/agent/ansible/vars/README.md file for dealing with physical servers.
tongli (Mon, 11 Sep 2017 14:17:17 GMT):
if you need some help using the ansible script, I can certainly help out.
tongli (Mon, 11 Sep 2017 14:17:56 GMT):
at the end, you will be able to layout a fabric network on your physical server very easily.
rubi_1432 (Mon, 11 Sep 2017 14:20:07 GMT):
@tongli Oh okay. Thank you so much. I will definitely look into and will seek help from you. Thanks a lot :-)
tongli (Mon, 11 Sep 2017 14:20:31 GMT):
sure.
baohua (Mon, 11 Sep 2017 14:43:40 GMT):
@yoyokeen it's `admin:pass` as indicated in http://cello.readthedocs.io/en/latest/scenario/
baohua (Mon, 11 Sep 2017 14:52:42 GMT):
https://gerrit.hyperledger.org/r/#/c/13267
hightall (Tue, 12 Sep 2017 01:45:58 GMT):
Congrats @HenryZhang for his 1st merged patchset on Hyperledger Cello. https://gerrit.hyperledger.org/r/#/c/13267/
yoyokeen (Tue, 12 Sep 2017 01:47:04 GMT):
got it.@baohua
yoyokeen (Tue, 12 Sep 2017 01:47:46 GMT):
the user-dashboard isnt open?
yoyokeen (Tue, 12 Sep 2017 01:47:59 GMT):
I cant login.
HenryZhang (Tue, 12 Sep 2017 01:49:08 GMT):
@hightall Thanks for your review. We will continue with more code soon.
yoyokeen (Tue, 12 Sep 2017 02:23:58 GMT):
what kind of people can join Cell? nodejs and others?
baohua (Tue, 12 Sep 2017 03:52:02 GMT):
@yoyokeen yeap, we didn't connect the user-dashboard now. will evaluate which solution is better.
baohua (Tue, 12 Sep 2017 03:52:42 GMT):
Cello is open to everyone, the backend is mostly written in python, while there are lots of code in javascript, css and html too.
baohua (Tue, 12 Sep 2017 03:53:13 GMT):
pls feel free to ping @hightall if you feel interested with the dashboard component. @yoyokeen
yoyokeen (Tue, 12 Sep 2017 03:53:39 GMT):
blockchain-explorer is alive?
yoyokeen (Tue, 12 Sep 2017 03:53:46 GMT):
@baohua
baohua (Tue, 12 Sep 2017 03:54:15 GMT):
yes, i believe so. you can feel free to reach the projects' maintainers.
baohua (Tue, 12 Sep 2017 03:54:29 GMT):
can find the maintainers.md at the git repo
baohua (Tue, 12 Sep 2017 03:54:35 GMT):
@HenryZhang Congrats to the vmware team, look forward to seeing more contributions!
yoyokeen (Tue, 12 Sep 2017 03:54:53 GMT):
there are some difference from blockchain-explorer and user-dashboard(cello)
baohua (Tue, 12 Sep 2017 03:58:34 GMT):
yes, there is
baohua (Tue, 12 Sep 2017 03:58:55 GMT):
maybe can take as a quick workaround until our user dashboard is done.
yoyokeen (Tue, 12 Sep 2017 05:28:00 GMT):
The user dashboard will replace the blockchain-explorer or the blockchain-explorer will been integrated by user dashboard.
AuHuR (Tue, 12 Sep 2017 08:00:08 GMT):
Hey folks, I'm unable to login to the dashboard. I'm getting a 401 error when I try to auth. I already double checked the user and password. Also changed the credentials in the docker-compose file to look if there is an error but I'm still getting the 401 error.
sk (Tue, 12 Sep 2017 08:47:48 GMT):
can we setup worker node on Mac OS as this (http://cello.readthedocs.io/en/latest/installation_worker_docker/) instructions are for mainly Ubuntu and Linux distros
sk (Tue, 12 Sep 2017 08:47:48 GMT):
can we setup worker node on Mac OS as this (http://cello.readthedocs.io/en/latest/installation_worker_docker/) instructions are for mainly for Ubuntu and Linux distros
erNail (Tue, 12 Sep 2017 09:16:28 GMT):
Is it possible to run master node and worker node on one machine ?
colinGrahms (Tue, 12 Sep 2017 09:51:00 GMT):
Hi, what the default id/password in order to log into cello ?
AuHuR (Tue, 12 Sep 2017 09:51:21 GMT):
admin/pass
AuHuR (Tue, 12 Sep 2017 09:51:21 GMT):
admin/pass (http://cello.readthedocs.io/en/latest/scenario)
colinGrahms (Tue, 12 Sep 2017 09:51:38 GMT):
Thanks !
ArvsIndrarys (Tue, 12 Sep 2017 10:04:58 GMT):
Has joined the channel.
ArvsIndrarys (Tue, 12 Sep 2017 10:06:26 GMT):
Hello guys !
I was wondering if Cello was able to simply connect to a fabric instance instead of creating it?
we already have fabric deployed and are considering adding cello to it. Is it possible to integrate it or should we start from scratch via Cello?
baohua (Tue, 12 Sep 2017 10:57:00 GMT):
@yoyokeen there're different, so no such kind of plan.
baohua (Tue, 12 Sep 2017 10:57:18 GMT):
@AuHuR can u check the logs of the containers on the master node?
baohua (Tue, 12 Sep 2017 10:58:11 GMT):
@sk mostly they are the same, except the docker installation part. after that, it's safe to run `make start`. and remember to create the db dir manually and mount to docker at MacOS.
baohua (Tue, 12 Sep 2017 10:58:48 GMT):
@ArvsIndrarys currently cello deploy fabric, instead of connecting to one. you may try the blockchain-explorer project.
sk (Tue, 12 Sep 2017 10:59:07 GMT):
Thanks @baohua mainly facing issues to find Docker config file /etc/default/docker on Mac OS
sk (Tue, 12 Sep 2017 10:59:28 GMT):
there is no Docker config file /etc/default/docker on Mac OS its only exist on Ubuntu/Linux
baohua (Tue, 12 Sep 2017 11:00:38 GMT):
no worry about it, as at MacOS, things are different. just follow the official steps to setup Docker.
baohua (Tue, 12 Sep 2017 11:01:07 GMT):
and remember to create and mount the db path into docker. and change its privilege to ur user role.
ArvsIndrarys (Tue, 12 Sep 2017 11:01:10 GMT):
@baohua thanks ! I'll look into it
baohua (Tue, 12 Sep 2017 11:01:34 GMT):
sure, and welcome to help add some docs to describe the steps on use Cello at MacOS.
baohua (Tue, 12 Sep 2017 11:01:44 GMT):
http://cello.readthedocs.io/en/latest/CONTRIBUTING/
baohua (Tue, 12 Sep 2017 11:01:44 GMT):
http://cello.readthedocs.io/en/latest/CONTRIBUTING/
baohua (Tue, 12 Sep 2017 11:10:46 GMT):
erNail (Tue, 12 Sep 2017 11:29:47 GMT):
Hi, i'm not able to delete a chain created in the dashboard. I get a "Failed! Failed to delete cluster 59b7bf44292d370011fa610e"
baohua (Tue, 12 Sep 2017 11:36:40 GMT):
pls check if there's any error in the log `make logs`
erNail (Tue, 12 Sep 2017 11:41:44 GMT):
dashboard | [2017-09-12 19:40:21,434] ERROR [agent.docker.docker_swarm] [docker_swarm.py:431 compose_clean()] - Error in stop compose project, will clean
dashboard | [2017-09-12 19:40:21,434] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:432 compose_clean()] - .FileNotFoundError: [Errno 2] No such file or directory: './agent/docker/_compose_files/fabric-1.0/local/fabric-kafka-4.yaml'
dashboard | [2017-09-12 19:40:21,443] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:75 _clean_project_containers()] - Clean project containers, worker_api=tcp://10.10.10.117:2375, prefix=59b7bf44292d370011fa610e
dashboard | [2017-09-12 19:40:21,465] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:53 _clean_chaincode_images()] - clean chaincode images with prefix=59b7bf44292d370011fa610e
dashboard | [2017-09-12 19:40:21,505] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:58 _clean_chaincode_images()] - chaincode image id to removes=
dashboard | [2017-09-12 19:40:21,506] WARNING [agent.docker.docker_swarm] [docker_swarm.py:447 compose_clean()] - Exception when cleaning project 59b7bf44292d370011fa610e
dashboard | [2017-09-12 19:40:21,507] WARNING [modules.cluster] [cluster.py:294 delete()] - Error to run compose clean work
dashboard | [2017-09-12 19:40:21,509] WARNING [resources.cluster_api] [cluster_api.py:290 cluster_delete()] - Failed to delete cluster 59b7bf44292d370011fa610e
dashboard | 172.18.0.6 - - [12/Sep/2017 19:40:21] "DELETE /api/cluster HTTP/1.0" 400 -
These is the output of make logs
erNail (Tue, 12 Sep 2017 11:46:01 GMT):
Also i'm not able to stop a running chain. I'm getting this output:
dashboard | [2017-09-12 19:45:28,570] DEBUG [resources.cluster_api] [utils.py:97 request_debug()] - path=/api/cluster_op, method=POST
dashboard | [2017-09-12 19:45:28,571] DEBUG [resources.cluster_api] [utils.py:98 request_debug()] - request args:
dashboard | [2017-09-12 19:45:28,575] DEBUG [resources.cluster_api] [utils.py:101 request_debug()] - request form:
dashboard | [2017-09-12 19:45:28,577] DEBUG [resources.cluster_api] [utils.py:103 request_debug()] - Form: cluster_id:59b7bf44292d370011fa610e
dashboard | [2017-09-12 19:45:28,579] DEBUG [resources.cluster_api] [utils.py:103 request_debug()] - Form: action:stop
dashboard | [2017-09-12 19:45:28,587] DEBUG [resources.cluster_api] [utils.py:104 request_debug()] - request raw body data:
dashboard | [2017-09-12 19:45:28,588] DEBUG [resources.cluster_api] [utils.py:105 request_debug()] - b''
dashboard | [2017-09-12 19:45:28,590] DEBUG [resources.cluster_api] [utils.py:106 request_debug()] - None
dashboard | [2017-09-12 19:45:28,590] INFO [resources.cluster_api] [cluster_api.py:163 cluster_actions()] - cluster_op with action=stop
dashboard | [2017-09-12 19:45:28,593] DEBUG [modules.host] [host.py:329 get_active_host_by_id()] - check host with id = 59b7a4d2292d370011fa6102
dashboard | [2017-09-12 19:45:28,599] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:541 compose_stop()] - Compose stop 59b7bf44292d370011fa610e with worker_api=tcp://10.10.10.117:2375, mapped_ports={'ecap': 7054, 'tlscaa': 7059, 'grpc': 7051, 'cli': 7052, 'tlscap': 7058, 'ecaa': 7055, 'tcaa': 7057, 'rest': 7050, 'event': 7053, 'tcap': 7056}, config={'network_type': 'fabric-1.0', 'size': 4, 'consensus_plugin': 'kafka'}, log_type=local
dashboard | 172.18.0.6 - - [12/Sep/2017 19:45:28] "POST /api/cluster_op HTTP/1.0" 500 -
baohua (Tue, 12 Sep 2017 13:11:41 GMT):
FileNotFoundError: [Errno 2] No such file or directory: './agent/docker/_compose_files/fabric-1.0/local/fabric-kafka-4.yaml'
baohua (Tue, 12 Sep 2017 13:12:15 GMT):
maybe the problem is to use kafka, can u try solo?
AuHuR (Tue, 12 Sep 2017 16:23:52 GMT):
[ ](https://chat.hyperledger.org/channel/cello?msg=KFw2osKgnkt5ZyG3K) @baohua The problem evolves when you use react as the view controller, as soon as I stoped using it, I was able to login
baohua (Tue, 12 Sep 2017 23:31:59 GMT):
@AuHuR thanks. Yes, since react has license problem, we now use the basic theme by default. @hightall could u help confirm?
hightall (Tue, 12 Sep 2017 23:36:31 GMT):
@AuHuR yes,now we stopped dev on react theme,and will rewrite basic theme with vue framework. So will you contribute code for the UI?:grinning:
erNail (Wed, 13 Sep 2017 05:38:10 GMT):
@baohua When i try to start a solo-chain, i get another error. This is the output:
dashboard | [2017-09-13 13:35:03,203] INFO [resources.cluster_api] [cluster_api.py:215 cluster_create()] - /cluster action=POST
dashboard | [2017-09-13 13:35:03,203] DEBUG [resources.cluster_api] [utils.py:97 request_debug()] - path=/api/cluster, method=POST
dashboard | [2017-09-13 13:35:03,205] DEBUG [resources.cluster_api] [utils.py:98 request_debug()] - request args:
dashboard | [2017-09-13 13:35:03,208] DEBUG [resources.cluster_api] [utils.py:101 request_debug()] - request form:
dashboard | [2017-09-13 13:35:03,214] DEBUG [resources.cluster_api] [utils.py:103 request_debug()] - Form: name:new_chain
dashboard | [2017-09-13 13:35:03,216] DEBUG [resources.cluster_api] [utils.py:103 request_debug()] - Form: consensus_mode:batch
dashboard | [2017-09-13 13:35:03,216] DEBUG [resources.cluster_api] [utils.py:103 request_debug()] - Form: consensus_plugin:solo
dashboard | [2017-09-13 13:35:03,216] DEBUG [resources.cluster_api] [utils.py:103 request_debug()] - Form: host_id:59b7a4d2292d370011fa6102
dashboard | [2017-09-13 13:35:03,216] DEBUG [resources.cluster_api] [utils.py:103 request_debug()] - Form: network_type:fabric-1.0
dashboard | [2017-09-13 13:35:03,216] DEBUG [resources.cluster_api] [utils.py:103 request_debug()] - Form: size:4
dashboard | [2017-09-13 13:35:03,216] DEBUG [resources.cluster_api] [utils.py:104 request_debug()] - request raw body data:
dashboard | [2017-09-13 13:35:03,217] DEBUG [resources.cluster_api] [utils.py:105 request_debug()] - b''
dashboard | [2017-09-13 13:35:03,217] DEBUG [resources.cluster_api] [utils.py:106 request_debug()] - None
dashboard | [2017-09-13 13:35:03,217] INFO [modules.cluster] [cluster.py:111 create()] - Create cluster new_chain, host_id=59b7a4d2292d370011fa6102, config={'consensus_plugin': 'solo', 'network_type': 'fabric-1.0', 'size': 4}, start_port=0, user_id=
erNail (Wed, 13 Sep 2017 05:38:20 GMT):
dashboard | [2017-09-13 13:35:03,217] DEBUG [modules.host] [host.py:329 get_active_host_by_id()] - check host with id = 59b7a4d2292d370011fa6102
dashboard | [2017-09-13 13:35:03,218] DEBUG [modules.cluster] [cluster.py:122 create()] - worker_api=tcp://10.10.10.117:2375
dashboard | [2017-09-13 13:35:03,218] DEBUG [modules.cluster] [cluster.py:641 find_free_start_ports()] - Find 1 start ports for host 59b7a4d2292d370011fa6102
dashboard | [2017-09-13 13:35:03,228] DEBUG [modules.cluster] [cluster.py:656 find_free_start_ports()] - The ports existed: []
dashboard | [2017-09-13 13:35:03,233] DEBUG [modules.cluster] [cluster.py:665 find_free_start_ports()] - Free ports are [7050]
dashboard | [2017-09-13 13:35:03,234] DEBUG [modules.cluster] [cluster.py:139 create()] - mapped_ports={'tlscap': 7058, 'event': 7053, 'ecaa': 7055, 'cli': 7052, 'tlscaa': 7059, 'rest': 7050, 'tcaa': 7057, 'tcap': 7056, 'grpc': 7051, 'ecap': 7054}
dashboard | [2017-09-13 13:35:03,248] DEBUG [modules.cluster] [cluster.py:185 create()] - Start compose project with name=59b8c387292d3700117d5a6f
dashboard | [2017-09-13 13:35:03,249] DEBUG [agent.docker.cluster] [cluster.py:53 create()] - Start compose project with name=59b8c387292d3700117d5a6f
dashboard | [2017-09-13 13:35:03,249] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:390 compose_up()] - Compose start: name=59b8c387292d3700117d5a6f, host=vagr_117, mapped_port={'tlscap': 7058, 'event': 7053, 'ecaa': 7055, 'cli': 7052, 'tlscaa': 7059, 'rest': 7050, 'tcaa': 7057, 'tcap': 7056, 'grpc': 7051, 'ecap': 7054},config={'consensus_plugin': 'solo', 'network_type': 'fabric-1.0', 'size': 4}
dashboard | [2017-09-13 13:35:05,393] WARNING [agent.docker.docker_swarm] [docker_swarm.py:406 compose_up()] - Exception when compose start=404 Client Error: Not Found ("b'manifest for hyperledger/fabric-peer:latest not found'")
dashboard | [2017-09-13 13:35:05,398] WARNING [agent.docker.cluster] [cluster.py:58 create()] - failed to create cluster, with container={}
dashboard | [2017-09-13 13:35:05,398] WARNING [modules.cluster] [cluster.py:190 create()] - failed to start cluster=new_chain, then delete
dashboard | [2017-09-13 13:35:05,406] DEBUG [modules.cluster] [cluster.py:244 delete()] - Delete cluster: id=59b8c387292d3700117d5a6f, forced=True
dashboard | [2017-09-13 13:35:05,422] DEBUG [modules.host] [host.py:329 get_active_host_by_id()] - check host with id = 59b7a4d2292d370011fa6102
dashboard | [2017-09-13 13:35:05,429] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:573 compose_down()] - Compose remove 59b8c387292d3700117d5a6f with worker_api=tcp://10.10.10.117:2375, config={'consensus_plugin': 'solo', 'network_type': 'fabric-1.0', 'size': 4}
dashboard | [2017-09-13 13:35:05,433] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:579 compose_down()] - environ({'ECAA_PORT': '7055', 'TLSCAA_PORT': '7059', 'COMPOSE_FILE': 'fabric-solo-4.yaml', 'LOG_LEVEL': 'DEBUG', 'MONGO_URL': 'mongodb://mongo:27017', 'HOME': '/root', 'TEMPLATE_FOLDER': 'themes/basic/templates', 'MONGO_DB': 'dev', 'EVENT_PORT': '7053', 'PYTHON_PIP_VERSION': '9.0.1', 'CLUSTER_LOG_LEVEL': 'DEBUG', 'TCAA_PORT': '7057', 'STATIC_FOLDER': 'themes/basic/static', 'PATH': '/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'LANG': 'C.UTF-8', 'CLI_PORT': '7052', 'PYTHON_VERSION': '3.5.4', 'CLUSTER_NETWORK': 'cello_net_solo', 'GPG_KEY': '97FC712E4C024BBEA48A61ED3A5CA953F73C700D', 'WERKZEUG_SERVER_FD': '7', 'VM_ENDPOINT': 'tcp://10.10.10.117:2375', 'REST_PORT': '7050', 'TLSCAP_PORT': '7058', 'TCAP_PORT': '7056', 'VM_DOCKER_HOSTCONFIG_NETWORKMODE': 'cello_net_solo', 'ECAP_PORT': '7054', 'COMPOSE_PROJECT_NAME': '59b8c387292d3700117d5a6f', 'WERKZEUG_RUN_MAIN': 'true', 'COMPOSE_PROJECT_PATH': '/opt/cello/fabric-1.0/local', 'GRPC_PORT': '7051', 'DOCKER_HOST': 'tcp://10.10.10.117:2375', 'NETWORK_TYPES': 'fabric-1.0', 'TZ': 'Asia/Shanghai', 'HOSTNAME': 'cello-dashboard', 'DEBUG': 'True', 'PEER_NETWORKID': '59b8c387292d3700117d5a6f'})
dashboard | [2017-09-13 13:35:05,627] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:75 _clean_project_containers()] - Clean project containers, worker_api=tcp://10.10.10.117:2375, prefix=59b8c387292d3700117d5a6f
dashboard | [2017-09-13 13:35:05,640] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:53 _clean_chaincode_images()] - clean chaincode images with prefix=59b8c387292d3700117d5a6f
dashboard | [2017-09-13 13:35:05,688] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:58 _clean_chaincode_images()] - chaincode image id to removes=
dashboard | [2017-09-13 13:35:05,690] DEBUG [resources.cluster_api] [cluster_api.py:249 cluster_create()] - cluster creation failed using handlder
erNail (Wed, 13 Sep 2017 05:38:24 GMT):
dashboard | 172.18.0.5 - - [13/Sep/2017 13:35:05] "POST /api/cluster HTTP/1.0" 400 -
dashboard | [2017-09-13 13:35:07,743] DEBUG [resources.cluster_view] [utils.py:97 request_debug()] - path=/view/clusters, method=GET
dashboard | [2017-09-13 13:35:07,743] DEBUG [resources.cluster_view] [utils.py:98 request_debug()] - request args:
dashboard | [2017-09-13 13:35:07,750] DEBUG [resources.cluster_view] [utils.py:100 request_debug()] - Arg: type:active
dashboard | [2017-09-13 13:35:07,750] DEBUG [resources.cluster_view] [utils.py:101 request_debug()] - request form:
dashboard | [2017-09-13 13:35:07,751] DEBUG [resources.cluster_view] [utils.py:104 request_debug()] - request raw body data:
dashboard | [2017-09-13 13:35:07,751] DEBUG [resources.cluster_view] [utils.py:105 request_debug()] - b''
dashboard | [2017-09-13 13:35:07,751] DEBUG [resources.cluster_view] [utils.py:106 request_debug()] - None
dashboard | [2017-09-13 13:35:07,751] DEBUG [modules.cluster] [cluster.py:64 list()] - List all active clusters
dashboard | 172.18.0.5 - - [13/Sep/2017 13:35:07] "GET /view/clusters?type=active HTTP/1.0" 200 -
dashboard | 172.18.0.5 - - [13/Sep/2017 13:35:08] "GET /static/img/sort_both.png HTTP/1.0" 404 -
dashboard | 172.18.0.5 - - [13/Sep/2017 13:35:08] "GET /static/img/sort_asc.png HTTP/1.0" 404 -
baohua (Wed, 13 Sep 2017 05:40:41 GMT):
@erNail from the log, it indicates that u do not have the required images: [agent.docker.docker_swarm] [docker_swarm.py:406 compose_up()] - Exception when compose start=404 Client Error: Not Found ("b'manifest for hyperledger/fabric-peer:latest not found'")
baohua (Wed, 13 Sep 2017 05:40:51 GMT):
Would u like to follow our installation steps? http://cello.readthedocs.io/en/latest/installation/
erNail (Wed, 13 Sep 2017 07:04:55 GMT):
Okay, really stupid mistake, i didn't realise that i have to execute '$ make setup-worker' after running the docker deamon. It works now, thank you very much for the help :).
erNail (Wed, 13 Sep 2017 08:44:18 GMT):
Is there any way to manage running chains created via composer ? Or any way to deploy business networks to chains created via cello ?
ArvsIndrarys (Wed, 13 Sep 2017 10:14:29 GMT):
Has left the channel.
markparz (Wed, 13 Sep 2017 12:25:40 GMT):
@erNail right now no on the composer question, however I think it's coming soon, might want to ask when on the composer channel
erNail (Wed, 13 Sep 2017 12:26:34 GMT):
@markparz okay, thanks
waaagh (Thu, 14 Sep 2017 15:22:49 GMT):
hi https://jenkins.hyperledger.org/job/cello-verify-x86_64/257/console this CI job failed bcs there's no tox command in that box?
tongli (Thu, 14 Sep 2017 22:21:41 GMT):
@here, please review this patch set, https://gerrit.hyperledger.org/r/#/c/13489/ which added fabric-ca and few other cleanup work.
tongli (Thu, 14 Sep 2017 22:21:45 GMT):
thanks in advance.
waaagh (Fri, 15 Sep 2017 01:37:56 GMT):
```
14:24:42 + make check
14:24:42 tox
14:24:42 make: tox: Command not found
14:24:42 Makefile:55: recipe for target 'check' failed
14:24:42 make: *** [check] Error 127
14:24:42 Build step 'Execute shell' marked build as failure
```
@here hi, anyone can help me figure out what happened in CI job?
harsha (Fri, 15 Sep 2017 01:38:42 GMT):
Did you install tox ?
waaagh (Fri, 15 Sep 2017 01:39:13 GMT):
this happened in a jenkin job
harsha (Fri, 15 Sep 2017 01:39:49 GMT):
To me it seems tox isn't installed hence the issue.
waaagh (Fri, 15 Sep 2017 01:40:50 GMT):
yep, so it confuse me bcs our CI box should have this command
waaagh (Fri, 15 Sep 2017 01:41:45 GMT):
https://jenkins.hyperledger.org/job/cello-verify-x86_64/257/console
baohua (Fri, 15 Sep 2017 01:51:28 GMT):
@waaagh can u help contact rjones@linuxfoundation.org for the confirmation. he is responsible to the CI.
baohua (Fri, 15 Sep 2017 01:51:48 GMT):
His rktchat id should be @rjones
rjones (Fri, 15 Sep 2017 01:51:48 GMT):
Has joined the channel.
waaagh (Fri, 15 Sep 2017 01:52:29 GMT):
sure
leogzl (Fri, 15 Sep 2017 11:18:29 GMT):
Has joined the channel.
tongli (Fri, 15 Sep 2017 14:16:58 GMT):
@baohua, you asked me to provide a bit more details on what I suggested being an interface between UI and backend agent.
tongli (Fri, 15 Sep 2017 14:17:23 GMT):
Since I need to give you an example on what I meant, I thought that doing it here is probably a bit easier.
tongli (Fri, 15 Sep 2017 14:22:15 GMT):
The UI is a component which allows users to specify how nodes or fabric network should look like, it is a component gathers user request/intention, once the user request is collected, then that request gets sent to backend agent to perform actual work. So if we have a clear spec for user to describe their intention, then the UI can be developed any way a dashboard developer wants, the backend agents (fabric, sawtooth, Iroha, or whatever) can take that request, then perform the actual work accordingly.
tongli (Fri, 15 Sep 2017 14:22:43 GMT):
For ansible deployer agent, what I have is this.
tongli (Fri, 15 Sep 2017 14:22:50 GMT):
I believe that I have showed you.
tongli (Fri, 15 Sep 2017 14:23:26 GMT):
``` network: {
fabric001: {
cas: [],
peers: ["anchor@peer1st.orga", "anchor@peer1st.orgb"],
orderers: ["orderer1st.orgc", "orderer1st.orgd"],
zookeepers: ["zookeeper1st"],
kafkas: ["kafka1st"]
},
fabric002: {
cas: [],
peers: ["worker@peer2nd.orga", "worker@peer2nd.orgb"],
orderers: ["orderer2nd.orgc", "orderer2nd.orgd"],
zookeepers: ["zookeeper2nd"],
kafkas: ["kafka2nd"]
},
fabric003: {
cas: [],
peers: ["worker@peer3rd.orga", "worker@peer3rd.orgb"],
orderers: [],
zookeepers: ["zookeeper3rd"],
kafkas: ["kafka3rd"]
}```
tongli (Fri, 15 Sep 2017 14:24:48 GMT):
``` network: {
fabric001: {
cas: ["ca1st.orga"],
peers: ["anchor@peer1st.orga", "anchor@peer1st.orgb"],
orderers: ["orderer1st.orgc", "orderer1st.orgd"],
zookeepers: ["zookeeper1st"],
kafkas: ["kafka1st"]
},
fabric002: {
cas: ["cattt.orgb"],
peers: ["worker@peer2nd.orga", "worker@peer2nd.orgb"],
orderers: ["orderer2nd.orgc", "orderer2nd.orgd"],
zookeepers: ["zookeeper2nd"],
kafkas: ["kafka2nd"]
},
fabric003: {
cas: ["caxxx.orgc", "caxxx.orgd"],
peers: ["worker@peer3rd.orga", "worker@peer3rd.orgb"],
orderers: [],
zookeepers: ["zookeeper3rd"],
kafkas: ["kafka3rd"]
}```
tongli (Fri, 15 Sep 2017 14:26:35 GMT):
this type of file serves as a spec between UI and backend agent. regardless what backend you may have, a blockchain network can be specified as such. the spec can be quite different for different agents, but it will be a single yaml file.
tongli (Fri, 15 Sep 2017 14:27:05 GMT):
hope that helps.
yuanlv (Fri, 15 Sep 2017 21:45:36 GMT):
Has joined the channel.
rjones (Fri, 15 Sep 2017 23:40:55 GMT):
@waaagh @baohua fixed, I re-ran the two failed builds
baohua (Sat, 16 Sep 2017 02:43:20 GMT):
@rjones thanks!
baohua (Sat, 16 Sep 2017 02:46:03 GMT):
@tongli thanks for ur comments. Now i get ur idea. The dashboard discussed is actually for creating a new host. But i guess we can use some config files as u suggest to maintain the clusters we created. Currently we only suppose fixed size of network, this config file can be a good start for supporting use-defined blockchain network. One problem might be is it general enough for all blockchains (for fabric, it should be OK as adopted in those SDK projects already).
waaagh (Sat, 16 Sep 2017 06:25:46 GMT):
@hightall @baohua plz help review this patch, i add a pull-down box to let user chose the host_type https://gerrit.hyperledger.org/r/#/c/13461/
tongli (Mon, 18 Sep 2017 02:55:24 GMT):
@baohua for different type of block chains, that file can be different even in format. As long as the ui knows all it needs to do is to craft necessary field values from a file, the. The back end different agent will understand particular type of config file, then it is ok. For example, the ui for fabric may look quite different from the ui for sawtooth, but that is ok. The architecture is still intact. The server basically take ui fields from the dashboard component and pass to the corespondent agent. That flow won't change at all. With that, we can support any type of agent and ui.
tongli (Mon, 18 Sep 2017 02:57:42 GMT):
@baohua @hightall please review this patch and our ci is waiting for the patch. We are ready to switch the performance test to cello to setup nfabric network. @here, please review this patch set, https://gerrit.hyperledger.org/r/#/c/13489/ which added fabric-ca and few other cleanup work.
https://gerrit.hyperledger.org/r/#/c/13489/
tongli (Mon, 18 Sep 2017 02:58:34 GMT):
@baohua @hightall please review this patch. @here, please review this patch set, https://gerrit.hyperledger.org/r/#/c/13489/ which added fabric-ca and few other cleanup work.
https://gerrit.hyperledger.org/r/#/c/13489/
tongli (Mon, 18 Sep 2017 02:58:38 GMT):
thanks
baohua (Mon, 18 Sep 2017 05:49:34 GMT):
Add seeral review comments, thanks
baohua (Mon, 18 Sep 2017 05:50:39 GMT):
@waaagh would u like to see the comments in https://gerrit.hyperledger.org/r/#/c/13461/?
SimonOberzan (Mon, 18 Sep 2017 08:46:37 GMT):
Hi. If I add chain with default certificates it initializes fine, but if I generate new certificates from a given crypto-config.yaml create channel gets unexpected status BAD_REQUEST.
SimonOberzan (Mon, 18 Sep 2017 08:47:10 GMT):
Additional logs from orderer: 2017-09-18 08:39:18.681 UTC [msp/identity] newIdentity -> DEBU 1ae Creating identity instance for ID &{Org1MSP 21fba6df324ea4a2c978de34eee20fa9e89ff8f8efac9bcce6882b3ead095099}
2017-09-18 08:39:18.681 UTC [cauthdsl] func2 -> ERRO 1af Principal deserialization failure (The supplied identity is not valid, Verify() returned x509: certificate signed by unknown authority (possibly because of "x509: ECDSA verification failure" while trying to verify candidate authority certificate "ca.org1.example.com")) for identity 0a074f7267314d535012fc052d2d2d2d2d424547494e202d2d2d2d2d0a4d4949434744434341622b674177494241674951625a552b34534e427266364c32575930743637655a44414b42676771686b6a4f50515144416a427a4d5173770a435159445651514745774a56557a45544d4245474131554543424d4b5132467361575a76636d3570595445574d4251474131554542784d4e5532467549455a790a5957356a61584e6a627a455a4d4263474131554543684d5162334a6e4d53356c654746746347786c4c6d4e76625445634d426f474131554541784d54593245750a62334a6e4d53356c654746746347786c4c6d4e7662544165467730784e7a41354d5467774f444d344d6a5a61467730794e7a41354d5459774f444d344d6a5a610a4d467378437a414a42674e5642415954416c56544d524d77455159445651514945777044595778705a6d3979626d6c684d5259774641594456515148457731540a59573467526e4a68626d4e7063324e764d523877485159445651514444425a425a473170626b4276636d63784c6d56345957317762475575593239744d466b770a457759484b6f5a497a6a3043415159494b6f5a497a6a3044415163445167414549675565466e3270676c624e575a5343474a624e4b513671554c415a684435570a4132616e3065717a426b37514b38516d446952394d2b3465634b4434713772702b6e7a3239324b553941316a71703046726e724c624b4e4e4d457377446759440a565230504151482f42415144416765414d41774741315564457745422f7751434d4141774b7759445652306a42435177496f41674d2b4f5039766d572f73454f0a654b5768525277434d644b644776614361686535792f2f4f42636c334e534577436759494b6f5a497a6a3045417749445277417752414967597057652b3174720a6244586c717631556578667a517a6d533533637832302f6a44487638363834566c775543494835484148336935377767576d6e2b6a564362764b38304b7732680a52686f6851453166534c49526c3670770a2d2d2d2d2d454e44202d2d2d2d2d0a
2017-09-18 08:39:18.682 UTC [cauthdsl] func2 -> DEBU 1b7 0xc42016e958 principal evaluation fails
2017-09-18 08:39:18.682 UTC [cauthdsl] func1 -> DEBU 1b8 0xc42016e958 gate 1505723958681604428 evaluation fails
2017-09-18 08:39:18.682 UTC [orderer/common/broadcast] Handle -> WARN 1b9 Rejecting CONFIG_UPDATE because: Error authorizing update: Error validating DeltaSet: Policy for [Groups] /Channel/Application not satisfied: Failed to reach implicit threshold of 1 sub-policies, required 1 remaining
2017-09-18 08:39:18.683 UTC [orderer/main] func1 -> DEBU 1ba Closing Broadcast stream
2017-09-18 08:39:18.684 UTC [grpc] Printf -> DEBU 1bb transport: http2Server.HandleStreams failed to read frame: read tcp 172.20.0.4:7050->172.20.0.8:40140: read: connection reset by peer
2017-09-18 08:39:18.684 UTC [orderer/common/deliver] Handle -> WARN 1bc Error reading from stream: rpc error: code = Canceled desc = context canceled
SimonOberzan (Mon, 18 Sep 2017 08:47:10 GMT):
Additional logs from orderer: `2017-09-18 08:39:18.681 UTC [msp/identity] newIdentity -> DEBU 1ae Creating identity instance for ID &{Org1MSP 21fba6df324ea4a2c978de34eee20fa9e89ff8f8efac9bcce6882b3ead095099}
2017-09-18 08:39:18.681 UTC [cauthdsl] func2 -> ERRO 1af Principal deserialization failure (The supplied identity is not valid, Verify() returned x509: certificate signed by unknown authority (possibly because of "x509: ECDSA verification failure" while trying to verify candidate authority certificate "ca.org1.example.com")) for identity 0a074f7267314d535012fc052d2d2d2d2d424547494e202d2d2d2d2d0a4d4949434744434341622b674177494241674951625a552b34534e427266364c32575930743637655a44414b42676771686b6a4f50515144416a427a4d5173770a435159445651514745774a56557a45544d4245474131554543424d4b5132467361575a76636d3570595445574d4251474131554542784d4e5532467549455a790a5957356a61584e6a627a455a4d4263474131554543684d5162334a6e4d53356c654746746347786c4c6d4e76625445634d426f474131554541784d54593245750a62334a6e4d53356c654746746347786c4c6d4e7662544165467730784e7a41354d5467774f444d344d6a5a61467730794e7a41354d5459774f444d344d6a5a610a4d467378437a414a42674e5642415954416c56544d524d77455159445651514945777044595778705a6d3979626d6c684d5259774641594456515148457731540a59573467526e4a68626d4e7063324e764d523877485159445651514444425a425a473170626b4276636d63784c6d56345957317762475575593239744d466b770a457759484b6f5a497a6a3043415159494b6f5a497a6a3044415163445167414549675565466e3270676c624e575a5343474a624e4b513671554c415a684435570a4132616e3065717a426b37514b38516d446952394d2b3465634b4434713772702b6e7a3239324b553941316a71703046726e724c624b4e4e4d457377446759440a565230504151482f42415144416765414d41774741315564457745422f7751434d4141774b7759445652306a42435177496f41674d2b4f5039766d572f73454f0a654b5768525277434d644b644776614361686535792f2f4f42636c334e534577436759494b6f5a497a6a3045417749445277417752414967597057652b3174720a6244586c717631556578667a517a6d533533637832302f6a44487638363834566c775543494835484148336935377767576d6e2b6a564362764b38304b7732680a52686f6851453166534c49526c3670770a2d2d2d2d2d454e44202d2d2d2d2d0a
2017-09-18 08:39:18.682 UTC [cauthdsl] func2 -> DEBU 1b7 0xc42016e958 principal evaluation fails
2017-09-18 08:39:18.682 UTC [cauthdsl] func1 -> DEBU 1b8 0xc42016e958 gate 1505723958681604428 evaluation fails
2017-09-18 08:39:18.682 UTC [orderer/common/broadcast] Handle -> WARN 1b9 Rejecting CONFIG_UPDATE because: Error authorizing update: Error validating DeltaSet: Policy for [Groups] /Channel/Application not satisfied: Failed to reach implicit threshold of 1 sub-policies, required 1 remaining
2017-09-18 08:39:18.683 UTC [orderer/main] func1 -> DEBU 1ba Closing Broadcast stream
2017-09-18 08:39:18.684 UTC [grpc] Printf -> DEBU 1bb transport: http2Server.HandleStreams failed to read frame: read tcp 172.20.0.4:7050->172.20.0.8:40140: read: connection reset by peer
2017-09-18 08:39:18.684 UTC [orderer/common/deliver] Handle -> WARN 1bc Error reading from stream: rpc error: code = Canceled desc = context canceled`
SimonOberzan (Mon, 18 Sep 2017 08:47:10 GMT):
Additional logs from orderer: ```2017-09-18 08:39:18.681 UTC [msp/identity] newIdentity -> DEBU 1ae Creating identity instance for ID &{Org1MSP 21fba6df324ea4a2c978de34eee20fa9e89ff8f8efac9bcce6882b3ead095099}
2017-09-18 08:39:18.681 UTC [cauthdsl] func2 -> ERRO 1af Principal deserialization failure (The supplied identity is not valid, Verify() returned x509: certificate signed by unknown authority (possibly because of "x509: ECDSA verification failure" while trying to verify candidate authority certificate "ca.org1.example.com")) for identity 0a074f7267314d535012fc052d2d2d2d2d424547494e202d2d2d2d2d0a4d4949434744434341622b674177494241674951625a552b34534e427266364c32575930743637655a44414b42676771686b6a4f50515144416a427a4d5173770a435159445651514745774a56557a45544d4245474131554543424d4b5132467361575a76636d3570595445574d4251474131554542784d4e5532467549455a790a5957356a61584e6a627a455a4d4263474131554543684d5162334a6e4d53356c654746746347786c4c6d4e76625445634d426f474131554541784d54593245750a62334a6e4d53356c654746746347786c4c6d4e7662544165467730784e7a41354d5467774f444d344d6a5a61467730794e7a41354d5459774f444d344d6a5a610a4d467378437a414a42674e5642415954416c56544d524d77455159445651514945777044595778705a6d3979626d6c684d5259774641594456515148457731540a59573467526e4a68626d4e7063324e764d523877485159445651514444425a425a473170626b4276636d63784c6d56345957317762475575593239744d466b770a457759484b6f5a497a6a3043415159494b6f5a497a6a3044415163445167414549675565466e3270676c624e575a5343474a624e4b513671554c415a684435570a4132616e3065717a426b37514b38516d446952394d2b3465634b4434713772702b6e7a3239324b553941316a71703046726e724c624b4e4e4d457377446759440a565230504151482f42415144416765414d41774741315564457745422f7751434d4141774b7759445652306a42435177496f41674d2b4f5039766d572f73454f0a654b5768525277434d644b644776614361686535792f2f4f42636c334e534577436759494b6f5a497a6a3045417749445277417752414967597057652b3174720a6244586c717631556578667a517a6d533533637832302f6a44487638363834566c775543494835484148336935377767576d6e2b6a564362764b38304b7732680a52686f6851453166534c49526c3670770a2d2d2d2d2d454e44202d2d2d2d2d0a
2017-09-18 08:39:18.682 UTC [cauthdsl] func2 -> DEBU 1b7 0xc42016e958 principal evaluation fails
2017-09-18 08:39:18.682 UTC [cauthdsl] func1 -> DEBU 1b8 0xc42016e958 gate 1505723958681604428 evaluation fails
2017-09-18 08:39:18.682 UTC [orderer/common/broadcast] Handle -> WARN 1b9 Rejecting CONFIG_UPDATE because: Error authorizing update: Error validating DeltaSet: Policy for [Groups] /Channel/Application not satisfied: Failed to reach implicit threshold of 1 sub-policies, required 1 remaining
2017-09-18 08:39:18.683 UTC [orderer/main] func1 -> DEBU 1ba Closing Broadcast stream
2017-09-18 08:39:18.684 UTC [grpc] Printf -> DEBU 1bb transport: http2Server.HandleStreams failed to read frame: read tcp 172.20.0.4:7050->172.20.0.8:40140: read: connection reset by peer
2017-09-18 08:39:18.684 UTC [orderer/common/deliver] Handle -> WARN 1bc Error reading from stream: rpc error: code = Canceled desc = context canceled```
SimonOberzan (Mon, 18 Sep 2017 08:48:46 GMT):
Any help appreciated
SimonOberzan (Mon, 18 Sep 2017 10:53:29 GMT):
Nevermind, I just fixed that problem by updating images to 1.0.2 and regenerating certificates and channel-artifacts
robert.feng (Mon, 18 Sep 2017 10:58:24 GMT):
hello everyone , I am robert , I fabric-explorer developer I'm a developer
robert.feng (Mon, 18 Sep 2017 11:00:30 GMT):
hello everyone . I am robter . The fabric-explorer developer. I'm glad to join you in the discussion。
baohua (Mon, 18 Sep 2017 13:32:43 GMT):
@SimonOberzan yes, the 1.0.2 change the code in diff format.
baohua (Mon, 18 Sep 2017 13:32:50 GMT):
@robert.feng welcome~
waaagh (Mon, 18 Sep 2017 14:15:11 GMT):
@baohua sorry for the delay, already edit the commit msg as ur suggestion :)
baohua (Mon, 18 Sep 2017 14:16:22 GMT):
thanks
tongli (Mon, 18 Sep 2017 14:40:23 GMT):
@baohua @hightall please review the patch set, I have addressed your comments. Thanks. https://gerrit.hyperledger.org/r/#/c/13489/
HintikkaKimmo (Mon, 18 Sep 2017 16:32:27 GMT):
Has joined the channel.
baohua (Mon, 18 Sep 2017 23:26:27 GMT):
thanks, tong, will review. and btw, do u know how to do the perf test now? Using PTE tool?
tongli (Tue, 19 Sep 2017 01:18:39 GMT):
@baohua for the performance test, we have been doing it manually. Now with the ansible agent, we can automatically provision vms, built and deploy fabric network with different layout, so that performance test can be done automatically. We are in process to run the entire performance test on ci and collect data for each run and make the result public
yoyokeen (Tue, 19 Sep 2017 01:18:47 GMT):
@robert.feng ,can u help me see the fabric-explorer issue7?
tongli (Tue, 19 Sep 2017 01:19:36 GMT):
Thanks @baohua for your review and plus 2 vote
tongli (Tue, 19 Sep 2017 01:20:25 GMT):
@hightall can you please also review the patch to get it merged? Our test team is waiting for this. Thanks a lot.
hightall (Tue, 19 Sep 2017 01:22:50 GMT):
@tongli in reviewing:slight_smile:
tongli (Tue, 19 Sep 2017 01:23:20 GMT):
@hightall really appreciate your work.
hightall (Tue, 19 Sep 2017 01:29:47 GMT):
@tongli I have no questions, merged
tongli (Tue, 19 Sep 2017 01:42:24 GMT):
@hightall @baohua thanks a lot for merging the patch. The team is really in need of this. Soon we can say that cello is used at fabric ci.
baohua (Tue, 19 Sep 2017 01:43:24 GMT):
glad to hear that!
yoyokeen (Tue, 19 Sep 2017 05:47:54 GMT):
@lixucheng
Yixing (Tue, 19 Sep 2017 07:38:42 GMT):
Has joined the channel.
Yixing (Tue, 19 Sep 2017 07:45:07 GMT):
@baohua @hightall we have update the UI proposal for vSphere agent, Please take a look when you have time.
https://docs.google.com/document/d/1ltbA3UIyUN2c6MUs2ilqjuMqm_-i7qNUaxEKGhqMDIk/edit?usp=sharing
Yixing (Tue, 19 Sep 2017 07:45:15 GMT):
Thanks.
baohua (Tue, 19 Sep 2017 07:45:35 GMT):
sure, thanks yixing
Yixing (Tue, 19 Sep 2017 09:30:03 GMT):
@baohua @hightall The UI proposal has been updated accordingly . Please help recheck. We plan to start implement the UI after your review.
baohua (Tue, 19 Sep 2017 09:36:17 GMT):
done, pls go ahead
Yixing (Tue, 19 Sep 2017 09:45:52 GMT):
Thanks!
yoyokeen (Wed, 20 Sep 2017 06:07:51 GMT):
@lujiang
baohua (Wed, 20 Sep 2017 13:46:05 GMT):
https://gerrit.hyperledger.org/r/#/c/13645/
baohua (Wed, 20 Sep 2017 23:47:12 GMT):
https://gerrit.hyperledger.org/r/#/c/13649/
sk (Thu, 21 Sep 2017 14:23:16 GMT):
I could setup and run Cello (master + worker) successfully and also created host and active chains. (1) How can I deploy chaincode (*.go files) program into active-chain network? (2) how can I access different method of that deployed chaincode?
baohua (Fri, 22 Sep 2017 01:14:50 GMT):
@sk currently we do not enable the user dashboard for help operate the chaincode. but after the chain bootup, it will expose the access API url, and you can feel free to use the sdk to call those API's.
net0410 (Fri, 22 Sep 2017 01:25:12 GMT):
Has joined the channel.
Ryan--Yang (Fri, 22 Sep 2017 02:54:17 GMT):
Has joined the channel.
sk (Fri, 22 Sep 2017 05:04:51 GMT):
@baohua thanks, but I think I am not getting it. Does it mean there is no way to deploy chaincode (cc) as of now? All I want it (after setup) want to run ChainCode on Fabric n/w.
sk (Fri, 22 Sep 2017 05:04:51 GMT):
@baohua thanks, but I think I am not getting it. Does it mean there is no way to deploy chaincode (cc) as of now? All I want is (after setup) that -> want to run ChainCode on Fabric n/w. how can I achieve it? May be with or without Cello?
sk (Fri, 22 Sep 2017 05:04:51 GMT):
@baohua thanks, but I think I am not getting it. Does it mean there is no way to deploy chaincode (cc) as of now? All I want is (after setup) want to run ChainCode on Fabric n/w.
sk (Fri, 22 Sep 2017 05:04:51 GMT):
@baohua thanks, but I think I am not getting it. Does it mean there is no way to deploy chaincode (cc) as of now? All I want is (after setup) that want to run ChainCode on Fabric n/w.
sk (Fri, 22 Sep 2017 05:04:51 GMT):
@baohua thanks, but I think I am not getting it. Does it mean there is no way to deploy chaincode (cc) as of now? All I want is (after setup) that -> want to run ChainCode on Fabric n/w. how can I achieve it? May be with or without Cell?
sk (Fri, 22 Sep 2017 05:11:24 GMT):
Message Attachments
baohua (Fri, 22 Sep 2017 06:35:41 GMT):
yes, then u can use sdk to operate the chaincode using the API.
sk (Fri, 22 Sep 2017 06:43:37 GMT):
Right, I already tried hitting that API with PostMan e.g. https://10.*.*.*:7050/chaincode with required POST body but it does not work
sk (Fri, 22 Sep 2017 06:55:14 GMT):
Also will try with https://fabric-sdk-node.github.io/ - (is this SDK are you referring, right?)
baohua (Fri, 22 Sep 2017 07:18:12 GMT):
there're several sdks in diff languages, feel free to pick one u prefer, good luck :)
sk (Fri, 22 Sep 2017 07:33:14 GMT):
@baohua Thanks :)
wampum (Fri, 22 Sep 2017 11:40:06 GMT):
Has joined the channel.
tongli (Fri, 22 Sep 2017 14:06:59 GMT):
@baohua, we have started using the cello ansible agent to do the performance test at local CI, once everything is working, we will move to LF fabric CI.
suresh_g_nair (Fri, 22 Sep 2017 18:55:20 GMT):
Has joined the channel.
nishanthkp (Fri, 22 Sep 2017 20:52:38 GMT):
Has joined the channel.
nishanthkp (Fri, 22 Sep 2017 20:53:53 GMT):
Hello All, I am Nishanth.
new to cello and would like to explore more . can tutorials to start and how to get cello dashboard
baohua (Sat, 23 Sep 2017 00:15:38 GMT):
@nishanthkp welcome, hope this can help you: cello.readthedocs.io
baohua (Sat, 23 Sep 2017 12:14:25 GMT):
https://gerrit.hyperledger.org/r/#/c/13755/
TrucNguyen (Sun, 24 Sep 2017 14:41:53 GMT):
Has joined the channel.
nishanthkp (Mon, 25 Sep 2017 09:00:32 GMT):
Thank you @baohua
rjones (Mon, 25 Sep 2017 20:10:34 GMT):
Has left the channel.
miiiiiyoung (Tue, 26 Sep 2017 02:13:09 GMT):
Has joined the channel.
miiiiiyoung (Tue, 26 Sep 2017 02:15:35 GMT):
Hi, I'm Miyoung.
I want to login user dashboard page, but it has an error.
When i tried to login or register, "System maintenance, please try again later!" message is printed.
How can i access user dashboard page?
baohua (Tue, 26 Sep 2017 05:26:21 GMT):
@miiiiiyoung the user dashboard is not connected yet. you can only use the admin dashboard now. http://cello.readthedocs.io/en/latest/
miiiiiyoung (Tue, 26 Sep 2017 05:34:11 GMT):
Then, when can i use user dashboard?
xiaoyz (Tue, 26 Sep 2017 07:40:58 GMT):
Has joined the channel.
baohua (Tue, 26 Sep 2017 07:55:54 GMT):
it should be after the v0.7 release
yushan (Tue, 26 Sep 2017 09:01:28 GMT):
Has joined the channel.
hightall (Wed, 27 Sep 2017 01:38:48 GMT):
https://gerrit.hyperledger.org/r/#/c/13841/
baohua (Wed, 27 Sep 2017 01:40:18 GMT):
thanks, @hightall +2!
hightall (Wed, 27 Sep 2017 01:40:56 GMT):
@baohua thanks
atulkamble (Wed, 27 Sep 2017 04:37:40 GMT):
Has joined the channel.
tiennv (Wed, 27 Sep 2017 12:43:12 GMT):
Has joined the channel.
ksachdeva (Thu, 28 Sep 2017 14:34:55 GMT):
Has joined the channel.
baohua (Thu, 28 Sep 2017 15:27:37 GMT):
Hyperledger community survey: https://www.surveymonkey.com/r/VNYZ6D7
hightall (Thu, 28 Sep 2017 15:27:48 GMT):
https://gerrit.hyperledger.org/r/#/c/13929/
ysim (Thu, 28 Sep 2017 15:35:44 GMT):
Has joined the channel.
baohua (Fri, 29 Sep 2017 00:54:12 GMT):
thanks haitao!
Yixing (Fri, 29 Sep 2017 02:05:15 GMT):
Hi @baohua @hightall I have just submit the refactor code vSphere agent driver. Please help review when you have time. Thank!
https://gerrit.hyperledger.org/r/13951
baohua (Fri, 29 Sep 2017 04:38:47 GMT):
thanks, and seems there are ci failures. can u help fix?
baohua (Fri, 29 Sep 2017 04:38:51 GMT):
@Yixing
Luke_Chen (Fri, 29 Sep 2017 09:20:18 GMT):
@baohua Failures are fixed, thanks for your kindly reminder.
baohua (Sat, 30 Sep 2017 01:21:26 GMT):
thanks
AnilOner (Mon, 02 Oct 2017 16:40:28 GMT):
Has joined the channel.
rjones (Wed, 04 Oct 2017 18:56:20 GMT):
Has joined the channel.
rjones (Wed, 04 Oct 2017 18:58:10 GMT):
@baohua @david_dornseifer I am working on fixing verification for https://gerrit.hyperledger.org/r/#/c/14177/ and I will re-trigger it for you I apologize for the delay
cryptozee (Wed, 04 Oct 2017 20:47:04 GMT):
Has joined the channel.
baohua (Thu, 05 Oct 2017 00:55:07 GMT):
thanks for the help, ry
rjones (Thu, 05 Oct 2017 01:09:12 GMT):
sorry I haven't figured it out yet, something changed, very strange
rjones (Thu, 05 Oct 2017 06:35:18 GMT):
@baohua @david_dornseifer I fixed the issue. The build failure in 14177 is real: trailing whitespace
baohua (Thu, 05 Oct 2017 06:36:04 GMT):
thanks for the confirmation @rjones, and i believe this will help use fix it soon @david_dornseifer :)
rjones (Thu, 05 Oct 2017 06:37:01 GMT):
the issue was the current version of Jenkins doesn't support the older version of Java we have baked in, so I need to build a new image to fix that. I have a workaround in place though :)
baohua (Thu, 05 Oct 2017 07:15:05 GMT):
good work, ry!
clima (Thu, 05 Oct 2017 23:57:10 GMT):
Has joined the channel.
baohua (Fri, 06 Oct 2017 02:59:23 GMT):
https://gerrit.hyperledger.org/r/#/c/14177/
jonaxx20 (Mon, 09 Oct 2017 06:27:59 GMT):
Has joined the channel.
jonaxx20 (Mon, 09 Oct 2017 06:28:58 GMT):
Hello @baohua , I was wondering, if its possible to take out docker from cello, and use VMs instead
baohua (Mon, 09 Oct 2017 09:05:21 GMT):
u mean the run-time of cello services? @jonaxx20
baohua (Mon, 09 Oct 2017 09:05:38 GMT):
@here, dear all, as planed, v0.7 should be release in this month, pls help suggest problems and bugs we need to fix for the release. Thanks!
baohua (Tue, 10 Oct 2017 02:10:12 GMT):
https://gerrit.hyperledger.org/r/#/c/14331/
julian (Tue, 10 Oct 2017 11:03:14 GMT):
Has joined the channel.
yoheiueda (Thu, 12 Oct 2017 04:07:18 GMT):
Has joined the channel.
qiangminqian (Thu, 12 Oct 2017 08:14:09 GMT):
Has joined the channel.
douglasthiel (Fri, 13 Oct 2017 19:42:53 GMT):
Has joined the channel.
ydk210999 (Sun, 15 Oct 2017 13:33:37 GMT):
Has joined the channel.
baohua (Mon, 16 Oct 2017 05:47:24 GMT):
https://gerrit.hyperledger.org/r/#/c/14531/
rexxie (Mon, 16 Oct 2017 08:26:38 GMT):
Has joined the channel.
rexxie (Mon, 16 Oct 2017 08:28:27 GMT):
step by step install in Centos7,if you have? @baohua
saeedi (Mon, 16 Oct 2017 08:34:10 GMT):
Has joined the channel.
rexxie (Mon, 16 Oct 2017 08:38:34 GMT):
杨老师,按照书本,今天试着安装cello,安装完成但是没有数据,也没办法添加节点,有无更详细的例子介绍 @baohua
rubi_1432 (Mon, 16 Oct 2017 08:47:08 GMT):
Hi everyone, can anyone guide me on how to setup fabric network on physical machines using *ansible* ?
baohua (Mon, 16 Oct 2017 08:49:05 GMT):
@rexxie latest doc is at http://cello.readthedocs.io/en/latest/. if you have further question, pls send mail to mailto:hyperledger-cello@lists.hyperledger.org
baohua (Mon, 16 Oct 2017 08:49:37 GMT):
@rubi_1432 are u looking for sth like https://github.com/hyperledger/cello/tree/master/src/agent/ansible?
rubi_1432 (Mon, 16 Oct 2017 08:50:34 GMT):
@baohua Yes. Actua
rubi_1432 (Mon, 16 Oct 2017 08:51:57 GMT):
@baohua Yes i am looking into it but i am confused as for how i should edit script files for physical servers.
baohua (Mon, 16 Oct 2017 08:52:44 GMT):
i think @tongli could give some hints :)
rubi_1432 (Mon, 16 Oct 2017 08:52:52 GMT):
@baohua .yml files are for azure, os etc but for physical machines i have no idea where to start
baohua (Mon, 16 Oct 2017 08:53:01 GMT):
besides, there's doc inside the path, does that help u?
rubi_1432 (Mon, 16 Oct 2017 08:53:56 GMT):
Yes it did. I was able to understand a lot but there are few core concepts that i have ambiguity about.
rubi_1432 (Mon, 16 Oct 2017 08:55:21 GMT):
@baohua I have contacted @tongli but i didn't reicevd any response for him when i told about that i have physical machines ready for deployment :sweat_smile:
baohua (Mon, 16 Oct 2017 08:57:16 GMT):
aha, he might be offline, would u like to send a mail to to our list?
rubi_1432 (Mon, 16 Oct 2017 08:59:30 GMT):
Maybe. I sent him last message like a month ago :sweat_smile: . Yes sure but How can i send email to your list ?
baohua (Mon, 16 Oct 2017 09:34:21 GMT):
cello's maillist is mailto:hyperledger-cello@lists.hyperledger.org
baohua (Mon, 16 Oct 2017 09:34:41 GMT):
all hyperledger email lists can be found at https://lists.hyperledger.org/
baohua (Mon, 16 Oct 2017 09:34:54 GMT):
See how to join: https://lists.hyperledger.org/mailman/listinfo/hyperledger-cello
rubi_1432 (Mon, 16 Oct 2017 10:49:12 GMT):
Okay Thanks a lot :-)
waaagh (Mon, 16 Oct 2017 14:46:46 GMT):
@baohua Got a bit spare time recently, is there something i can do to improve our lovely Cello?
tongli (Mon, 16 Oct 2017 16:40:42 GMT):
@rubi_1432 hi, did not see your message, I was on vacation in China and just came back from my vacation. sorry for missing your call.
rubi_1432 (Tue, 17 Oct 2017 07:45:39 GMT):
@tongli Hi, No problem, i hope you had a great vacation. Please do let me know what time works for you as i believe we are 9 hours apart and currently it is 12:45 pm here
Yixing (Tue, 17 Oct 2017 14:49:25 GMT):
https://gerrit.hyperledger.org/r/#/c/14591/ @baohua @hightall Thanks!
tongli (Tue, 17 Oct 2017 18:38:46 GMT):
https://gerrit.hyperledger.org/r/#/c/14605/ @baohua @hightall , added a new document to give step by step instructions on how to stand up fabric network on single node. Please review. Thanks.
baohua (Wed, 18 Oct 2017 04:11:25 GMT):
@tongli reviewed, thanks!
baohua (Wed, 18 Oct 2017 04:45:26 GMT):
https://gerrit.hyperledger.org/r/#/c/14591/
Luke_Chen (Wed, 18 Oct 2017 07:05:44 GMT):
our subject https://gerrit.hyperledger.org/r/#/c/14591/1..2 didn't pass the Jobbuilder review, and some jenkins logs are below `+ make check
tox
make: tox: Command not found
Makefile:68: recipe for target 'check' failed
make: *** [check] Error 127
Build step 'Execute shell' marked build as failure
$ ssh-agent -k
unset SSH_AUTH_SOCK;
unset SSH_AGENT_PID;
echo Agent pid 1779 killed;
[ssh-agent] Stopped.
[PostBuildScript] - Execution post build scripts.
[cello-verify-x86_64] $ /bin/bash /tmp/jenkins1561893299464409422.sh
---> sysstat.sh
grep: /etc/default/sysstat: No such file or directory`
Luke_Chen (Wed, 18 Oct 2017 07:06:02 GMT):
our subject https://gerrit.hyperledger.org/r/#/c/14591/1..2 didn't pass the Jobbuilder review, and some jenkins logs are below `+ make check
tox
make: tox: Command not found
Makefile:68: recipe for target 'check' failed
make: *** [check] Error 127
Build step 'Execute shell' marked build as failure
$ ssh-agent -k
unset SSH_AUTH_SOCK;
unset SSH_AGENT_PID;
echo Agent pid 1779 killed;
[ssh-agent] Stopped.
[PostBuildScript] - Execution post build scripts.
[cello-verify-x86_64] $ /bin/bash /tmp/jenkins1561893299464409422.sh
---> sysstat.sh
grep: /etc/default/sysstat: No such file or directory`
Luke_Chen (Wed, 18 Oct 2017 07:06:20 GMT):
our subject https://gerrit.hyperledger.org/r/#/c/14591/1..2 didn't pass the Jobbuilder review, and some jenkins logs are below ```+ make check
tox
make: tox: Command not found
Makefile:68: recipe for target 'check' failed
make: *** [check] Error 127
Build step 'Execute shell' marked build as failure
$ ssh-agent -k
unset SSH_AUTH_SOCK;
unset SSH_AGENT_PID;
echo Agent pid 1779 killed;
[ssh-agent] Stopped.
[PostBuildScript] - Execution post build scripts.
[cello-verify-x86_64] $ /bin/bash /tmp/jenkins1561893299464409422.sh
---> sysstat.sh
grep: /etc/default/sysstat: No such file or directory```
Luke_Chen (Wed, 18 Oct 2017 07:08:33 GMT):
We passed all tox check in our local environment.
Yixing (Wed, 18 Oct 2017 07:13:03 GMT):
@Luke_Chen no hurry, I will trigger a rebuild.
Luke_Chen (Wed, 18 Oct 2017 07:16:42 GMT):
thanks :)
waaagh (Wed, 18 Oct 2017 08:15:09 GMT):
@Luke_Chen @Yixing seems there's something wrong with the builder image, i meet a similar issue a few days ago, try contact @rjones for help
Yixing (Wed, 18 Oct 2017 08:29:27 GMT):
@waaagh Thanks!
waaagh (Wed, 18 Oct 2017 08:31:12 GMT):
sure thing :)
rjones (Wed, 18 Oct 2017 15:12:38 GMT):
@jwagantall ^^^^
jwagantall (Wed, 18 Oct 2017 15:51:44 GMT):
Oh....tox command not found? So strange....let me work on it....does it happens in all images? Looking at it
jwagantall (Wed, 18 Oct 2017 16:02:14 GMT):
here is my change... will make a new images https://gerrit.hyperledger.org/r/#/c/14651/ I wonder how was tox working before if it was not part of the baseimage :(
jwagantall (Wed, 18 Oct 2017 19:05:02 GMT):
added few more packages... the image is now updated with all these upgrades and I am retriggering the cello verifications now
jwagantall (Wed, 18 Oct 2017 19:08:28 GMT):
looking good now
baohua (Thu, 19 Oct 2017 02:47:11 GMT):
great to know it's resolved quickly!
Luke_Chen (Thu, 19 Oct 2017 05:09:15 GMT):
@jwagantall thank you very much :)
lin2K (Fri, 20 Oct 2017 05:09:17 GMT):
Has joined the channel.
bogdancioc (Fri, 20 Oct 2017 08:50:53 GMT):
Has joined the channel.
bogdancioc (Fri, 20 Oct 2017 08:51:23 GMT):
Hi guys, I'm new to Cello and Fabric, so sorry if my questions are basic
bogdancioc (Fri, 20 Oct 2017 08:51:44 GMT):
I'm installing the cello master node on OSX, on my mac
bogdancioc (Fri, 20 Oct 2017 08:51:56 GMT):
I am able to start the localhost:8080 web interface
bogdancioc (Fri, 20 Oct 2017 08:52:13 GMT):
but the default login is not accepted
bogdancioc (Fri, 20 Oct 2017 08:52:34 GMT):
looking in the logs of the mongodb container
bogdancioc (Fri, 20 Oct 2017 08:52:40 GMT):
i get this message that
bogdancioc (Fri, 20 Oct 2017 09:02:05 GMT):
"chown: changing ownership of '/data/db': Operation not permitted"
bogdancioc (Fri, 20 Oct 2017 09:02:11 GMT):
any ideas?
bogdancioc (Fri, 20 Oct 2017 09:03:20 GMT):
I'm running Docker 17.09.0-ce-mac35 (19611) on OSX 10.12.6 (Sierra)
baohua (Fri, 20 Oct 2017 13:37:17 GMT):
aha
baohua (Fri, 20 Oct 2017 13:37:33 GMT):
When using Cello on MacOS, the mongodb container may fail to start. This is
because the container will try to mount `/opt/cello/mongo` path. To resolve
the problem, users need to add `/opt/cello` to Docker's sharing path.
bogdancioc (Fri, 20 Oct 2017 13:53:27 GMT):
Hi baohua, I added /opt/cell/mongo to the list of local directories that can be mounted from within Docker
bogdancioc (Fri, 20 Oct 2017 13:53:38 GMT):
the container starts succesfully
bogdancioc (Fri, 20 Oct 2017 13:54:42 GMT):
and then throws a whole lot of "chown: changing ownership of '/data/db': Operation not permitted" in the log
bogdancioc (Fri, 20 Oct 2017 13:59:13 GMT):
what I can see is that the mongo container is trying to use two volumes: /data/configdb and /data/db
bogdancioc (Fri, 20 Oct 2017 15:57:30 GMT):
I think this line in the /cello/scripts/master_node/setup.sh might be the problem:
bogdancioc (Fri, 20 Oct 2017 15:57:32 GMT):
echo_b "Add existing user to docker group"
sudo usermod -aG docker ${USER}
bogdancioc (Fri, 20 Oct 2017 15:57:42 GMT):
usermod doesn't work on OSX
bogdancioc (Fri, 20 Oct 2017 15:57:55 GMT):
any idea what I could replace it with?
bogdancioc (Fri, 20 Oct 2017 17:02:14 GMT):
I replaced the above line with this, and it worked
bogdancioc (Fri, 20 Oct 2017 17:02:15 GMT):
sudo dseditgroup -o edit -a ${USER} -t user staff
bogdancioc (Fri, 20 Oct 2017 17:02:41 GMT):
Can any of you guys tell me if this wrong or not?
bogdancioc (Fri, 20 Oct 2017 17:02:53 GMT):
thanks
bogdancioc (Fri, 20 Oct 2017 18:38:37 GMT):
Hi baohua, now I have another problem :)
bogdancioc (Fri, 20 Oct 2017 18:39:48 GMT):
Creating a working host doesn't work: "Failed to create host"
bogdancioc (Fri, 20 Oct 2017 18:42:16 GMT):
the dashboard container log says: "[2017-10-21 02:39:05,591] DEBUG [modules.host] [host.py:83 create()] - Create host: name=xxxxxxxxx, worker_api=192.168.x.xxx:2375, host_type=docker, capacity=5, log=local/, autofill=false, schedulable=false
[2017-10-21 02:39:10,601] ERROR [agent.docker.docker_swarm] [docker_swarm.py:152 check_daemon()] - Exception in check_daemon Error while fetching server API version: HTTPConnectionPool(host='192.168.x.xxx', port=2375): Max retries exceeded with url: /version (Caused by ConnectTimeoutError(
bogdancioc (Fri, 20 Oct 2017 20:28:35 GMT):
I've just installed Cello in a Ubuntu 17.04 Zesty VirtualBox (running on my Mac)
bogdancioc (Fri, 20 Oct 2017 20:29:11 GMT):
I have exactly the same error> "Failed to create host"
bogdancioc (Fri, 20 Oct 2017 20:31:13 GMT):
perhaps the problem is Docker version 17.09.0-ce, build afdb6d4 ???
baohua (Fri, 20 Oct 2017 23:57:05 GMT):
@bogdancioc we do not have much doc on using cello on mac... would appreciate if you wanna add some. And can u connect to ur worker api from master? `sudo docker -H 192.168.x.xxx:2375 info` return success?
Yixing (Sat, 21 Oct 2017 04:00:46 GMT):
@bogdancioc by default Docker 17.09 will not listen on port 2375, you should specify the port when start docker daemon.
Abhijeet.K (Sun, 22 Oct 2017 10:23:26 GMT):
Has joined the channel.
Abhijeet.K (Sun, 22 Oct 2017 10:25:03 GMT):
Hi, How does cello works? we need to have docker swarm first installed? each worker node is is treated a peer or blockchain will be started on each worker node?
Yixing (Mon, 23 Oct 2017 09:46:42 GMT):
Please help review https://gerrit.hyperledger.org/r/#/c/14765/
aviralwal (Mon, 23 Oct 2017 10:20:07 GMT):
Has joined the channel.
SubhodI (Mon, 23 Oct 2017 12:29:32 GMT):
Has joined the channel.
baohua (Tue, 24 Oct 2017 14:35:49 GMT):
@Abhijeet.K yes, u need to install docker first if you wanna add a docker host as worker node.
baohua (Tue, 24 Oct 2017 14:36:28 GMT):
worker nodes can be some virtual server and combine as a single infrastructure pool, as a Host in Cello.
baohua (Tue, 24 Oct 2017 14:55:18 GMT):
@Yixing reviewed, thanks. And pls mark the jira task as in review with adding the patchset link as a comment. thanks!
Yixing (Tue, 24 Oct 2017 15:04:01 GMT):
@baohua Thanks, I see you already help add the patch link in the jira. I will add the patch link in furture jira requests.
Yixing (Tue, 24 Oct 2017 15:06:06 GMT):
Also I find the patch cannot be submitted. Do we need to get more reviewer's approve?
baohua (Tue, 24 Oct 2017 15:54:09 GMT):
yes, let's wait see if there's other comments. After that, we can safely merge it.
MohammadObaid (Tue, 24 Oct 2017 18:19:03 GMT):
Has joined the channel.
MohammadObaid (Tue, 24 Oct 2017 18:21:50 GMT):
Has anyone here deploy hyperledger-fabric using cello? Is there any sample project available which deploy fabric using cello on different machines?
baohua (Tue, 24 Oct 2017 19:06:11 GMT):
yes, mohammad
baohua (Tue, 24 Oct 2017 19:10:01 GMT):
cello support multiple underly infrastructures like swarm, k8s, which will support cross-host networking. on top of that, we can deploy fabric on diff servers.
sk (Wed, 25 Oct 2017 05:45:20 GMT):
I could successfully setup Cello, created host and created chain also (via web-dashboard). Now I want to deploy chaincode, execute queries through web services. How can I do this? I visited http://cello.readthedocs.io/en/latest/api/restserver_v2/ but it does not have that functionalities. Please help.
baohua (Wed, 25 Oct 2017 17:02:55 GMT):
@sk cello doesn't provide additional support on chaincode rather than fabric itself, hence just use the network as fabric suggest to.
baohua (Wed, 25 Oct 2017 17:02:55 GMT):
@sk Right now, cello doesn't provide additional support on chaincode rather than fabric itself, hence just use the network as fabric suggest to.
baohua (Wed, 25 Oct 2017 17:03:10 GMT):
e.g., can use the fabric-sdk to trigger txs to a network.
MohammadObaid (Wed, 25 Oct 2017 17:49:21 GMT):
Hey when I run `make start` I am getting timeout error despite being my internet working normally .
MohammadObaid (Wed, 25 Oct 2017 17:50:58 GMT):
makestart.png
username343 (Thu, 26 Oct 2017 03:39:40 GMT):
Has joined the channel.
panickervinod (Thu, 26 Oct 2017 06:28:49 GMT):
Has joined the channel.
MohammadObaid (Thu, 26 Oct 2017 08:43:05 GMT):
@baohua Can I setup environment using ibm blumeix instead of aws ? Like one local host on my virtual box and one host on ibm blumeix . Can we create such networks and run fabric on it?
ascatox (Thu, 26 Oct 2017 10:14:04 GMT):
Has joined the channel.
lclclc (Thu, 26 Oct 2017 12:11:20 GMT):
Has joined the channel.
lclclc (Thu, 26 Oct 2017 12:11:26 GMT):
Hi
lclclc (Thu, 26 Oct 2017 12:11:53 GMT):
Is it possible to let cello master and worker deployed in on experiment machine?
lclclc (Thu, 26 Oct 2017 12:14:37 GMT):
Will be there potential port or process collision?
baohua (Thu, 26 Oct 2017 13:58:50 GMT):
what is an experiment machine? you can deploy on the same server, there's no collision.
knagware9 (Thu, 26 Oct 2017 16:55:57 GMT):
Has left the channel.
ThiagoAzevedo (Thu, 26 Oct 2017 19:24:48 GMT):
Has joined the channel.
lclclc (Fri, 27 Oct 2017 03:29:49 GMT):
[ ](https://chat.hyperledger.org/channel/cello?msg=QmeEfBQAZWCYotxj5) @baohua Thanks for the answer.
lclclc (Fri, 27 Oct 2017 04:54:09 GMT):
I saw the cello master has experimantal support for docker: `Docker 17.0+ support is experimental`, and I am using 17.09.0-ce of docker client and server. Is there potential risk I will meet?
lclclc (Fri, 27 Oct 2017 04:55:15 GMT):
Clipboard
lclclc (Fri, 27 Oct 2017 04:55:56 GMT):
Now I can set up the master's services, but default admin/pass can not make me logged in.
lclclc (Fri, 27 Oct 2017 04:56:11 GMT):
Is it related to docker version? How can I debug this?
lclclc (Fri, 27 Oct 2017 06:34:32 GMT):
```[2017-10-27 14:17:49,789] ERROR [modules.user.user] [user.py:99 get_by_username()] - get user exc mongo:27017: [Errno -2] Name or service not known
[2017-10-27 14:17:49,792] INFO [modules.user.auth.login] [login.py:70 post()] - error 'NoneType' object has no attribute 'password'```
lclclc (Fri, 27 Oct 2017 06:35:21 GMT):
I check dashboard service, looks like the mongo does not initiate admin/pass.
lclclc (Fri, 27 Oct 2017 07:06:00 GMT):
Clipboard
lclclc (Fri, 27 Oct 2017 07:06:19 GMT):
Looks like it is related to user privilege. I am using sudo make restart.
baohua (Fri, 27 Oct 2017 07:19:09 GMT):
@hightall can u help have a look?
lclclc (Fri, 27 Oct 2017 07:26:58 GMT):
Thanks in advance
ascatox (Fri, 27 Oct 2017 08:11:58 GMT):
Hi All! Someone has encountered this error: `chown: changing ownership of '/data/db': Operation not permitted``
ascatox (Fri, 27 Oct 2017 08:12:19 GMT):
I'm using macOs!
lclclc (Fri, 27 Oct 2017 11:03:03 GMT):
Actually I re-read the document and found this: ```MacOSX, *nix, or Windows 10: Docker Docker version 17.06.2-ce or greater is required.```
lclclc (Fri, 27 Oct 2017 11:03:03 GMT):
Actually I re-read the fabric's document and found this: ```MacOSX, *nix, or Windows 10: Docker Docker version 17.06.2-ce or greater is required.```
lclclc (Fri, 27 Oct 2017 11:03:03 GMT):
Actually I re-read the fabric's document and found this: ```MacOSX, *nix, or Windows 10: Docker Docker version 17.06.2-ce or greater is required.```
lclclc (Fri, 27 Oct 2017 11:03:55 GMT):
Master nodes has only experimental support for 17.0+ docker
lclclc (Fri, 27 Oct 2017 11:03:55 GMT):
But cello Master nodes has only experimental support for 17.0+ docker
lclclc (Fri, 27 Oct 2017 11:03:55 GMT):
But cello Master nodes has only experimental support for 17.0+ docker
lclclc (Fri, 27 Oct 2017 11:04:11 GMT):
Wokers nodes even don't have such declaration.
lclclc (Fri, 27 Oct 2017 11:04:11 GMT):
cello Wokers nodes even don't have such declaration.
baohua (Fri, 27 Oct 2017 14:56:26 GMT):
yes, pls help report if that version works, thanks! We welcome for such report on more types of platforms and versions
MohammadObaid (Fri, 27 Oct 2017 18:35:54 GMT):
@baohua In worker installation on this guide http://hyperledger-cello.readthedocs.io/en/latest/installation_worker_docker/ first step is to install docker daemon . Does it mean to create swarm network between master and worker node? right?
MohammadObaid (Fri, 27 Oct 2017 18:45:06 GMT):
Hi @baohua Is there any tutorial or link available as how to add fabric as a chain in hyperledger-cello. I have ready fabric-sdk network including all images and other artifacts. Now how do I will add that into cello chain? Any little clue or assist will really help me a lot. :)
baohua (Fri, 27 Oct 2017 21:45:31 GMT):
aha, @MohammadObaid we do not support this feature, only way is that u create a fabric network through cello. if you wanna it, pls help open an jira task, thanks!
MohammadObaid (Sat, 28 Oct 2017 10:44:26 GMT):
@baohua Hey when I click on `Add Chain` I am getting error `Failed to add cluster ` . Here are my logs. Any clue to solve this?
MohammadObaid (Sat, 28 Oct 2017 17:21:10 GMT):
hey @baohua I have successfully setup cello on docker 17.09 ce version and able to install default fabric 1.0 chain using two machines. Now I want to know how can I customize default chain like modifying chaincode and other stuffs etc. Any doc link available for this?
ivaylobb (Sun, 29 Oct 2017 23:47:22 GMT):
Has joined the channel.
ivaylobb (Sun, 29 Oct 2017 23:47:29 GMT):
hello
lclclc (Mon, 30 Oct 2017 02:12:39 GMT):
[ ](https://chat.hyperledger.org/channel/cello?msg=Pp9vxCjmYwf7pAB8K) @MohammadObaid What's your environment? Actually my Cello works without special error, except my Mongo service can not be initialized. I and not quite sure, what is the root cause of it. I don't have root account, but I can run sudo commands. I am using 17.09 ce, and I have a local Mongodb installed.
lclclc (Mon, 30 Oct 2017 02:13:11 GMT):
Can it relate to mongo conflict?
baohua (Mon, 30 Oct 2017 02:50:41 GMT):
@MohammadObaid cello doesn't add any limit to fabric operations like chaincode usage, just follow the steps in the fabric docs. http://hlf.readthedocs.io/en/latest/
MohammadObaid (Mon, 30 Oct 2017 04:50:40 GMT):
@lclclc I run cello on ubuntu 16 version
lclclc (Mon, 30 Oct 2017 04:50:56 GMT):
I am running on Mac
MohammadObaid (Mon, 30 Oct 2017 04:51:46 GMT):
What you got on watchdog logs?
lclclc (Mon, 30 Oct 2017 04:53:46 GMT):
let me paste this again.
lclclc (Mon, 30 Oct 2017 04:54:09 GMT):
Clipboard
lclclc (Mon, 30 Oct 2017 04:54:54 GMT):
As I said before, the mongo operation failed at first. I don't if it is the root cause of other failure.
lclclc (Mon, 30 Oct 2017 04:56:25 GMT):
Clipboard
lclclc (Mon, 30 Oct 2017 04:56:27 GMT):
I have share that path for mongo via docker preference:
MohammadObaid (Mon, 30 Oct 2017 05:18:13 GMT):
I dont think its a problem with a docker.
lclclc (Mon, 30 Oct 2017 05:18:42 GMT):
Actually this is should be random case of Mongo 3.x image, what I have done with Docker preference is what I googled and found.
lclclc (Mon, 30 Oct 2017 05:19:18 GMT):
I assume there is a real physical file system path the docker is trying to map, but it is denied.
lclclc (Mon, 30 Oct 2017 05:19:57 GMT):
I just can't know how to debug it. Because I am running Cello, not directly a mongo container.
MohammadObaid (Mon, 30 Oct 2017 05:27:26 GMT):
Hmm . @lclclc Sorry I cannot help you on this. Havent encountered any sort of error like this
lclclc (Mon, 30 Oct 2017 05:28:04 GMT):
Now I am setting up a ubuntu docker container, trying to run Cello inside it.
MohammadObaid (Mon, 30 Oct 2017 05:34:06 GMT):
@baohua I beleive this is the directory I need to make changes for my custom fabric-network like adding host in docker-compose files right? But I havent find any folder which contains chaincode path? What is the default directory?In normal fabric directory we does have chaincode direcotory where we put all chaincodes
MohammadObaid (Mon, 30 Oct 2017 05:34:06 GMT):
@baohua I beleive this is the directory where I need to make changes for my custom fabric-network like adding host in docker-compose files right? But I havent find any folder which contains chaincode path? What is the default directory?In normal fabric directory we does have chaincode direcotory where we put all chaincodes
MohammadObaid (Mon, 30 Oct 2017 05:34:39 GMT):
fabric-cello.png
Yixing (Mon, 30 Oct 2017 09:19:01 GMT):
Currently Cello support deploy Fabric 0.6 and Fabric 1.0 cluster.
but the image tags used in the docker compose file are missing.
For example : image: hyperledger/fabric-peer
Yixing (Mon, 30 Oct 2017 09:24:40 GMT):
Currently Cello support deploy Fabric 0.6 and Fabric 1.0 cluster.
But the image tags used in the docker compose file are missing.
For example : image: hyperledger/fabric-peer
This will cause our work node can only run one version of facbric, Ideally the work node should be decouple with what is running on it.
Is there any background that we don't specify the image tag in the docker compose file?
I know that the chain-code image always use latest tag, is this the reason ?
Yixing (Mon, 30 Oct 2017 09:30:04 GMT):
I create a issue to track the discussion https://jira.hyperledger.org/browse/CE-160
hightall (Mon, 30 Oct 2017 09:30:05 GMT):
@indirajith Are you still dealing with JIRA task CE-97?
lclclc (Mon, 30 Oct 2017 09:44:13 GMT):
@hightall can you help to diagnose my Mac issue? Do you have any idea about when the mongo service will fail on Mac? Even after I uninstall the mongo application, I can't bootstrap the mongo service, so I can't log into dashboard.
Luke_Chen (Mon, 30 Oct 2017 09:58:57 GMT):
https://gerrit.hyperledger.org/r/#/c/14951/ feel free to leave your comments
indirajith (Mon, 30 Oct 2017 10:11:13 GMT):
@hightall hi, yes. will update the patch in few days time.
baohua (Mon, 30 Oct 2017 13:14:36 GMT):
@lclclc what's the log for the mongo container?
baohua (Mon, 30 Oct 2017 13:15:29 GMT):
i just add some notes on using Cello on MacOS: https://gerrit.hyperledger.org/r/#/c/14953/
baohua (Mon, 30 Oct 2017 13:15:40 GMT):
hope this can help solve the issues
nicolapaoli (Mon, 30 Oct 2017 16:10:13 GMT):
Has joined the channel.
rameshthoomu (Tue, 31 Oct 2017 00:22:08 GMT):
Has joined the channel.
hightall (Tue, 31 Oct 2017 01:15:44 GMT):
@indirajith thanks, I'm working on the user dashboard, now we need health check for fabric 1.0.
baohua (Tue, 31 Oct 2017 01:44:57 GMT):
The hyperledger summit will be held in singapore on Nov 9,10. Who will attend, too? Let's see if opportunity to meet each other and discuss the project.
lclclc (Tue, 31 Oct 2017 02:00:21 GMT):
[ ](https://chat.hyperledger.org/channel/cello?msg=G6cuxTFeraRWDLFFi) @baohua Thanks for replying me. Actually I figure out the issue myself yesterday. The issue is That /opt/cello is not share by root account, even I turn on the file sharing according to others experience, I finally chmoded and chowned to fix it. The lesson I learn is, Kitematic does not have the magic as it appears.
baohua (Tue, 31 Oct 2017 02:01:04 GMT):
aha, this is really helpful for us, i will add ur finding into the patchset, too
baohua (Tue, 31 Oct 2017 02:01:31 GMT):
Thanks and welcome for more reporting and patchsets, too
lclclc (Tue, 31 Oct 2017 02:02:14 GMT):
I suggest you to add this into set-up steps. Because Kitematic is default tool for Docker daemon in Mac.
lclclc (Tue, 31 Oct 2017 02:02:43 GMT):
And google result and github tips usually consider the file-sharing menu will help.
lclclc (Tue, 31 Oct 2017 02:03:30 GMT):
And you notes on random port of docker daemon will also be helpful to new users. Yesterday I also met that.
baohua (Tue, 31 Oct 2017 02:04:22 GMT):
random port?
baohua (Tue, 31 Oct 2017 02:04:29 GMT):
sorry, can u elaborate that?
baohua (Tue, 31 Oct 2017 02:06:51 GMT):
@lclclc in `scripts/master_node/setup.sh`, we have the code to check the path already.
lclclc (Tue, 31 Oct 2017 02:09:04 GMT):
Users need to use some tools to config Docker daemon to listen at network manually, e.g., to config Docker daemon to listen on `127.0.0.1:2375`,
lclclc (Tue, 31 Oct 2017 02:09:10 GMT):
I mean this is helpful
lclclc (Tue, 31 Oct 2017 02:09:54 GMT):
What I mean by random port should be "/var/run/docker.sock"
baohua (Tue, 31 Oct 2017 02:20:07 GMT):
ok, got it, thanks
baohua (Tue, 31 Oct 2017 02:54:43 GMT):
Welcome to help review: https://gerrit.hyperledger.org/r/#/c/14953/, a quick doc optimization :)
lclclc (Tue, 31 Oct 2017 03:25:04 GMT):
Nice doc
lclclc (Tue, 31 Oct 2017 03:25:56 GMT):
currently I set up a proxy via socat via docker container, the telnet is responding, but the docker_swarm script is failing:
lclclc (Tue, 31 Oct 2017 03:26:18 GMT):
```$ telnet 127.0.0.1 2375
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
HTTP/1.1 400 Bad Request
Content-Type: text/plain; charset=utf-8
Connection: close
400 Bad RequestConnection closed by foreign host.```
lclclc (Tue, 31 Oct 2017 03:26:38 GMT):
```$ curl localhost:2375/version
{"Version":"17.09.0-ce","ApiVersion":"1.32","MinAPIVersion":"1.12","GitCommit":"afdb6d4","GoVersion":"go1.8.3","Os":"linux","Arch":"amd64","KernelVersion":"4.9.49-moby","BuildTime":"2017-09-26T22:45:38.000000000+00:00"}```
lclclc (Tue, 31 Oct 2017 03:26:51 GMT):
```dashboard | [2017-10-31 11:22:08,764] DEBUG [modules.host] [host.py:89 create()] - Create host: name=Valiant, worker_api=127.0.0.1:2375, host_type=docker, capacity=2, log=local/, autofill=false, schedulable=false
dashboard | [2017-10-31 11:22:08,767] ERROR [agent.docker.docker_swarm] [docker_swarm.py:152 check_daemon()] - Exception in check_daemon Error while fetching server API version: HTTPConnectionPool(host='127.0.0.1', port=2375): Max retries exceeded with url: /version (Caused by NewConnectionError('
lclclc (Tue, 31 Oct 2017 03:27:35 GMT):
How is the docker_swarm.py trying to connect to the remote api?
baohua (Tue, 31 Oct 2017 03:37:23 GMT):
notice cello service itself is running inside a container
baohua (Tue, 31 Oct 2017 03:37:41 GMT):
hence `127.0.0.1:2375` to him means inside the container, too
baohua (Tue, 31 Oct 2017 03:37:49 GMT):
should point to the public address
lclclc (Tue, 31 Oct 2017 03:41:36 GMT):
I don't see a configuration about it then.
lclclc (Tue, 31 Oct 2017 03:41:50 GMT):
Document didn't mention it.
baohua (Tue, 31 Oct 2017 03:42:52 GMT):
no, this is not about configuration, it's a mis-usage... but maybe u can help add some Q&A?
lclclc (Tue, 31 Oct 2017 03:44:31 GMT):
Oh, I see your point……
lclclc (Tue, 31 Oct 2017 03:45:32 GMT):
It will be helpful to add a note, 127.0.0.1 is the container itself, not the host os
lclclc (Tue, 31 Oct 2017 03:46:38 GMT):
It is good to have this in Q&A, because careless user like me usually didn't realize about.
lclclc (Tue, 31 Oct 2017 03:47:05 GMT):
how can I help with Q&A?
Yixing (Tue, 31 Oct 2017 04:06:27 GMT):
Please help review https://gerrit.hyperledger.org/r/14973
Yixing (Tue, 31 Oct 2017 04:06:30 GMT):
Thanks
baohua (Tue, 31 Oct 2017 04:22:07 GMT):
@hightall pls help review
lclclc (Tue, 31 Oct 2017 06:11:01 GMT):
Why cello's input for host address treat this address illegal: docker.for.mac.localhost:2375
lclclc (Tue, 31 Oct 2017 06:11:33 GMT):
We can only use ip:port pair. Is there any compelling reason for that?
lclclc (Tue, 31 Oct 2017 06:20:28 GMT):
Actually I want to report that from inside the container, the public address of host os is usable, this is known problem of Docker in mac: https://docs.docker.com/docker-for-mac/networking/#known-limitations-use-cases-and-workarounds
lclclc (Tue, 31 Oct 2017 06:20:28 GMT):
Actually I want to report that from inside the container, the public address of host os is not usable, this is known problem of Docker in mac: https://docs.docker.com/docker-for-mac/networking/#known-limitations-use-cases-and-workarounds
lclclc (Tue, 31 Oct 2017 06:20:39 GMT):
https://stackoverflow.com/questions/29598504/how-to-connect-to-service-in-the-host-machine-from-inside-a-docker-container
lclclc (Tue, 31 Oct 2017 06:21:10 GMT):
The workaround for it is ```I WANT TO CONNECT FROM A CONTAINER TO A SERVICE ON THE HOST
The Mac has a changing IP address (or none if you have no network access). From 17.06 onwards our recommendation is to connect to the special Mac-only DNS name docker.for.mac.localhost which will resolve to the internal IP address used by the host.```
lclclc (Tue, 31 Oct 2017 06:22:16 GMT):
If the cello does not support domain:port. I suggest to add a note about Mac that, user has to log into dashbord container to see what ip is for "docker.for.mac.localhost"
lclclc (Tue, 31 Oct 2017 06:22:16 GMT):
If the cello does not support domain:port. I suggest to add a note about Mac that: `user has to log into dashbord container to see what ip is for "docker.for.mac.localhost"`
lclclc (Tue, 31 Oct 2017 06:22:41 GMT):
So user can let cello master to acess worker host in one machine.
lclclc (Tue, 31 Oct 2017 06:22:41 GMT):
So user can let cello master access worker host in one machine.
pancake (Tue, 31 Oct 2017 07:16:40 GMT):
Has joined the channel.
miiiiiyoung (Tue, 31 Oct 2017 08:29:33 GMT):
User-dashboard container has to run in master_node?
miiiiiyoung (Tue, 31 Oct 2017 08:45:23 GMT):
In my user-dashboard page, i can't access smart contract, analytics and store page. : `Cannot GET /dashboard/contract` How can i get this data?
lclclc (Tue, 31 Oct 2017 09:09:13 GMT):
I want to report the second thing, that in the setup scripts:``` 93 echo_b "Add existing user to docker group"
94 sudo usermod -aG docker ${USER}```
lclclc (Tue, 31 Oct 2017 09:09:57 GMT):
actually mac don't have usermod command, and this will always fail.
lclclc (Tue, 31 Oct 2017 09:12:14 GMT):
And in the beginning of this script, if it can't determine the linux destro, it will set the distribution variable to *Unknown*, then there is no recognition of Mac OS yet. For now I think there are special case for mac, it'd better to have some branch to deal with Mac.
hightall (Tue, 31 Oct 2017 09:12:44 GMT):
@miiiiiyoung user-dashboard is developing now. now it's only initial code. the user-dashboard feature will be released at the end of this year. And the user dashboard can be deployed separated.
miiiiiyoung (Tue, 31 Oct 2017 09:18:19 GMT):
Thank you. :-) And, after setting worker node, worker node's network was disconnected almost always. Why does this error occur?
lclclc (Tue, 31 Oct 2017 09:23:27 GMT):
The third thing I want to report is: `sysctl -w net.ipv4.ip_forward=1` does not exist in Mac, either. The equal thing in mac should be `sudo sysctl -w net.inet.ip.forwarding=1
`
lclclc (Tue, 31 Oct 2017 09:23:27 GMT):
The third thing I want to report is: `sysctl -w net.ipv4.ip_forward=1` does not exist in Mac, either. The equal thing in mac should be `sudo sysctl -w net.inet.ip.forwarding=1`
hightall (Tue, 31 Oct 2017 09:23:45 GMT):
I don't know what's mean about disconnected? you mean can not visit worker node? or can not visit fabric network on the worker node? @miiiiiyoung
hightall (Tue, 31 Oct 2017 09:24:37 GMT):
@lclclc great, you can commit a patch.
miiiiiyoung (Tue, 31 Oct 2017 09:26:01 GMT):
all local network is disconnected. It means I can't access any of web site. (ex) google, hyperledger site, and so on)
lclclc (Tue, 31 Oct 2017 09:26:41 GMT):
@hightall Can you brief how to patch this project?
miiiiiyoung (Tue, 31 Oct 2017 09:27:10 GMT):
During using cello project, this error is occured usually.
hightall (Tue, 31 Oct 2017 09:28:39 GMT):
@lclclc First you need submit a jira task at https://jira.hyperledger.org then fork the code from gerrit.hyperledger.org , then submit the patch through git review
baohua (Tue, 31 Oct 2017 09:29:36 GMT):
@lclclc thanks for ur reporting. if u believe, pls file an issue at jira.hyperledger.org, where we collect issues and bugs.
hightall (Tue, 31 Oct 2017 09:30:00 GMT):
@miiiiiyoung is your dns service working fine?
miiiiiyoung (Tue, 31 Oct 2017 09:30:26 GMT):
no, it also not working
lclclc (Tue, 31 Oct 2017 09:30:44 GMT):
You are welcome, actually now I figure out why the /opt/cello/mongo has mount issue, it is also related to this setup script does not recognize MacOS environment. But it is very trival.
lclclc (Tue, 31 Oct 2017 12:56:39 GMT):
@baohua I have create a task here: https://jira.hyperledger.org/browse/CE-162
lclclc (Tue, 31 Oct 2017 12:57:19 GMT):
I don't know if it is valid according to the code of conduct of this community, can you check that task and give some notes about what to do next?
knagware9 (Tue, 31 Oct 2017 17:54:36 GMT):
Has joined the channel.
tongli (Tue, 31 Oct 2017 20:48:42 GMT):
please review this patch set https://gerrit.hyperledger.org/r/#/c/14999/
tongli (Tue, 31 Oct 2017 20:48:55 GMT):
one line change to support latest version of ansible.
tongli (Tue, 31 Oct 2017 20:49:11 GMT):
@hightall @baohua please review https://gerrit.hyperledger.org/r/#/c/14999/ Thanks.
miiiiiyoung (Tue, 31 Oct 2017 23:58:47 GMT):
There is one question. Add a worker_node named docker_host in master_node and add a chain to docker_host, then worker_node is created a block-chain network. So, it means i can deploy a business network archive to this network in worker_node? or something else?
baohua (Wed, 01 Nov 2017 00:24:34 GMT):
@tongli will review, thanks!
baohua (Wed, 01 Nov 2017 00:25:18 GMT):
@miiiiiyoung after u got a chain, then u can feel free to adding chaincode atop it, certainly
tongli (Wed, 01 Nov 2017 01:20:35 GMT):
@baohua thanks.
lclclc (Wed, 01 Nov 2017 02:09:43 GMT):
@baohua Now I can create a chain in my Mac, but I don't see it is exposing its nodes tcp port outside of containers. So how can I talk to the chain, like installing chaincode, deploying business network by composer?
baohua (Wed, 01 Nov 2017 02:10:21 GMT):
are u using cello to create the chain?
lclclc (Wed, 01 Nov 2017 02:11:00 GMT):
Yes, cello's classic size 4 chain.
lclclc (Wed, 01 Nov 2017 02:11:02 GMT):
Clipboard
lclclc (Wed, 01 Nov 2017 02:13:31 GMT):
Clipboard
baohua (Wed, 01 Nov 2017 02:26:59 GMT):
seems sth wrong with ur fabric network, as the container status is "restarting"...
lclclc (Wed, 01 Nov 2017 02:29:37 GMT):
I see, let me check docker log. Maybe it is related to my poor hardware of Mac. I am running Cello master and worker host in same machine.
lclclc (Wed, 01 Nov 2017 02:35:28 GMT):
Can I assume if network is functional, every port can be access from outside the container like sample network works?
lclclc (Wed, 01 Nov 2017 02:37:50 GMT):
A lot of this: ```2017-11-01 02:36:45.004 UTC [main] main -> ERRO 001 Cannot run peer because error when setting up MSP from directory /etc/hyperledger/fabric/msp: err Could not load a valid signer certificate from directory /etc/hyperledger/fabric/msp/signcerts, err stat /etc/hyperledger/fabric/msp/signcerts: no such file or directory```
lclclc (Wed, 01 Nov 2017 02:41:02 GMT):
Clipboard
lclclc (Wed, 01 Nov 2017 02:41:22 GMT):
After inspect the container, I found it is trying to bind this directory to a opt sub-directory, but actually opt does not have that directory, and cello seems don't create that for it. Is that something wrong with cello or my filesystem?
miiiiiyoung (Wed, 01 Nov 2017 02:45:04 GMT):
When I connect my PC to worker_node, all external internet connection except internal network is disconnected. and if i delete the worker_node from the host, the Internet connection is returned. What is this about?
JoshuaRippon (Wed, 01 Nov 2017 13:59:32 GMT):
Has joined the channel.
JoshuaRippon (Wed, 01 Nov 2017 14:02:24 GMT):
Hi guys. I am trying to set up a worker node on a Ubuntu 16.04 server. When I run the command `ExecStart=/usr/bin/dockerd -H fd:// $DOCKER_OPTS` , I get the error `-H: command not found`. Can anyone please help me set up my worker node?
tongli (Wed, 01 Nov 2017 14:58:10 GMT):
@JoshuaRippon , you want to try this process? https://github.com/hyperledger/cello/blob/master/src/agent/ansible/vars/All-in-one.md
tongli (Wed, 01 Nov 2017 14:58:40 GMT):
this ansible agent does pretty much everything for you. if you want to try it, I can certainly help with it.
tongli (Wed, 01 Nov 2017 15:04:38 GMT):
@hightall can you please review this patch and get it in, thanks, https://gerrit.hyperledger.org/r/#/c/14999/
JoshuaRippon (Wed, 01 Nov 2017 15:05:26 GMT):
Hi @tongli . I would really like to run a Fabric network on multiple hosts. This is how I stumbled across Cello
JoshuaRippon (Wed, 01 Nov 2017 15:06:43 GMT):
Will following the process you linked allow me to do this?
tongli (Wed, 01 Nov 2017 15:09:13 GMT):
absolutely. the ansible agent was designed to support that natively. all-in-one is just a slight different configuration to allow people to get things up running very quickly when people do not have multiple nodes.
tongli (Wed, 01 Nov 2017 15:09:30 GMT):
see this doc. https://github.com/hyperledger/cello/blob/master/src/agent/ansible/vars/README.md
tongli (Wed, 01 Nov 2017 15:09:49 GMT):
and this doc. https://github.com/hyperledger/cello/blob/master/src/agent/ansible/README.md
tongli (Wed, 01 Nov 2017 15:10:32 GMT):
this doc explains a bit what all the configuration files mean https://github.com/hyperledger/cello/blob/master/src/agent/ansible/vars/README.md
tongli (Wed, 01 Nov 2017 15:10:49 GMT):
but you can always ask questions here on the ansible agent, I will be happy to help.
JoshuaRippon (Wed, 01 Nov 2017 15:11:39 GMT):
So I am currently using Docker, is ansible an alternative?
tongli (Wed, 01 Nov 2017 15:12:06 GMT):
no, ansible is just a tool to get things lay out on multiple machines.
tongli (Wed, 01 Nov 2017 15:12:16 GMT):
you will be still using docker.
tongli (Wed, 01 Nov 2017 15:12:46 GMT):
docker is your container engine. ansible is just a tool to place and configure things at right place.
tongli (Wed, 01 Nov 2017 15:13:46 GMT):
once your fabric network got setup, ansible is out of the way.
JoshuaRippon (Wed, 01 Nov 2017 15:26:45 GMT):
@tongli I have got up to the stage where I have to create runhosts.tpl
JoshuaRippon (Wed, 01 Nov 2017 15:26:59 GMT):
cloud ansible_host=127.0.0.1 ansible_python_interpreter=python
$ip private_ip=$ip public_ip=$ip inter_name=fabric001
[allnodes]
$ip
[etcdnodes]
$ip
[builders]
$ip
tongli (Wed, 01 Nov 2017 15:27:11 GMT):
yeah
tongli (Wed, 01 Nov 2017 15:27:23 GMT):
run the little script
tongli (Wed, 01 Nov 2017 15:27:45 GMT):
I think you need to go to directory ~/cello/src/agent/ansible to run the two commands.
JoshuaRippon (Wed, 01 Nov 2017 15:27:57 GMT):
but do I have to replace all the $ip
tongli (Wed, 01 Nov 2017 15:28:01 GMT):
the instruction needs to be a little more clear on changing the directory.
tongli (Wed, 01 Nov 2017 15:28:05 GMT):
yes.
tongli (Wed, 01 Nov 2017 15:28:12 GMT):
the little script does that for you.
tongli (Wed, 01 Nov 2017 15:29:02 GMT):
create that tpl file and run these two commands in ~/cello/src/agent/ansible directory
tongli (Wed, 01 Nov 2017 15:29:07 GMT):
```ipaddr=$(ip -4 addr show | awk -F '/' '/inet / {print $1}' | grep -v '127.0.0.1' | awk -F ' ' '{print $2;exit}')
sed "s/\$ip/$ipaddr/g" run/runhosts.tpl > run/runhosts```
tongli (Wed, 01 Nov 2017 15:29:27 GMT):
that will produce a new file in run directory named runhosts
tongli (Wed, 01 Nov 2017 15:29:58 GMT):
if you replace the $ip manually, then you do not have to run the commands.
tongli (Wed, 01 Nov 2017 15:30:30 GMT):
also, I think that the instruction missed how to setup ssh key in .ssh/authorized_key
tongli (Wed, 01 Nov 2017 15:30:46 GMT):
if you do not have that, then you can not use ssh to connect, the process will fail.
tongli (Wed, 01 Nov 2017 15:30:57 GMT):
so you need to do that as well.
tongli (Wed, 01 Nov 2017 15:31:04 GMT):
here is the process.
tongli (Wed, 01 Nov 2017 15:31:30 GMT):
1. in your ~/.ssh directory , create a file named authorized_keys
tongli (Wed, 01 Nov 2017 15:31:47 GMT):
2. Place the content of ~/.ssh/fd.pub file in that file and save it.
tongli (Wed, 01 Nov 2017 15:33:28 GMT):
once you have done that before you start step 4, log out and log back in and make sure that you ssh-agent is running, then you can go on step 4.
tongli (Wed, 01 Nov 2017 15:33:42 GMT):
I will make the steps a bit more clear in next patch set.
tongli (Wed, 01 Nov 2017 15:34:38 GMT):
@baohua @hightall can one of you merge this? https://gerrit.hyperledger.org/r/#/c/14999/ thanks.
JoshuaRippon (Wed, 01 Nov 2017 15:35:49 GMT):
I already have an authorized_keys file. Is it ok to follow your first steps. Will it wipe important data?
tongli (Wed, 01 Nov 2017 15:36:02 GMT):
no.
tongli (Wed, 01 Nov 2017 15:36:28 GMT):
if you already have authorized_keys, then you just need to add the created fd.pub at the end of the file.
tongli (Wed, 01 Nov 2017 15:36:56 GMT):
or best yet, skip creating the new ssh key pair in step #1, just go ahead using your existing key.
tongli (Wed, 01 Nov 2017 15:37:16 GMT):
but you will have to change couple files in vars directory as the instruction stated.
tongli (Wed, 01 Nov 2017 15:37:47 GMT):
authorized_keys file simply have bunch of public keys, it is a text file, you can edit it.
tongli (Wed, 01 Nov 2017 15:38:02 GMT):
it would not mess up anything.
JoshuaRippon (Wed, 01 Nov 2017 15:38:06 GMT):
sweet
tongli (Wed, 01 Nov 2017 15:40:21 GMT):
@hightall thanks for merging the patch set, now we are supporting latest version of ansible and as old as 2.3.0.0
JoshuaRippon (Wed, 01 Nov 2017 15:46:55 GMT):
@tongli is it literally a copy/paste to the end of authorized_keys? or do I remove ssh-rsa before pasting?
JoshuaRippon (Wed, 01 Nov 2017 15:47:08 GMT):
sorry for these newb questions!
tongli (Wed, 01 Nov 2017 15:48:19 GMT):
@JoshuaRippon it is literally copy and paste to the end of the file. true statement.
tongli (Wed, 01 Nov 2017 15:48:33 GMT):
should be on its own line.
tongli (Wed, 01 Nov 2017 15:48:47 GMT):
each public key should occupy one line.
tongli (Wed, 01 Nov 2017 15:49:05 GMT):
I suspect that you have quite few keys already there.
tongli (Wed, 01 Nov 2017 15:49:16 GMT):
and I hope this is a ubuntu 16.04 system.
tongli (Wed, 01 Nov 2017 15:49:31 GMT):
if you are using different OS, your miles will be probably very different.
tongli (Wed, 01 Nov 2017 15:49:43 GMT):
may not even work.
JoshuaRippon (Wed, 01 Nov 2017 15:51:02 GMT):
yes ubuntu 16.04
tongli (Wed, 01 Nov 2017 15:51:38 GMT):
k.
JoshuaRippon (Wed, 01 Nov 2017 15:56:22 GMT):
when running `-i run/runhosts -e "mode=apply env=vb" initcluster.yml --skip-tags="resetconn"` I get error `ERROR! Unexpected Exception: 'module' object has no attribute 'SSL_ST_INIT'`
JoshuaRippon (Wed, 01 Nov 2017 15:56:42 GMT):
Is this to do with using a different userID?
kayadhami (Wed, 01 Nov 2017 15:58:55 GMT):
Has joined the channel.
tongli (Wed, 01 Nov 2017 16:01:46 GMT):
hmmm.
tongli (Wed, 01 Nov 2017 16:02:04 GMT):
do you have ssh-agent running? in step #1
JoshuaRippon (Wed, 01 Nov 2017 16:04:24 GMT):
I ran `eval $(ssh-agent -s)` and got `Agent pid 94714`
tongli (Wed, 01 Nov 2017 16:04:51 GMT):
```eval $(ssh-agent -s) && ssh-add ~/.ssh/fd```
tongli (Wed, 01 Nov 2017 16:05:06 GMT):
do not miss the second part.
JoshuaRippon (Wed, 01 Nov 2017 16:05:52 GMT):
Yes I did that
JoshuaRippon (Wed, 01 Nov 2017 16:06:00 GMT):
still same error
tongli (Wed, 01 Nov 2017 16:06:57 GMT):
hmmm. the full error log?
tongli (Wed, 01 Nov 2017 16:07:33 GMT):
did you change the authorized_keys file?
tongli (Wed, 01 Nov 2017 16:07:44 GMT):
if you did, you will have to log out and log back in.
JoshuaRippon (Wed, 01 Nov 2017 16:08:59 GMT):
Just logged out and in again
tongli (Wed, 01 Nov 2017 16:09:01 GMT):
if you did not, and use your own key, then you will need to change vars/vb.yml file.
JoshuaRippon (Wed, 01 Nov 2017 16:09:02 GMT):
still same error
tongli (Wed, 01 Nov 2017 16:09:27 GMT):
when you log back in, you need to do this again
tongli (Wed, 01 Nov 2017 16:09:32 GMT):
```eval $(ssh-agent -s) && ssh-add ~/.ssh/fd```
JoshuaRippon (Wed, 01 Nov 2017 16:10:41 GMT):
did that, same error....
tongli (Wed, 01 Nov 2017 16:11:29 GMT):
ok. what is the full error msg?
tongli (Wed, 01 Nov 2017 16:12:25 GMT):
and , if your userid is not ubuntu, you need to change vars/vb.yml file as well.
JoshuaRippon (Wed, 01 Nov 2017 16:13:01 GMT):
2017-11-01 16:12:09,106 p=96148 u=ubuntu | ERROR! Unexpected Exception: 'module' object has no attribute 'SSL_ST_INIT'
2017-11-01 16:12:09,106 p=96148 u=ubuntu | to see the full traceback, use -vvv
2017-11-01 16:12:09,107 p=96148 u=ubuntu | the full traceback was:
Traceback (most recent call last):
File "/usr/local/bin/ansible-playbook", line 109, in
tongli (Wed, 01 Nov 2017 16:13:50 GMT):
hmm. I wonder if your ansible version is not correct.
tongli (Wed, 01 Nov 2017 16:13:55 GMT):
do this command.
tongli (Wed, 01 Nov 2017 16:14:03 GMT):
```ansible --version```
JoshuaRippon (Wed, 01 Nov 2017 16:14:40 GMT):
ansible 2.3.0.0
config file =
configured module search path = Default w/o overrides
python version = 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609]
tongli (Wed, 01 Nov 2017 16:14:55 GMT):
hmmm. that looks ok to me.
tongli (Wed, 01 Nov 2017 16:15:16 GMT):
you did this as well I assume ```sudo apt-get install python-dev python-pip libssl-dev libffi-dev -y```
JoshuaRippon (Wed, 01 Nov 2017 16:15:20 GMT):
my usierid is ubuntu
JoshuaRippon (Wed, 01 Nov 2017 16:15:53 GMT):
yes
tongli (Wed, 01 Nov 2017 16:16:56 GMT):
hmmm. looks like that you pyOpenSSL may be old. I am not sure. let me check few things
tongli (Wed, 01 Nov 2017 16:18:49 GMT):
do the following
tongli (Wed, 01 Nov 2017 16:19:25 GMT):
run the following command to check your pyOpenSSL
tongli (Wed, 01 Nov 2017 16:19:28 GMT):
python
tongli (Wed, 01 Nov 2017 16:19:33 GMT):
import OpenSSL
tongli (Wed, 01 Nov 2017 16:19:39 GMT):
OpenSSL.__version__
tongli (Wed, 01 Nov 2017 16:19:45 GMT):
this should give you a number.
tongli (Wed, 01 Nov 2017 16:19:56 GMT):
mine is 16.0.0 which worked fine.
tongli (Wed, 01 Nov 2017 16:22:03 GMT):
The error is reported here. https://github.com/MobSF/Mobile-Security-Framework-MobSF/issues/293
JoshuaRippon (Wed, 01 Nov 2017 16:22:04 GMT):
when I do `import OpenSSL` I get `'module' object has no attribute 'SSL_ST_INIT'`
tongli (Wed, 01 Nov 2017 16:22:26 GMT):
hmmm. you can not even do that in the python prompt?
JoshuaRippon (Wed, 01 Nov 2017 16:22:33 GMT):
correct
tongli (Wed, 01 Nov 2017 16:22:58 GMT):
hmm, your environment is having some issues.
tongli (Wed, 01 Nov 2017 16:25:01 GMT):
maybe try this command
tongli (Wed, 01 Nov 2017 16:25:05 GMT):
```sudo pip install pyOpenSSL --upgrade```
tongli (Wed, 01 Nov 2017 16:25:49 GMT):
or this
tongli (Wed, 01 Nov 2017 16:25:53 GMT):
```sudo pip install pyOpenSSL==16.0.0```
tongli (Wed, 01 Nov 2017 16:26:05 GMT):
this should upgrade you to a version which I know works.
tongli (Wed, 01 Nov 2017 16:26:19 GMT):
if you can not even do these things, then I am really running out of ideas.
tongli (Wed, 01 Nov 2017 16:26:36 GMT):
normally, I would recommend using a clean ubuntu 16.04 system.
tongli (Wed, 01 Nov 2017 16:27:06 GMT):
if your system can not do a simple python import, then something is really wrong.
JoshuaRippon (Wed, 01 Nov 2017 16:28:07 GMT):
it looks like my pip install isn't working properly
JoshuaRippon (Wed, 01 Nov 2017 16:28:21 GMT):
any time I try to pip install something I get the same error
JoshuaRippon (Wed, 01 Nov 2017 16:28:41 GMT):
should I try to reinstall pip
JoshuaRippon (Wed, 01 Nov 2017 16:28:44 GMT):
?
tongli (Wed, 01 Nov 2017 16:29:22 GMT):
did you try this?
tongli (Wed, 01 Nov 2017 16:29:25 GMT):
```sudo apt-get install python-dev python-pip```
tongli (Wed, 01 Nov 2017 16:29:54 GMT):
after that, do this
tongli (Wed, 01 Nov 2017 16:30:12 GMT):
``` sudo pip install pip --upgrade```, hope that fixes your issue.
JoshuaRippon (Wed, 01 Nov 2017 16:31:12 GMT):
didn't work unfortunately
tongli (Wed, 01 Nov 2017 16:34:01 GMT):
your python env is not doing well.
JoshuaRippon (Wed, 01 Nov 2017 16:34:17 GMT):
hahaha
tongli (Wed, 01 Nov 2017 16:34:18 GMT):
probably fix your python issue.
tongli (Wed, 01 Nov 2017 16:34:26 GMT):
first.
tongli (Wed, 01 Nov 2017 16:34:54 GMT):
I would suggest to have a relatively clean system for this.
JoshuaRippon (Wed, 01 Nov 2017 16:49:12 GMT):
@tongli I removed OpenSSL and reinstalled it
JoshuaRippon (Wed, 01 Nov 2017 16:49:30 GMT):
not getting that error any more!
JoshuaRippon (Wed, 01 Nov 2017 17:08:58 GMT):
I ran `ansible-playbook -i run/runhosts -e "mode=apply env=vb" initcluster.yml --skip-tags="resetconn"` which was fine
JoshuaRippon (Wed, 01 Nov 2017 17:10:09 GMT):
However when I run `ansible-playbook -i run/runhosts -e "mode=apply env=bc2nd" setupfabric.yml` I get the error ```TASK [Make targets] *************************************************************************************************************************************************************************************************************************
changed: [10.0.4.7] => (item=cryptogen)
failed: [10.0.4.7] (item=configtxgen) => {"changed": true, "cmd": "make configtxgen >> /opt/gopath/bc2nd/build.log", "delta": "0:00:01.466978", "end": "2017-11-01 17:08:17.536043", "failed": true, "item": "configtxgen", "rc": 2, "start": "2017-11-01 17:08:16.069065", "stderr": "go install vendor/golang_org/x/crypto/chacha20poly1305: open /opt/go/pkg/linux_amd64/vendor/golang_org/x/crypto/chacha20poly1305.a: permission denied\nmake: *** [build/bin/configtxgen] Error 1", "stderr_lines": ["go install vendor/golang_org/x/crypto/chacha20poly1305: open /opt/go/pkg/linux_amd64/vendor/golang_org/x/crypto/chacha20poly1305.a: permission denied", "make: *** [build/bin/configtxgen] Error 1"], "stdout": "", "stdout_lines": []}
to retry, use: --limit @/home/ubuntu/hyperledger/cello/src/agent/ansible/setupfabric.retry```
tongli (Wed, 01 Nov 2017 17:30:04 GMT):
hmmm. @JoshuaRippon seems that you can not run make to build the cryptogen.
JoshuaRippon (Wed, 01 Nov 2017 17:31:02 GMT):
Hmm why might that be?
tongli (Wed, 01 Nov 2017 17:31:35 GMT):
can your user ubuntu do this?
tongli (Wed, 01 Nov 2017 17:31:42 GMT):
```sudo su```
JoshuaRippon (Wed, 01 Nov 2017 17:32:06 GMT):
yup
tongli (Wed, 01 Nov 2017 17:32:08 GMT):
without promote password, actually, that might not be the problem.
tongli (Wed, 01 Nov 2017 17:32:49 GMT):
you may try to go to fabric directory, and run make configtxgen and see what happens.
tongli (Wed, 01 Nov 2017 17:34:24 GMT):
can you check who owns this directory?
JoshuaRippon (Wed, 01 Nov 2017 17:34:50 GMT):
`make: Nothing to be done for 'configtxgen'`
tongli (Wed, 01 Nov 2017 17:34:51 GMT):
can you check who owns this direct /opt/go/pkg
tongli (Wed, 01 Nov 2017 17:37:06 GMT):
hmm, you should have a directory like this /opt/gopath/bc2nd/src/github.com/hyperledger/fabric
tongli (Wed, 01 Nov 2017 17:37:21 GMT):
did you see a directory like that on your system?
JoshuaRippon (Wed, 01 Nov 2017 17:37:41 GMT):
It is owned by root
tongli (Wed, 01 Nov 2017 17:38:11 GMT):
then there should be build/bin, in that directory, you should have configtxgen amd cryptogen
tongli (Wed, 01 Nov 2017 17:38:17 GMT):
which should be just built.
JoshuaRippon (Wed, 01 Nov 2017 17:38:20 GMT):
yes I have that
tongli (Wed, 01 Nov 2017 17:39:16 GMT):
even configtxgen?
tongli (Wed, 01 Nov 2017 17:39:22 GMT):
but it said that it failed
JoshuaRippon (Wed, 01 Nov 2017 17:39:39 GMT):
it has cryptogen but not configtxgen in it
tongli (Wed, 01 Nov 2017 17:39:56 GMT):
right.
JoshuaRippon (Wed, 01 Nov 2017 17:40:09 GMT):
it only has cryptogen, no other files
tongli (Wed, 01 Nov 2017 17:40:36 GMT):
can you try ```make configtxgen``` in this directory /opt/gopath/bc2nd/src/github.com/hyperledger/fabric
JoshuaRippon (Wed, 01 Nov 2017 17:41:24 GMT):
```can't load package: package github.com/hyperledger/fabric/core/chaincode/shim: cannot find package "github.com/hyperledger/fabric/core/chaincode/shim" in any of:
/usr/lib/go-1.6/src/github.com/hyperledger/fabric/core/chaincode/shim (from $GOROOT)
($GOPATH not set)
find: ‘/src/github.com/hyperledger/fabric/core/chaincode/shim’: No such file or directory
build/bin/configtxgen
CGO_CFLAGS=" " GOBIN=/opt/gopath/bc2nd/src/github.com/hyperledger/fabric/build/bin go install -tags "experimental nopkcs11" -ldflags "-X github.com/hyperledger/fabric/common/tools/configtxgen/metadata.Version=1.1.0-snapshot-a47bf65" github.com/hyperledger/fabric/common/tools/configtxgen
can't load package: package github.com/hyperledger/fabric/common/tools/configtxgen: cannot find package "github.com/hyperledger/fabric/common/tools/configtxgen" in any of:
/usr/lib/go-1.6/src/github.com/hyperledger/fabric/common/tools/configtxgen (from $GOROOT)
($GOPATH not set)
Makefile:233: recipe for target 'build/bin/configtxgen' failed
make: *** [build/bin/configtxgen] Error 1
```
tongli (Wed, 01 Nov 2017 17:41:48 GMT):
yeah, seems same error.
tongli (Wed, 01 Nov 2017 17:41:55 GMT):
check your env varilable
tongli (Wed, 01 Nov 2017 17:42:03 GMT):
```echo $GOROOT```
tongli (Wed, 01 Nov 2017 17:42:11 GMT):
```echo $GOPATH```
JoshuaRippon (Wed, 01 Nov 2017 17:42:26 GMT):
nothing in either
tongli (Wed, 01 Nov 2017 17:42:42 GMT):
hmmm.
tongli (Wed, 01 Nov 2017 17:42:44 GMT):
do this.
tongli (Wed, 01 Nov 2017 17:42:54 GMT):
export GOPATH=/opt/gopath
tongli (Wed, 01 Nov 2017 17:43:00 GMT):
export GOROOT=/opt/go
JoshuaRippon (Wed, 01 Nov 2017 17:43:37 GMT):
Ok done
tongli (Wed, 01 Nov 2017 17:43:45 GMT):
then run make configtxgen again
JoshuaRippon (Wed, 01 Nov 2017 17:44:01 GMT):
I remember defining GOPATH before, do these wipe when I log out?
tongli (Wed, 01 Nov 2017 17:44:14 GMT):
in this dir /opt/gopath/bc2nd/src/github.com/hyperledger/fabric
JoshuaRippon (Wed, 01 Nov 2017 17:44:16 GMT):
na same error
tongli (Wed, 01 Nov 2017 17:44:37 GMT):
export just does it for the current session.
JoshuaRippon (Wed, 01 Nov 2017 17:45:21 GMT):
I have the configtxgen file but not in this directory
JoshuaRippon (Wed, 01 Nov 2017 17:45:35 GMT):
should I just copy it over?
tongli (Wed, 01 Nov 2017 17:46:11 GMT):
no, it won't help
tongli (Wed, 01 Nov 2017 17:46:28 GMT):
you have go installed in different directory, that messed up the go build.
JoshuaRippon (Wed, 01 Nov 2017 17:49:00 GMT):
I installed go for go-ipfs a while back
tongli (Wed, 01 Nov 2017 17:49:19 GMT):
sigh! yeah. a lot of conflict.
tongli (Wed, 01 Nov 2017 17:49:28 GMT):
these kind of problems are hard to fix.
JoshuaRippon (Wed, 01 Nov 2017 17:50:03 GMT):
I have a different server, maybe I will try launch it on that one!!
tongli (Wed, 01 Nov 2017 17:50:29 GMT):
make sure it is really clean.
tongli (Wed, 01 Nov 2017 17:50:42 GMT):
no go installed.
tongli (Wed, 01 Nov 2017 17:50:53 GMT):
you can install docker if you want to.
tongli (Wed, 01 Nov 2017 17:51:10 GMT):
before you do anything on that machine, you can simply do this one thing.
tongli (Wed, 01 Nov 2017 17:51:21 GMT):
```apt update```
tongli (Wed, 01 Nov 2017 17:51:30 GMT):
```apt install docker.io```
tongli (Wed, 01 Nov 2017 17:51:36 GMT):
this will help
JoshuaRippon (Wed, 01 Nov 2017 17:51:44 GMT):
So I guess I should find out what the end result will be once I have it up and running. Will I be able to simply edit that runhosts file to add multiple hosts to the network
tongli (Wed, 01 Nov 2017 17:51:44 GMT):
make sure that go lang is not on it.
tongli (Wed, 01 Nov 2017 17:52:00 GMT):
exactly.
tongli (Wed, 01 Nov 2017 17:52:34 GMT):
runhosts is considered as a configuration file.
JoshuaRippon (Wed, 01 Nov 2017 17:53:05 GMT):
Damn I have Go on this one too haha
tongli (Wed, 01 Nov 2017 17:53:12 GMT):
you will also need to change couple of other files in vars directory to make it working.
tongli (Wed, 01 Nov 2017 17:53:24 GMT):
for multiple nodes.
tongli (Wed, 01 Nov 2017 17:53:34 GMT):
can you remove it before you go on?
tongli (Wed, 01 Nov 2017 17:53:50 GMT):
I normally just install VirtualBox and have a VM running on VirtualBox.
tongli (Wed, 01 Nov 2017 17:53:57 GMT):
each time I get a brand new clean system.
JoshuaRippon (Wed, 01 Nov 2017 17:54:33 GMT):
Ok, what I would like to do is have a web application where an organisation can choose a number of other organisations to start a network with.
tongli (Wed, 01 Nov 2017 17:55:11 GMT):
to setup a fabric network for their own organization?
JoshuaRippon (Wed, 01 Nov 2017 17:57:34 GMT):
To set up a fabric network that includes the other organisations they wish to rade with
JoshuaRippon (Wed, 01 Nov 2017 17:57:37 GMT):
trade*
lclclc (Thu, 02 Nov 2017 03:02:37 GMT):
Hi, now I can set up cello worker locally, but all peers' ports are mixed up
lclclc (Thu, 02 Nov 2017 03:03:12 GMT):
Clipboard
lclclc (Thu, 02 Nov 2017 03:03:28 GMT):
How can I distinguish which port is a peer's request port
lclclc (Thu, 02 Nov 2017 03:04:52 GMT):
which port is event port?
MohammadObaid (Thu, 02 Nov 2017 14:07:49 GMT):
Hey when I run ansible-playbook -e "mode=apply" aws.yml I am getting error could not locate file in lookup: /home/ubuntu/.ssh/fd.pub . Is this due to aws access key settings or local problem. Any clue or trick to solve this?
MohammadObaid (Thu, 02 Nov 2017 14:07:49 GMT):
Hey when I run `ansible-playbook -e "mode=apply" aws.yml` I am getting error could not locate file in lookup: `/home/ubuntu/.ssh/fd.pub` . Is this due to aws access key settings or local problem. Any clue or trick to solve this?
MohammadObaid (Thu, 02 Nov 2017 14:07:49 GMT):
Hey when I run `ansible-playbook -e "mode=apply" aws.yml` I am getting `error could not locate file in lookup: /home/ubuntu/.ssh/fd.pub` . Is this due to aws access key settings or local problem. Any clue or trick to solve this?
sk (Thu, 02 Nov 2017 14:21:10 GMT):
Has left the channel.
ajksharma (Thu, 02 Nov 2017 16:01:14 GMT):
Has joined the channel.
tongli (Thu, 02 Nov 2017 16:59:10 GMT):
@baohua @hightall please review this patch, only docs https://gerrit.hyperledger.org/r/#/c/15133/ to improve the all in one document and also fixed the missing license statement in the main agent README.md file.
ajksharma (Thu, 02 Nov 2017 18:23:54 GMT):
Hi @tongli I read your conversation with @JoshuaRippon
I too am trying to run nodes on different hosts. But this is only for the Hyperledger Marbles demo.
The idea is to make sure if one host fails, the blockchain is not affected as there are supporting nodes on n number of hosts.
I have installed Cello on one host(a) and the marbles demo on another host(b).
I am trying to add the marbles host(b) to cello host(a) with docker port 2375
Is this the way to do it?
Could I add more nodes on other hosts(c&d) ?
Thanks
ajksharma (Thu, 02 Nov 2017 18:23:54 GMT):
Hi @tongli I read your conversation with @JoshuaRippon
I too am trying to run nodes on different hosts. But this is only for the Hyperledger Marbles demo.
The idea is to make sure if one host fails, the blockchain is not affected as there are supporting nodes on n number of hosts.
I have installed Cello on one host(a) and the marbles demo on another host(b).
I am trying to add the marbles host(b) to cello host(a) with docker port 2375
Is this the way to do it?
Could I add more nodes on other hosts(c&d) and the marble demos works with all these hosts connected together?
Sorry for these newbee questions.
Thanks
tongli (Thu, 02 Nov 2017 18:33:03 GMT):
@ajksharma actually I am not really sure. I think the way you use cello is different from what the ansible agent does.
tongli (Thu, 02 Nov 2017 18:33:24 GMT):
@ajksharma cello ansible agent sets up multiple node fabric network in one go.
tongli (Thu, 02 Nov 2017 18:33:36 GMT):
regardless it is 3 nodes or 300 nodes.
tongli (Thu, 02 Nov 2017 18:34:17 GMT):
currently the agent can not add new nodes after initial fabric network started running. we need to add that function.
tongli (Thu, 02 Nov 2017 18:34:59 GMT):
with multiple nodes in fabric network, multiple peers, orderers, one peer goes down should not affect the availability of your fabric network.
tongli (Thu, 02 Nov 2017 18:36:31 GMT):
btw, ansible agent uses kafka for orderers.
tongli (Thu, 02 Nov 2017 18:36:45 GMT):
kafka and zookeeper be exact.
ajksharma (Thu, 02 Nov 2017 18:39:06 GMT):
Thanks @tongli . So in one go I can add n numbers of nodes on n number of hosts?
I have 5 hosts with me. Marbles demo is working fine on one host.
If i can't add more nodes, can I run the ansible script with marbles somehow and create this multi-node setup?
Question is , can the marbles demo be run on multiple nodes on multiple hosts?
Its on github and easy to setup on one host.
ajksharma (Thu, 02 Nov 2017 18:39:06 GMT):
Thanks @tongli . So in one go I can add n numbers of nodes on n number of hosts?
I have 5 hosts with me. Marbles demo is working fine on one host.
If i can't add more nodes, can I run the ansible script with marbles somehow and create this multi-node setup?
Question is , can the marbles demo be run on multiple nodes on multiple hosts?
Marbles is on github and easy to setup on one host.
tongli (Thu, 02 Nov 2017 18:40:04 GMT):
@ajksharma ansible agent can not add nodes to an existing fabric network.
tongli (Thu, 02 Nov 2017 18:40:25 GMT):
it stands up multiple node fabric network.
ajksharma (Thu, 02 Nov 2017 18:40:41 GMT):
I understand that.
I am ready to run on a clean state
tongli (Thu, 02 Nov 2017 18:41:42 GMT):
right. in that case, say you have 5 machines, you can use cello to stand up a 5 node fabric network with multiple orgs, multiple peers and orderers in each org.
tongli (Thu, 02 Nov 2017 18:42:23 GMT):
the easiest to do this if you have AWS account, you can use cello against AWS cloud.
tongli (Thu, 02 Nov 2017 18:42:55 GMT):
cello will do provisioning of VMs, establish secuirty rules, and eventually get your 5 node fabric network up running.
ajksharma (Thu, 02 Nov 2017 18:43:07 GMT):
yes, I have an aws account and have 5 hosts already made.
ajksharma (Thu, 02 Nov 2017 18:43:28 GMT):
So cello would spine "new" VMs?
tongli (Thu, 02 Nov 2017 18:43:31 GMT):
https://github.com/hyperledger/cello/tree/master/src/agent/ansible
tongli (Thu, 02 Nov 2017 18:43:52 GMT):
yes, Cello will spin up new VMs.
tongli (Thu, 02 Nov 2017 18:44:41 GMT):
```1. make changes to vars/aws.yml according to your aws cloud
2. export AWS_SECRET_KEY="your secret key of your aws account"
3. To stand up the fabric network::
ansible-playbook -e "mode=apply" aws.yml
4. To tear down the fabric network::
ansible-playbook -e "mode=destroy" aws.yml```
tongli (Thu, 02 Nov 2017 18:45:40 GMT):
if you have a aws account, and allow you to create new VMs, then you can run cello ansible agent from your own laptop.
tongli (Thu, 02 Nov 2017 18:46:09 GMT):
your own laptop will serve as an ansible controller.
tongli (Thu, 02 Nov 2017 18:47:40 GMT):
@ajksharma please also look at this doc which deals with aws cloud. https://github.com/hyperledger/cello/tree/master/src/agent/ansible/roles/cloud_aws
tongli (Thu, 02 Nov 2017 18:48:26 GMT):
ansible aws cloud module requires boto and boto3, please install using this command before you run
tongli (Thu, 02 Nov 2017 18:48:38 GMT):
```sudo pip install boto boto3```
tongli (Thu, 02 Nov 2017 18:48:50 GMT):
that is only required on your ansible controller.
tongli (Thu, 02 Nov 2017 18:49:26 GMT):
where you actually run the ansible command, it is not needed on actual AWS VMs
ajksharma (Thu, 02 Nov 2017 18:49:52 GMT):
great, i would try that @tongli
Once I set that up, how do I up the marbles demo https://github.com/IBM-Blockchain/marbles on this network?
Can I use an
ajksharma (Thu, 02 Nov 2017 18:49:52 GMT):
great, i would try that @tongli
Once I set that up, how do I up the marbles demo https://github.com/IBM-Blockchain/marbles on this network?
tongli (Thu, 02 Nov 2017 18:50:12 GMT):
ansible agent does not setup any apps.
tongli (Thu, 02 Nov 2017 18:50:39 GMT):
I would suggest that you pick one of your AWS VMs and get your actual app up running.
tongli (Thu, 02 Nov 2017 18:51:39 GMT):
I will be really happy to see a patch to cello ansible agent for deploying marble app on top of the fabric network which cello ansible agent sets up.
ajksharma (Thu, 02 Nov 2017 18:52:16 GMT):
yep, I would select one VM and install marbles.
tongli (Thu, 02 Nov 2017 18:52:42 GMT):
right. these VMs will have a local DNS services setup.
ajksharma (Thu, 02 Nov 2017 18:52:53 GMT):
but then do I have to install marbles on all VMs?
tongli (Thu, 02 Nov 2017 18:52:56 GMT):
you will be able to refer their dns name for access.
tongli (Thu, 02 Nov 2017 18:53:23 GMT):
that I do not know. I would assume no. your app just need to access a fabric network.
tongli (Thu, 02 Nov 2017 18:54:07 GMT):
should not matter how big the fabric network is or if runs cross multiple nodes.
ajksharma (Thu, 02 Nov 2017 18:54:10 GMT):
yes, If I could n number of nodes working for the marbles demo, that is teh goal.
tongli (Thu, 02 Nov 2017 18:54:52 GMT):
I would think it depends how marbles demo were setup. can marbles demo app be spread out to multiple nodes?
tongli (Thu, 02 Nov 2017 18:55:20 GMT):
it should not have anything to do with how fabric network gets setup or how many nodes.
ajksharma (Thu, 02 Nov 2017 18:56:09 GMT):
I don't know. But I hope it doesn't matter.
ajksharma (Thu, 02 Nov 2017 18:56:31 GMT):
The chain-code would be one one VM.
tongli (Thu, 02 Nov 2017 18:57:03 GMT):
chaincode can be installed on many peers.
ajksharma (Thu, 02 Nov 2017 18:57:53 GMT):
if chaincode is absent on other hosts, the blockchain wouldn't run? right?
tongli (Thu, 02 Nov 2017 18:58:59 GMT):
I think it is per peer, not host
tongli (Thu, 02 Nov 2017 18:59:10 GMT):
one host can run multiple peers.
ajksharma (Thu, 02 Nov 2017 19:00:49 GMT):
yep, so maybe install the chaincode on all peers.
I read on my post that docker swarm works.
Though I am not sure how to add more nodes to the fabric network
ajksharma (Thu, 02 Nov 2017 19:00:49 GMT):
yep, so maybe install the chaincode on all peers.
I read on one post that docker swarm works.
Though I am not sure how to add more nodes to the fabric network
ajksharma (Thu, 02 Nov 2017 19:00:49 GMT):
yep, so maybe install the chaincode on all peers.
I read on one post that docker swarm works in this scenario.
Though I am not sure how to add more nodes to the fabric network
ajksharma (Thu, 02 Nov 2017 19:10:38 GMT):
@tongli though I know golang and was thinking that by now (3 weeks) I would be able to design and code hyperledger networks, I am still stuck in configuration issues.
What is a good resource or is there a way that this environment becomes easy to handle and with a little hard work, bottle necks could be overcome?
Do I need to see the guts of the system to understand more?
tongli (Thu, 02 Nov 2017 19:24:17 GMT):
@ajksharma are you asking how you can get a deep understanding of fabric?
ajksharma (Thu, 02 Nov 2017 19:25:24 GMT):
yes, I get stuck all the time and lack of knowledge of Hyperledger Fabric internal may be the reason. Though I am not sure.
ajksharma (Thu, 02 Nov 2017 19:25:24 GMT):
yes, I get stuck all the time and lack of knowledge of Hyperledger Fabric internals may be the reason. Though I am not sure.
tongli (Thu, 02 Nov 2017 19:25:54 GMT):
I am still learning fabric myself though I have been working on this thing for 5 months.
ajksharma (Thu, 02 Nov 2017 19:26:13 GMT):
10 + yrs programming experience and enough AWS knowledge.
tongli (Thu, 02 Nov 2017 19:26:34 GMT):
I mostly just read the source code and ask questions in the community.
tongli (Thu, 02 Nov 2017 19:27:10 GMT):
yeah, I am sure you can with that kind of experiences.
ajksharma (Thu, 02 Nov 2017 19:28:45 GMT):
Would try that.
Would let you know how this goes. Thanks!
tongli (Fri, 03 Nov 2017 13:12:34 GMT):
@ajksharma good luck.
MohammadObaid (Fri, 03 Nov 2017 14:10:33 GMT):
Hi. After setting fabric environment on aws using ansible agent , I ran this query `peer channel list -o orderer1st-orgd:7050` to see which channel is been joined by peers . I got empty output as shown in pic below
MohammadObaid (Fri, 03 Nov 2017 14:11:54 GMT):
channellist.png
MohammadObaid (Fri, 03 Nov 2017 14:15:16 GMT):
I assume that no channel is being created by ansible . So when I ran query to create channel using this command `peer channel create -c mychannel -o orderer1st-orgd:7050 --cafile /etc/hyperledger/fabric-ca-server-config/ca/ca.orga-cert.pem ` I got `BAD REQUEST` error . Here are docker logs of orderer . Any clue how to solve this? Dosent ansible already create channels and instantiate chaincodes?
MohammadObaid (Fri, 03 Nov 2017 14:16:32 GMT):
dockerlogs.png
tongli (Fri, 03 Nov 2017 15:56:01 GMT):
@MohammadObaid is this the env on AWS?
tongli (Fri, 03 Nov 2017 15:56:22 GMT):
It seems to me that your network was not created successfully.
tongli (Fri, 03 Nov 2017 15:57:07 GMT):
the picture is from which container log?
tongli (Fri, 03 Nov 2017 15:59:43 GMT):
@KaiUsher r u Chen Kai, saw your comment on a patch set.
KaiUsher (Fri, 03 Nov 2017 15:59:44 GMT):
Has joined the channel.
tongli (Fri, 03 Nov 2017 16:00:18 GMT):
would like to discuss ideas on how to integrate ansible agent created fabric cluster to cello dashboard.
bennettneale (Fri, 03 Nov 2017 16:17:54 GMT):
Has joined the channel.
MohammadObaid (Sat, 04 Nov 2017 07:21:22 GMT):
@tongli that pciture is from orderer logs.
MohammadObaid (Sat, 04 Nov 2017 07:21:36 GMT):
These are all my containers running on aws instances
MohammadObaid (Sat, 04 Nov 2017 07:22:08 GMT):
dockercontainers.png
MohammadObaid (Sat, 04 Nov 2017 07:38:46 GMT):
Here is docker logs of orderer of fabric002
MohammadObaid (Sat, 04 Nov 2017 07:39:30 GMT):
dockerlogsorderer.png
MohammadObaid (Sat, 04 Nov 2017 07:40:17 GMT):
Problem is I unable to locate any error niether during ansible command nor in docker logs
MohammadObaid (Sat, 04 Nov 2017 07:55:58 GMT):
It may be due to low memory. While configuring ansible on aws file I decided to give `t2.micro` instead of `t2.medium` beacuse of free tier I am using. On kafka logs I got an error `OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000c0000000, 1073741824, 0) failed; error='Cannot allocate memory' (errno=12)`
grapebaba (Sat, 04 Nov 2017 08:56:57 GMT):
@tongli it's me, create a JIRA first, let us discuss there
OlliKasari (Mon, 06 Nov 2017 07:25:21 GMT):
Has joined the channel.
OlliKasari (Mon, 06 Nov 2017 07:52:43 GMT):
Is there any tutorial available for utilizing Cello with the Fabric examples?
rubi_1432 (Mon, 06 Nov 2017 10:15:19 GMT):
Hi, i am having problems. I have setup ssh and added key to authorizedd_keys on other machines and i am still getting errors below ```fabricnodem1@fabric001:~/cello/src/agent/ansible$ ansible-playbook -i run/runhosts -e "mode=apply env=os" initcluster.yml --skip-tags="resetconn"
PLAY [Get start timestamp] ********************************************************************************************************************
TASK [Gathering Facts] ************************************************************************************************************************
ok: [cloud]
TASK [set_fact] *******************************************************************************************************************************
ok: [cloud]
PLAY [fast initialize all the servers] ********************************************************************************************************
TASK [Gathering Facts] ************************************************************************************************************************
fatal: [192.168.147.147]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: ssh: connect to host 192.168.147.147 port 22: Connection refused\r\n", "unreachable": true}
fatal: [192.168.147.148]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Permission denied (publickey,password).\r\n", "unreachable": true}
fatal: [192.168.147.149]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: ssh: connect to host 192.168.147.149 port 22: No route to host\r\n", "unreachable": true}
NO MORE HOSTS LEFT ****************************************************************************************************************************
to retry, use: --limit @/home/fabricnodem1/cello/src/agent/ansible/initcluster.retry
PLAY RECAP ************************************************************************************************************************************
192.168.147.147 : ok=0 changed=0 unreachable=1 failed=0
192.168.147.148 : ok=0 changed=0 unreachable=1 failed=0
192.168.147.149 : ok=0 changed=0 unreachable=1 failed=0
cloud : ok=2 changed=0 unreachable=0 failed=0
fabricnodem1@fabric001:~/cello/src/agent/ansible$ ssh fabric002
Welcome to Ubuntu 16.04.1 LTS (GNU/Linux 4.4.0-31-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
539 packages can be updated.
283 updates are security updates.
Last login: Mon Nov 6 02:11:53 2017 from 192.168.147.147
fabricnodem1@fabric002:~$
```
rubi_1432 (Mon, 06 Nov 2017 10:15:34 GMT):
i am able to use ssh manually but having problems when running command of *ansible-playbook -i run/runhosts -e "mode=apply env=os" initcluster.yml --skip-tags="resetconn"*
baohua (Mon, 06 Nov 2017 12:50:39 GMT):
@OlliKasari here are tutorial to start cello: http://cello.readthedocs.io/en/latest/tutorial/. for using fabric, pls see fabric documentation.
tongli (Mon, 06 Nov 2017 17:08:01 GMT):
@rubi_1432 how did you set up these 3 machines? they need to have the ssh public key on each machine in ~/.ssh/authorized_keys file.
tongli (Mon, 06 Nov 2017 17:08:15 GMT):
without the file, you can not access them.
tongli (Mon, 06 Nov 2017 17:08:35 GMT):
I mean that ansible can not access them using key paires
MohammadObaid (Mon, 06 Nov 2017 17:23:37 GMT):
@tongli without kafka channels and chaincode wont be initialized properly . Am I right?
tongli (Mon, 06 Nov 2017 17:24:55 GMT):
kafka(solo) is to allow orderers to work correctly, without either, orderers can not work and channels can not be created.
tongli (Mon, 06 Nov 2017 17:25:07 GMT):
I mean kafka or solo.
MohammadObaid (Mon, 06 Nov 2017 17:35:16 GMT):
Clipboard - November 6, 2017 10:31 PM
MohammadObaid (Mon, 06 Nov 2017 17:36:00 GMT):
Alright. what is the purpose of last two containers?
tongli (Mon, 06 Nov 2017 17:41:33 GMT):
@MohammadObaid the last two containers actually are what magic happens.
tongli (Mon, 06 Nov 2017 17:41:50 GMT):
for containers to communicate with each other using names, we need registrators and dns services.
tongli (Mon, 06 Nov 2017 17:43:02 GMT):
each node will have a registrator container and entire fabric network will have one dns services.
tongli (Mon, 06 Nov 2017 17:43:10 GMT):
dns service container.
tongli (Mon, 06 Nov 2017 17:44:08 GMT):
this way, all containers will be able to simply use their names to address each other, other than IP addresses , in many cases IP addresses are unknown before fabric network is up running. but names are known.
MohammadObaid (Mon, 06 Nov 2017 17:59:07 GMT):
and fabric001 will be consider as dns service container as mentioned in aws.yml file `builders: ["fabric001"]` right?
tongli (Mon, 06 Nov 2017 18:01:24 GMT):
skydns.fabric001 is the dns container
tongli (Mon, 06 Nov 2017 18:01:32 GMT):
it runs on fabric001 node.
tongli (Mon, 06 Nov 2017 18:01:47 GMT):
this should be the only container for dns service for your entire fabric network.
MohammadObaid (Mon, 06 Nov 2017 18:05:53 GMT):
Alright . Thanks for detailed info :)
tongli (Mon, 06 Nov 2017 18:25:33 GMT):
@MohammadObaid from your image posted above, it does not look like your network is running, the kafka container is not doing well.
MohammadObaid (Tue, 07 Nov 2017 09:36:49 GMT):
@tongli Yeah you are right. Basically in my aws.yml file I change this `flavor_name: "t2.medium"` to `flavor_name: "t2.micro"` which causes kafka container to break because kafka required 1 GB RAM and t2.micro dosent provide enough to run kafka properly
rubi_1432 (Tue, 07 Nov 2017 10:38:49 GMT):
@tongli I have three physical machines. With hostname fabirc001, fabric002 and fabirc003. I installed openssh-server on fabric002 and fabric003. Then i used made sure that every user was able to use *sudo su* without password. Then on fabric001 i executed these commands *mkdir -p ~/.ssh && cd ~/.ssh && ssh-keygen -t rsa -f fd -P ""
eval $(ssh-agent -s) && ssh-add ~/.ssh/fd* After thiso made sure that they are able to ping eachother and then i executed following command from fabric001 *cat ~/.ssh/fd.pub | ssh fabric002 "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"* *cat ~/.ssh/fd.pub | ssh fabric003 "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"*
rubi_1432 (Tue, 07 Nov 2017 10:40:58 GMT):
@tongli I have three physical machines. With hostname fabirc001, fabric002 and fabirc003. I installed openssh-server on fabric002 and fabric003. Then i used made sure that every user was able to use *sudo su* without password. Then on fabric001 i executed these commands
mkdir -p ~/.ssh && cd ~/.ssh && ssh-keygen -t rsa -f fd -P ""
eval $(ssh-agent -s) && ssh-add ~/.ssh/fd
After this i made sure that they are able to ping each other and then i executed following command from fabric001
cat ~/.ssh/fd.pub | ssh fabric002 "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"
cat ~/.ssh/fd.pub | ssh fabric003 "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"
rubi_1432 (Tue, 07 Nov 2017 10:40:58 GMT):
@tongli I have three physical machines. With hostname fabirc001, fabric002 and fabirc003. I installed openssh-server on fabric002 and fabric003. Then i used made sure that every user was able to use *sudo su* without password. Then on fabric001 i executed these commands
mkdir -p ~/.ssh && cd ~/.ssh && ssh-keygen -t rsa -f fd -P ""
eval $(ssh-agent -s) && ssh-add ~/.ssh/fd
After this i made sure that they are able to ping each other and then i executed following command from fabric001
````cat ~/.ssh/fd.pub | ssh fabric002 "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"
cat ~/.ssh/fd.pub | ssh fabric003 "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"````
rubi_1432 (Tue, 07 Nov 2017 10:40:58 GMT):
@tongli I have three physical machines. With hostname fabirc001, fabric002 and fabirc003. I installed openssh-server on fabric002 and fabric003. Then i used made sure that every user was able to use *sudo su* without password. Then on fabric001 i executed these commands
mkdir -p ~/.ssh && cd ~/.ssh && ssh-keygen -t rsa -f fd -P ""
eval $(ssh-agent -s) && ssh-add ~/.ssh/fd
After this i made sure that they are able to ping each other and then i executed following command from fabric001
```cat ~/.ssh/fd.pub | ssh fabric002 "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"
cat ~/.ssh/fd.pub | ssh fabric003 "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"```
rubi_1432 (Tue, 07 Nov 2017 10:40:58 GMT):
@tongli I have three physical machines. With hostname fabirc001, fabric002 and fabirc003. I installed openssh-server on fabric002 and fabric003. Then i used made sure that every user was able to use *sudo su* without password. Then on fabric001 i executed these commands
```mkdir -p ~/.ssh && cd ~/.ssh && ssh-keygen -t rsa -f fd -P ""
eval $(ssh-agent -s) && ssh-add ~/.ssh/fd ```
After this i made sure that they are able to ping each other and then i executed following command from fabric001
```cat ~/.ssh/fd.pub | ssh fabric002 "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"
cat ~/.ssh/fd.pub | ssh fabric003 "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"```
tongli (Tue, 07 Nov 2017 14:34:43 GMT):
@rubi_1432 it seems to me that you are using fabric001 also as your ansible controller, is that right?
tongli (Tue, 07 Nov 2017 14:35:20 GMT):
the step you referenced above only is needed where you run the ansible project.
tongli (Tue, 07 Nov 2017 14:36:14 GMT):
the fd.pub file will need to be on all the servers .ssh directory in file authorized_keys file.
tongli (Tue, 07 Nov 2017 14:37:01 GMT):
the private key only needs to be on the ansible controller (in your case, fabric001).
tongli (Tue, 07 Nov 2017 14:38:06 GMT):
looks like that your command above did that.
tongli (Tue, 07 Nov 2017 14:38:46 GMT):
if all the steps are correct, after you did this ``eval $(ssh-agent -s) && ssh-add ~/.ssh/fd```
tongli (Tue, 07 Nov 2017 14:39:15 GMT):
you should be able to simply do the following to hit fabric002 and fabric003.
tongli (Tue, 07 Nov 2017 14:39:35 GMT):
```ssh fabric002_ip``` from fabric001 machine.
tongli (Tue, 07 Nov 2017 14:39:48 GMT):
if you can not do that, then your setup has not been correct.
tongli (Tue, 07 Nov 2017 14:40:27 GMT):
good news is that if you are using cello ansible agent against a cloud like what @MohammadObaid has done, all these things will be done for you by the project.
tongli (Tue, 07 Nov 2017 14:40:42 GMT):
a cloud like AWS or OpenStack cloud
tongli (Tue, 07 Nov 2017 14:41:53 GMT):
all you have to do will be just change few things in vars/aws.yml file, and run one command to get things going. see things at this link. https://github.com/hyperledger/cello/tree/master/src/agent/ansible#on-aws-cloud
tongli (Tue, 07 Nov 2017 14:42:49 GMT):
when you have physical servers or Virtual Machines, it will be a bit tedious because ansible agent has to know where your servers are and how to access them.
tongli (Tue, 07 Nov 2017 14:43:54 GMT):
what you are trying to do in these manual steps are basically setting up ssh access so that your ssh client (which ansible depends on) can work with these machines.
tongli (Tue, 07 Nov 2017 14:44:32 GMT):
the reason that ansible was chosen is because when you have very large numbers of servers, ansible will still be able to handle them regardless you have one or 100.
tongli (Tue, 07 Nov 2017 14:44:47 GMT):
and the way to handle one or 100 is absolutely identical.
JoshuaRippon (Tue, 07 Nov 2017 15:02:58 GMT):
Hi @tongli I am going to try your method again with a fresh ubuntu server!!
JoshuaRippon (Tue, 07 Nov 2017 15:04:21 GMT):
I am confused at the point of the key pair. I think this time I should just keep the key pair that I already have and include it in the vars/vb.yml and vars/bc2nd.yml files
tongli (Tue, 07 Nov 2017 15:04:25 GMT):
@JoshuaRippon sure, sure, if I remember you are using one node , right?
JoshuaRippon (Tue, 07 Nov 2017 15:04:30 GMT):
yes
tongli (Tue, 07 Nov 2017 15:04:53 GMT):
if you like, I was trying to get the all-in-one doc improved so that it will be easier to follow.
tongli (Tue, 07 Nov 2017 15:05:07 GMT):
unfortnately it has not been merged but you can still follow it if you like.
tongli (Tue, 07 Nov 2017 15:05:10 GMT):
here it is https://gerrit.hyperledger.org/r/#/c/15133/3/src/agent/ansible/vars/All-in-one.md,unified
tongli (Tue, 07 Nov 2017 15:06:01 GMT):
crap, that link has older and newer version all merged together.
tongli (Tue, 07 Nov 2017 15:06:04 GMT):
not good.
tongli (Tue, 07 Nov 2017 15:06:18 GMT):
hold on, let me see if there is a way just to see the latest version.
JoshuaRippon (Tue, 07 Nov 2017 15:06:21 GMT):
haha
tongli (Tue, 07 Nov 2017 15:07:15 GMT):
this link is better https://gerrit.hyperledger.org/r/cat/15133%2C3%2Csrc/agent/ansible/vars/All-in-one.md%5E0
tongli (Tue, 07 Nov 2017 15:07:38 GMT):
I think that link let you see the only the new version.
tongli (Tue, 07 Nov 2017 15:08:00 GMT):
you can follow that. please make sure that your ubuntu system is clean.
tongli (Tue, 07 Nov 2017 15:08:32 GMT):
if it is polluted, you will have some strange issues which will be hard to figure out.
JoshuaRippon (Tue, 07 Nov 2017 15:08:47 GMT):
yes its completely clean
tongli (Tue, 07 Nov 2017 15:08:59 GMT):
I strongly suggest that you install VirtualBox on your laptop or desktop, then using it for the work.
tongli (Tue, 07 Nov 2017 15:09:14 GMT):
VirtualBox is free and will save you a lot of headaches down the road.
JoshuaRippon (Tue, 07 Nov 2017 15:10:06 GMT):
Ok
JoshuaRippon (Tue, 07 Nov 2017 15:10:56 GMT):
so this bit i am still really confused about. The key pair part. I already have a key pair. If I want to use that one instead, how do I include it in vars/vb.yml and vars/bc2nd.yml files
JoshuaRippon (Tue, 07 Nov 2017 15:10:59 GMT):
?
JoshuaRippon (Tue, 07 Nov 2017 15:12:35 GMT):
Where abouts do I put the key pair in here? ```---
# The url to the fabric source repository
GIT_URL: "http://gerrit.hyperledger.org/r/fabric"
# The gerrit patch set reference, should be automatically set by gerrit
GERRIT_REFSPEC: ""
# The following defines a fabric network with 2 peers, 1 orderer, 1 zookeeper and 1 kafka
# It also defines that the peers will be using goleveldb.
fabric: {
ssh_user: "ubuntu",
peer_db: "goleveldb",
tls: true,
network: {
fabric001: {
peers: ["leader@peer1st.mad","anchor@peer2nd.mad"],
orderers: ["1storderer.sin"],
zookeepers: ["zookeeper1st"],
kafkas: ["kafka1st"]
}
},
baseimage_tag: "1.0.2",
ca: { tag: "1.0.2", admin: "admin", adminpw: "adminpw" }
}
```
tongli (Tue, 07 Nov 2017 16:11:04 GMT):
@JoshuaRippon you include the key pair in vars/vb.yml file.
tongli (Tue, 07 Nov 2017 16:11:32 GMT):
the vars/vb2nd.yml is the file to layout your fabric network topology.
JoshuaRippon (Tue, 07 Nov 2017 16:13:22 GMT):
Here is my vb.yml
JoshuaRippon (Tue, 07 Nov 2017 16:13:28 GMT):
```
---
auth: {
# Define where the VirtualBox(VB) system is
auth_url: "192.168.0.36",
# User name to log in to the remote VB system
username: "tongli",
# Password to the VB system, can specify at command line
# or use env variable named PASSWORD
password: "{{ password | default(lookup('env', 'password')) }}"
}
# This variable defines cloud provision attributes
cluster: {
target_os: "ubuntu",
image_name: "fabricbase",
ssh_user: "ubuntu",
ssh_pass: "ps",
validate_certs: True,
public_key_file: "/home/ubuntu/.ssh/fd.pub",
private_key_file: "/home/ubuntu/.ssh/fd",
# This variable indicate what IP should be used, only valid values are
# private_ip or public_ip
node_ip: "public_ip",
container_network: {
Network: "172.16.0.0/16",
SubnetLen: 24,
SubnetMin: "172.16.0.0",
SubnetMax: "172.16.255.0",
Backend: {
Type: "udp",
Port: 8285
}
},
service_ip_range: "172.15.0.0/16",
dns_service_ip: "172.15.0.4",
# the section defines preallocated IP addresses for each node, if there is no
# preallocated IPs, leave it blank
node_ips: ["169.45.102.186", "169.45.102.187", "169.45.102.188"],
# fabric network node names expect to be using a clear pattern, this defines
# the prefix for the node names.
name_prefix: "fabric",
domain: "fabricnet",
# stack_size determines how many virtual or physical machines we will have
# each machine will be named ${name_prefix}001 to ${name_prefix}${stack_size}
stack_size: 2,
etcdnodes: ["fabric001"],
builders: ["fabric001"],
flannel_repo: "https://github.com/coreos/flannel/releases/download/v0.7.1/flannel-v0.7.1-linux-amd64.tar.gz",
etcd_repo: "https://github.com/coreos/etcd/releases/download/v3.2.0/etcd-v3.2.0-linux-amd64.tar.gz",
k8s_repo: "https://storage.googleapis.com/kubernetes-release/release/v1.7.0/bin/linux/amd64/",
go_ver: "1.8.3",
# If volume want to be used, specify a size in GB, make volume size 0 if wish
# not to use volume from your cloud
volume_size: 0,
# cloud block device name presented on virtual machines.
block_device_name: "/dev/vdb"
}
```
JoshuaRippon (Tue, 07 Nov 2017 16:13:53 GMT):
what do I change here?
tongli (Tue, 07 Nov 2017 16:16:45 GMT):
you see the public key and private key?
tongli (Tue, 07 Nov 2017 16:17:09 GMT):
public_key_file, private_key_file,
tongli (Tue, 07 Nov 2017 16:17:28 GMT):
you change the value of these two things.
tongli (Tue, 07 Nov 2017 16:17:55 GMT):
but the important thing is that you have to put the public key in .ssh/authorized_keys file.
tongli (Tue, 07 Nov 2017 16:18:13 GMT):
otherwise, you can not ssh to this machine.
JoshuaRippon (Tue, 07 Nov 2017 16:19:52 GMT):
ok cool so can I just leave that file and just create fd.pub and fd with my keys?
JoshuaRippon (Tue, 07 Nov 2017 16:21:44 GMT):
Also Hyperledger Fabric requires Docker version 17.03.0-ce or greater
JoshuaRippon (Tue, 07 Nov 2017 16:23:01 GMT):
Will I run into problems as `sudo apt-get install docker.io` gives me `Docker version 1.12.6, build 78d1802`
tongli (Tue, 07 Nov 2017 16:37:20 GMT):
@JoshuaRippon I am not following, you can either use your own key pair or create brand new key pair.
tongli (Tue, 07 Nov 2017 16:37:33 GMT):
I do not know what you mean by create fd.pub and fd with my keys.
JoshuaRippon (Tue, 07 Nov 2017 16:38:51 GMT):
when i run ```mkdir -p ~/.ssh && cd ~/.ssh && ssh-keygen -t rsa -f fd -P ""
eval $(ssh-agent -s) && ssh-add ~/.ssh/fd``` aren't I just creatin new keys
JoshuaRippon (Tue, 07 Nov 2017 16:39:22 GMT):
with public key in fd.pub and priv key in fd?
tongli (Tue, 07 Nov 2017 16:40:41 GMT):
right. that creates new key named fd.pub and fd
tongli (Tue, 07 Nov 2017 16:40:49 GMT):
one is private key and one is public key.
tongli (Tue, 07 Nov 2017 16:41:20 GMT):
to be able to allow ssh to a machine, you have to add the fd.pub content into this file ~/.ssh/authorized_keys
tongli (Tue, 07 Nov 2017 16:41:44 GMT):
it has nothing to do with whatever keys that you might already have.
tongli (Tue, 07 Nov 2017 16:42:03 GMT):
if your system is new, then you should not have any keys I would assume.
JoshuaRippon (Tue, 07 Nov 2017 16:43:12 GMT):
yes I have keys already set up
tongli (Tue, 07 Nov 2017 16:43:21 GMT):
k.
tongli (Tue, 07 Nov 2017 16:43:35 GMT):
before you do anything, you can verify if you can actually ssh to this server.
tongli (Tue, 07 Nov 2017 16:43:49 GMT):
simply do ssh ubuntu@theIPaddress.
tongli (Tue, 07 Nov 2017 16:43:59 GMT):
or the userid that you have.
tongli (Tue, 07 Nov 2017 16:44:10 GMT):
I use ubuntu as an example.
tongli (Tue, 07 Nov 2017 16:44:31 GMT):
notice that vars/vb.yml file also has ssh_user, make sure that is also changed,
tongli (Tue, 07 Nov 2017 16:44:38 GMT):
to match whtever you have.
tongli (Tue, 07 Nov 2017 16:44:51 GMT):
without changing that to what you have, ssh will also fail.
JoshuaRippon (Tue, 07 Nov 2017 16:45:50 GMT):
Haha you know what I'll leave it till tomorrow when my colleague is in. He sets up all the servers so will know about this stuff way more than me! My background is physics! lol
tongli (Tue, 07 Nov 2017 16:47:46 GMT):
@JoshuaRippon good try though.
JoshuaRippon (Tue, 07 Nov 2017 16:48:01 GMT):
cheers!
JoshuaRippon (Tue, 07 Nov 2017 16:50:04 GMT):
Just quickly though, will I run into problems connecting to Fabric as fabric requires Docker version 17.03.0-ce or greater?
tongli (Tue, 07 Nov 2017 16:51:18 GMT):
the process will install the right version of golang and docker on this clean system.
tongli (Tue, 07 Nov 2017 16:51:25 GMT):
it will be all taken care of for you.
tongli (Tue, 07 Nov 2017 16:51:29 GMT):
Go Cello.
tongli (Tue, 07 Nov 2017 16:51:51 GMT):
that is the reason why keep saying clean clean clean system.
shiyj93 (Wed, 08 Nov 2017 01:38:31 GMT):
Has joined the channel.
rubi_1432 (Wed, 08 Nov 2017 06:51:06 GMT):
@tongli yes i did that and i am able to use *ssh fabric002* and also *ssh 192.168.147.147* manually from terminal. Okay thanks i will look into it.
rubi_1432 (Wed, 08 Nov 2017 09:38:06 GMT):
@tongli Quick question. Can i try cello on vmareworkstation virtual machines or will it only work with virtual box?
cl4rke (Wed, 08 Nov 2017 11:57:37 GMT):
Has joined the channel.
tongli (Wed, 08 Nov 2017 13:35:38 GMT):
@rubi_1432 cello ansible agent does not care what kind of the server you have. as long as it can ssh to these machines.
ShravaniGandla (Thu, 09 Nov 2017 10:00:31 GMT):
Has joined the channel.
ShravaniGandla (Thu, 09 Nov 2017 10:24:04 GMT):
Hello i am very beginner to cello, if i try to add host always it failed.
ShravaniGandla (Thu, 09 Nov 2017 10:24:26 GMT):
please anyone help me to add a host in cello dashboard
Manasabingi (Thu, 09 Nov 2017 10:31:07 GMT):
Has joined the channel.
baohua (Thu, 09 Nov 2017 10:31:34 GMT):
hi @shaneG would u like to check the log to see if there's some error msg? you can run `make log` to see log messages.
shaneG (Thu, 09 Nov 2017 10:31:34 GMT):
Has joined the channel.
ShravaniGandla (Thu, 09 Nov 2017 10:36:16 GMT):
thanks for your quick response @baohua iam not getting any error while running make log
baohua (Thu, 09 Nov 2017 10:36:35 GMT):
pls check the log when u add a host
ShravaniGandla (Thu, 09 Nov 2017 10:38:04 GMT):
i am adding host from cello dashboard how can i check log from dashboard
ShravaniGandla (Thu, 09 Nov 2017 10:42:06 GMT):
Screenshot from 2017-11-09 16-11-16.png
ShravaniGandla (Thu, 09 Nov 2017 10:43:30 GMT):
what is daemon url ?
flash139 (Thu, 09 Nov 2017 10:56:58 GMT):
Has joined the channel.
Manasabingi (Thu, 09 Nov 2017 10:58:00 GMT):
Hello I am a beginner to hyperledger cello , I got the same problem. Please help me.
wininani (Thu, 09 Nov 2017 10:58:37 GMT):
Has joined the channel.
Manasabingi (Thu, 09 Nov 2017 10:58:45 GMT):
What is Daemon url?
flash139 (Thu, 09 Nov 2017 11:00:11 GMT):
Can anyone explain what is the Deamon url from above screenshot . Because when i tried to add local ip address to host in dashboard Getting failed error . please help me out how to use this. Thank you in advance..
baohua (Thu, 09 Nov 2017 11:12:27 GMT):
oh, can u follow the doc here?http://cello.readthedocs.io/en/latest/installation/
baohua (Thu, 09 Nov 2017 11:12:42 GMT):
u need to setup some worker node (e.g., a docker host), and then add it through dashboard
a.hyper (Thu, 09 Nov 2017 13:38:46 GMT):
Has joined the channel.
ajksharma (Thu, 09 Nov 2017 14:01:27 GMT):
Clipboard - November 9, 2017 7:31 PM
ajksharma (Thu, 09 Nov 2017 14:02:50 GMT):
@ShravaniGandla Hope you have access to the back end command-line where you started Cello?
You would need to type *make log* when you are in the cello directory.
ajksharma (Thu, 09 Nov 2017 14:02:50 GMT):
@ShravaniGandla Hope you have access to the back end command-line where you started Cello?
You would need to type *make log* when you are in the cello directory. Please see screenshot.
ajksharma (Thu, 09 Nov 2017 14:02:50 GMT):
@ShravaniGandla Hope you have access to the back end command-line where you started Cello?
You would need to type *make log* when you are in the cello directory to see the log and troubleshoot issues. Please see screenshot.
MohammadObaid (Thu, 09 Nov 2017 14:08:14 GMT):
Has anyonw here use ansible with openstack? I need some help related to this
tongli (Thu, 09 Nov 2017 14:28:54 GMT):
@baohua thanks for getting it +2ed. @hightall can you please get it +2ed as well and merge? Thanks
Manasabingi (Thu, 09 Nov 2017 14:40:16 GMT):
how can we find docker daemon url?
tongli (Thu, 09 Nov 2017 14:55:50 GMT):
@hightall at this link https://gerrit.hyperledger.org/r/#/c/15133/ thanks.
baohua (Thu, 09 Nov 2017 15:40:10 GMT):
@Manasabingi oh, you need to config it by urself. feel free to find something from https://docs.docker.com/engine/admin/
makdrive (Fri, 10 Nov 2017 00:54:50 GMT):
Has joined the channel.
Manasabingi (Fri, 10 Nov 2017 03:56:58 GMT):
okay Thank you
harsha (Fri, 10 Nov 2017 04:22:46 GMT):
Hi, Does `cello-user-dashboard` container fails to start for anyone else, it fails with `nodemon npm` not installed
```
docker logs 5fa81920ab7b
> bc-dashboard@1.0.0 dev /usr/app/src
> nodemon src/index.js --exec "babel-node src"
sh: 1: nodemon: not found
```
harsha (Fri, 10 Nov 2017 04:22:46 GMT):
Hi, does `cello-user-dashboard` container fails to start for anyone else, it fails with `nodemon npm` not installed
```
docker logs 5fa81920ab7b
> bc-dashboard@1.0.0 dev /usr/app/src
> nodemon src/index.js --exec "babel-node src"
sh: 1: nodemon: not found
```
harsha (Fri, 10 Nov 2017 04:23:18 GMT):
And why do we use `cello-user-dashboard` ?
baohua (Fri, 10 Nov 2017 05:01:31 GMT):
@hightall any comments?
ShravaniGandla (Fri, 10 Nov 2017 05:51:09 GMT):
thanks@ajksharma i am not getting any errors while running make log command, i had done with all master and worker node setup, but i am failed to add a host in the dashboard.
ajksharma (Fri, 10 Nov 2017 06:36:01 GMT):
@ShravaniGandla which document are you referring to run celllo?
ajksharma (Fri, 10 Nov 2017 06:36:01 GMT):
@ShravaniGandla which document are you referring to run cello?
ShravaniGandla (Fri, 10 Nov 2017 06:39:03 GMT):
@ajksharma iam following http://cello.readthedocs.io/en/latest/installation/
SimonOberzan (Fri, 10 Nov 2017 13:24:51 GMT):
Hi. I have just updated my master node setup, and when i start it I get the following top-level error inside restserver: ```File "/app/agent/vsphere/host_operations.py", line 13, in
tongli (Fri, 10 Nov 2017 14:20:19 GMT):
@hightall can you please take a look at this patch https://gerrit.hyperledger.org/r/#/c/15133 ?
Hansuuuuuuuuuu (Sat, 11 Nov 2017 04:13:52 GMT):
Has joined the channel.
Hansuuuuuuuuuu (Sat, 11 Nov 2017 04:14:19 GMT):
hello, i'm getting a 401 on my first login using the default un and pw :(
Hansuuuuuuuuuu (Sat, 11 Nov 2017 04:24:34 GMT):
I checked the logs and the error is
```
172.19.0.3 - - [11/Nov/2017 12:20:12] "GET /login HTTP/1.0" 200 -
[2017-11-11 12:21:41,567] ERROR [modules.user.user] [user.py:99 get_by_username()] - get user exc mongo:27017: [Errno -2] Name or service not known
[2017-11-11 12:21:41,567] INFO [modules.user.auth.login] [login.py:70 post()] - error 'NoneType' object has no attribute 'password'
172.19.0.3 - - [11/Nov/2017 12:21:41] "POST /api/auth/login HTTP/1.0" 401 -
```
Hansuuuuuuuuuu (Sat, 11 Nov 2017 04:40:17 GMT):
there seems to be a problem with the mongo image, it's bootlooping
Hansuuuuuuuuuu (Sat, 11 Nov 2017 04:49:50 GMT):
alright fixed it by remapping aw yes
sdarwin (Mon, 13 Nov 2017 15:11:30 GMT):
Has joined the channel.
tongli (Mon, 13 Nov 2017 17:23:41 GMT):
@baohua hope that you've had a great meeting in Singapore. I added two patch set.
tongli (Mon, 13 Nov 2017 17:24:08 GMT):
This one is to add supported ansible version .https://gerrit.hyperledger.org/r/#/c/15435/ this
tongli (Mon, 13 Nov 2017 17:24:33 GMT):
this one is to add a playbook to allow automation to display current fabric network status.
tongli (Mon, 13 Nov 2017 17:24:52 GMT):
this is the second one. https://gerrit.hyperledger.org/r/#/c/15437/
tongli (Mon, 13 Nov 2017 17:25:32 GMT):
I will be adding another patch set to move the docs around to address the jira issue that you opened once these two get merged. Let me know if you have any concerns.
tongli (Mon, 13 Nov 2017 17:25:59 GMT):
@hightall please do review these two patch sets as well. Thanks.
tongli (Mon, 13 Nov 2017 17:26:06 GMT):
https://gerrit.hyperledger.org/r/#/c/15435
tongli (Mon, 13 Nov 2017 17:26:13 GMT):
https://gerrit.hyperledger.org/r/#/c/15437
sdarwin (Mon, 13 Nov 2017 17:47:01 GMT):
Hi. We are experimenting with Cello for the first time, and wonder about some features:
- add more peers to a chain, but on a different physical host
- add more organizations to an existing chain.
- add a couchdb backend for the peers
- deploy chaincodes to peers with Cello
Are these on the roadmap? Or, they should be handled by another process?
Thanks.
myin2000 (Mon, 13 Nov 2017 21:35:23 GMT):
Has joined the channel.
Daniela_Barbosa (Tue, 14 Nov 2017 01:27:11 GMT):
Has joined the channel.
baohua (Tue, 14 Nov 2017 01:44:57 GMT):
@tongli thanks, tong, will review soon!
baohua (Tue, 14 Nov 2017 02:19:55 GMT):
@sdarwin currently u need to use the fabric tools to do the operations: http://hyperledger-fabric.readthedocs.io/en/latest/configtxlator.html. And feel free to open some jira issue, too :)
baohua (Tue, 14 Nov 2017 02:34:59 GMT):
welcome everyone to help review:
baohua (Tue, 14 Nov 2017 02:34:59 GMT):
https://gerrit.hyperledger.org/r/#/c/15435
https://gerrit.hyperledger.org/r/#/c/15437
seiferjp (Tue, 14 Nov 2017 05:05:23 GMT):
Has joined the channel.
SriramaSharma (Tue, 14 Nov 2017 05:27:38 GMT):
Has joined the channel.
MohammadObaid (Tue, 14 Nov 2017 06:40:26 GMT):
@tongli When ansible create hyperledger network on aws , a chaincode container `dev-peer1st-orga-firstchaincode-1.0` runs on two host `fabric001` and `fabric002` but not on third host `fabric003` ALthough there is no problem in invoking transaction or doing other stuffs but is there any specific reason that chaincode container would only run in two host?
tongli (Tue, 14 Nov 2017 16:41:02 GMT):
@MohammadObaid the example chain ode was installed on all peers and there should be one Chaincode container for every peer of each peer was accessed either by a query or a transaction
tongli (Tue, 14 Nov 2017 16:41:27 GMT):
Of=if
MohammadObaid (Tue, 14 Nov 2017 16:43:54 GMT):
Yeah. I understand but one thing strange for me is that no chaincode container runs on my fabric003 instance but still I was able to invoke transactions and run queries.
tongli (Tue, 14 Nov 2017 18:11:30 GMT):
@MohammadObaid if you should have containers on fabric003 depends on how you layout your fabric network. I can take a look at your configuration file.
MohammadObaid (Tue, 14 Nov 2017 18:14:45 GMT):
@tongli which configuration file ? bc1.yaml file in `ansible/vars` directory or `aws.yml` filr in vars directory?
tongli (Tue, 14 Nov 2017 18:15:14 GMT):
what command did you run to set up your cluster?
tongli (Tue, 14 Nov 2017 18:15:33 GMT):
@MohammadObaid see above question.
MohammadObaid (Tue, 14 Nov 2017 18:15:38 GMT):
ansible-playbook -e "mode=apply" aws.yml
MohammadObaid (Tue, 14 Nov 2017 18:18:31 GMT):
Here is aws.yml file ```---
# AWS Keys will be use to provision EC2 instances on AWS Cloud
auth: {
auth_url: "",
# This should be your AWS Access Key ID
username: "AKIAJCA47Q46DOT2M7WA",
# This should be your AWS Secret Access Key
# can be passed as part of cmd line when running the playbook
password: "{{ password | default(lookup('env', 'AWS_SECRET_KEY')) }}"
}
# These variable defines AWS cloud provision attributes
cluster: {
region_name: "us-east-1", #TODO Dynamic fetch
availability_zone: "", #TODO Dynamic fetch based on region
security_group: "Fabric",
target_os: "ubuntu",
image_name: "ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-*",
image_id: "ami-d15a75c7",
flavor_name: "t2.medium", # "m2.medium" is big enough for Fabric
ssh_user: "ubuntu",
validate_certs: True,
private_net_name: "demonet",
public_key_file: "/home/mobaid/.ssh/id_rsa.pub",
private_key_file: "/home/mobaid/.ssh/id_rsa",
ssh_key_name: "sample",
# This variable indicate what IP should be used, only valid values are
# private_ip or public_ip
node_ip: "public_ip",
container_network: {
Network: "172.16.0.0/16",
SubnetLen: 24,
SubnetMin: "172.16.0.0",
SubnetMax: "172.16.255.0",
Backend: {
Type: "udp",
Port: 8285
}
},
service_ip_range: "172.15.0.0/24",
dns_service_ip: "172.15.0.4",
# the section defines preallocated IP addresses for each node, if there is no
# preallocated IPs, leave it blank
node_ips: [ ],
# fabric network node names expect to be using a clear pattern, this defines
# the prefix for the node names.
name_prefix: "fabric",
domain: "fabricnet",
# stack_size determines how many virtual or physical machines we will have
# each machine will be named ${name_prefix}001 to ${name_prefix}${stack_size}
stack_size: 5,
etcdnodes: ["fabric001", "fabric002", "fabric003","fabric004","fabric005"],
builders: ["fabric001"],
flannel_repo: "https://github.com/coreos/flannel/releases/download/v0.7.1/flannel-v0.7.1-linux-amd64.tar.gz",
etcd_repo: "https://github.com/coreos/etcd/releases/download/v3.2.0/etcd-v3.2.0-linux-amd64.tar.gz",
k8s_repo: "https://storage.googleapis.com/kubernetes-release/release/v1.7.0/bin/linux/amd64/",
go_ver: "1.8.3",
# If volume want to be used, specify a size in GB, make volume size 0 if wish
# not to use volume from your cloud
volume_size: 8,
# cloud block device name presented on virtual machines.
block_device_name: "/dev/vdb"
} ```
tongli (Tue, 14 Nov 2017 18:45:24 GMT):
@MohammadObaid that file basically says you wanted to build a 5 nodes cluster and it looks fine to me.
tongli (Tue, 14 Nov 2017 18:45:49 GMT):
then there should also be the vars/bc1st.yml file.
tongli (Tue, 14 Nov 2017 18:45:54 GMT):
did you change that file?
tongli (Tue, 14 Nov 2017 18:46:33 GMT):
@MohammadObaid if you did not change that file, then fabric004 and fabric005 should be empty and you should have few containers on fabric003.
tongli (Tue, 14 Nov 2017 18:47:22 GMT):
by default, you are using this file to layout your fabric network.https://github.com/hyperledger/cello/blob/master/src/agent/ansible/vars/bc1st.yml
tongli (Tue, 14 Nov 2017 18:47:48 GMT):
if you did not change anything in that file, you should have two peer containers
tongli (Tue, 14 Nov 2017 18:47:59 GMT):
one rom orga and one from orgb.
tongli (Tue, 14 Nov 2017 18:48:33 GMT):
when you query or do a transaction against these two peers, each of them should spawn another container.
tongli (Tue, 14 Nov 2017 18:48:38 GMT):
chaincode container.
tongli (Tue, 14 Nov 2017 18:50:02 GMT):
@hightall please review and comment on these two patch sets, waiting for them to be merged. https://gerrit.hyperledger.org/r/#/c/15435/ https://gerrit.hyperledger.org/r/#/c/15437/
indirajith (Tue, 14 Nov 2017 22:09:49 GMT):
Review please https://gerrit.hyperledger.org/r/#/c/12627/
marryton007 (Wed, 15 Nov 2017 03:03:57 GMT):
Has joined the channel.
baohua (Wed, 15 Nov 2017 05:27:10 GMT):
thanks! @indirajith
MohammadObaid (Wed, 15 Nov 2017 05:35:08 GMT):
@tongli I also changed bc1st.yml file . Here is the content fo file ```a---
# The url to the fabric source repository
GIT_URL: "http://gerrit.hyperledger.org/r/fabric"
# The gerrit patch set reference, should be automatically set by gerrit
# GERRIT_REFSPEC: "refs/changes/23/11523/3" # 1.0.0
# GERRIT_REFSPEC: "refs/changes/47/12047/3" # 1.0.1
GERRIT_REFSPEC: "refs/changes/13/13113/1"
# This variable defines fabric network attributes
fabric: {
# The user to connect to the server
ssh_user: "ubuntu",
# options are "goleveldb", "CouchDB", default is goleveldb
peer_db: "CouchDB",
tls: false,
# The following section defines how the fabric network is going to be made up
# cas indicates certificate authority containers
# peers indicates peer containers
# orderers indicates orderer containers
# kafka indicates kafka containers
# all names must be in lower case. Numeric characters cannot be used to start
# or end a name. Character dot (.) can only be used in names of peers and orderers.
network: {
fabric001: {
cas: ["ca1st.orga"],
peers: ["anchor@peer1st.orga", "anchor@peer1st.orgb"],
orderers: ["orderer1st.orgc", "orderer1st.orgd"],
zookeepers: ["zookeeper1st"],
kafkas: ["kafka1st"]
},
fabric002: {
cas: ["ca1st.orgb"],
peers: ["worker@peer2nd.orga", "worker@peer2nd.orgb"],
orderers: ["orderer2nd.orgc", "orderer2nd.orgd"],
zookeepers: ["zookeeper2nd"],
kafkas: ["kafka2nd"]
},
fabric003: {
cas: ["ca1st.orgc", "ca1st.orgd"],
peers: ["worker@peer3rd.orga", "worker@peer3rd.orgb"],
orderers: [],
zookeepers: ["zookeeper3rd"],
kafkas: ["kafka3rd"]
},
fabric004: {
cas: ["ca1st.orgd"],
peers: ["worker@peer4th.orga", "worker@peer4th.orgb"],
orderers: [],
zookeepers: ["zookeeper4th"],
kafkas: ["kafka4th"]
},
fabric005: {
cas: ["ca1st.orgd"],
peers: ["worker@peer5th.orga", "worker@peer5th.orgb"],
orderers: [],
zookeepers: ["zookeeper5th"],
kafkas: ["kafka5th"]
}
},
baseimage_tag: "1.0.2",
ca: { tag: "1.0.2", admin: "admin", adminpw: "adminpw" }
} ```
OlliKasari (Wed, 15 Nov 2017 06:59:02 GMT):
Could someone explain to me the usage of CA (certificate authority) in hyperledger fabric? Do I need one for each organisation or can I use a centralized CA? How (and why so) would you set up the CA servers?
MohammadObaid (Wed, 15 Nov 2017 09:16:23 GMT):
@tongli thanks . I changed my bc1.yml file also may be that's the reason. Here is the gist which contain two files one is bc1s.yml and other network-config.json which is needed for node sdk .
ajksharma (Wed, 15 Nov 2017 11:22:37 GMT):
Hi @MohammadObaid did you use the aws.yml file from the cello git repo? as it looks different. Could you post a link for the source?
tongli (Wed, 15 Nov 2017 11:32:35 GMT):
@ajksharma @MohammadObaid bc1st.yml file controls how the fabric network layout look like. your fabric003 will have 2 containers from orga and orgb.
tongli (Wed, 15 Nov 2017 11:34:37 GMT):
@ajksharma @MohammadObaid general rule of thumb for both zookeeper and kafka is that you should have odd number of nodes for them, normally 3 for each will be good enough, 5 is a overkill.
tongli (Wed, 15 Nov 2017 11:50:32 GMT):
@hightall can you please review these two patch sets? https://gerrit.hyperledger.org/r/#/c/15435/, https://gerrit.hyperledger.org/r/#/c/15437/
MohammadObaid (Wed, 15 Nov 2017 16:25:32 GMT):
@tongl Here is the gist https://gist.github.com/mohammadobaid1/852038384d93330b0661624510f7b541
MohammadObaid (Wed, 15 Nov 2017 16:25:32 GMT):
@tongli @ajksharma Here is the gist https://gist.github.com/mohammadobaid1/852038384d93330b0661624510f7b541
tongli (Wed, 15 Nov 2017 16:27:15 GMT):
@MohammadObaid the fabric network layout looks fine to me.
tongli (Wed, 15 Nov 2017 16:27:34 GMT):
@MohammadObaid do not know what the network-config file is for.
MohammadObaid (Wed, 15 Nov 2017 16:28:24 GMT):
network-config file is for node sdk . Basically I need to tell node sdk about my fabric network so thet node api's can communicate with my network
tongli (Wed, 15 Nov 2017 16:39:00 GMT):
k. @MohammadObaid any more issue in terms of fabric network standing up?
tongli (Wed, 15 Nov 2017 16:39:24 GMT):
config file looks fine to me.
MohammadObaid (Wed, 15 Nov 2017 16:39:39 GMT):
Nope. I tried transactions , retreive results. Everything works fine for me uptill now
tongli (Wed, 15 Nov 2017 16:39:43 GMT):
hmmm. wait.
tongli (Wed, 15 Nov 2017 16:40:02 GMT):
actually you have two ca1st.orgd container
tongli (Wed, 15 Nov 2017 16:40:30 GMT):
one on fabric004 and fabric005
tongli (Wed, 15 Nov 2017 16:40:58 GMT):
I think that you @MohammadObaid should remove one ca container.
tongli (Wed, 15 Nov 2017 16:41:20 GMT):
two containers using exact same name will get dns confused.
tongli (Wed, 15 Nov 2017 16:42:01 GMT):
you also have ca1st.orgd on fabric003, cat1st.orgd on fabric004.
MohammadObaid (Wed, 15 Nov 2017 16:42:21 GMT):
hmm. One thing I am confused in it some server have one ca containers , some use multple ca's . What logic behind constructing such file?
tongli (Wed, 15 Nov 2017 16:42:24 GMT):
yeah, you have total of 3 cas for orgd, all named same ca1st
tongli (Wed, 15 Nov 2017 16:43:00 GMT):
cas on each node (fabric001, fabric002...) indicate what ca do you like to create.
tongli (Wed, 15 Nov 2017 16:43:20 GMT):
in your config file, you have the following
tongli (Wed, 15 Nov 2017 16:43:41 GMT):
```fabric003: {
cas: ["ca1st.orgc", "ca1st.orgd"],
peers: ["worker@peer3rd.orga", "worker@peer3rd.orgb"],
orderers: [],
zookeepers: ["zookeeper3rd"],
kafkas: ["kafka3rd"]
},```
tongli (Wed, 15 Nov 2017 16:43:51 GMT):
this configuration means the following
tongli (Wed, 15 Nov 2017 16:44:19 GMT):
1. you like to have two ca containers on this node, one ca is named ca1st.orgc, one ca is named ca1st.orgd
tongli (Wed, 15 Nov 2017 16:44:52 GMT):
2. you want to have two peer containers, one is named geer3rd in orga, one is named peer3rd in orgb
tongli (Wed, 15 Nov 2017 16:45:00 GMT):
3. no orderers for this node
tongli (Wed, 15 Nov 2017 16:45:16 GMT):
4. one zookeeper node named zookeerper3rd
tongli (Wed, 15 Nov 2017 16:45:26 GMT):
5. one kafka node named kafka3rd.
tongli (Wed, 15 Nov 2017 16:46:06 GMT):
looking at all the node combined, you have 3 ca containers spread out to these 5 nodes, they are all named ca1st.orgd which will cause dns issues.
MohammadObaid (Wed, 15 Nov 2017 16:46:27 GMT):
SO basically this network is running between 4 organizations and we have constructed different peers related to these organizations in random order right?
tongli (Wed, 15 Nov 2017 16:46:28 GMT):
ultimately inconsistence when you use that ca to register new users.
MohammadObaid (Wed, 15 Nov 2017 16:46:41 GMT):
Alright
tongli (Wed, 15 Nov 2017 16:46:45 GMT):
right, you will have orga, orgb, orgc and orgd.
tongli (Wed, 15 Nov 2017 16:47:47 GMT):
you should just have 1 ca per org, if you want to have multiple ca per org, then you have to figure out how two cas communicate with each other, which is not the concern for cello.
tongli (Wed, 15 Nov 2017 16:49:01 GMT):
to fix the issue , you simply need to remove ca1st.orgd from fabric003 ca array, and fabric005 ca array.
tongli (Wed, 15 Nov 2017 16:49:27 GMT):
just leave one ca for orgd on fabric004 maybe or on fabric005 depends on where you want it to be.
tongli (Wed, 15 Nov 2017 16:50:37 GMT):
also I wrote earlier, having 5 containers for both kafka and zookeeper is an over kill, it will work fine, but you do not really have to do that. anyways.
MohammadObaid (Wed, 15 Nov 2017 16:51:07 GMT):
Yeah got it.
MohammadObaid (Wed, 15 Nov 2017 16:51:13 GMT):
orgc and orgd is used as ordering service while orga and orgb is used as two organziation for transfer asset right?
tongli (Wed, 15 Nov 2017 16:51:28 GMT):
correct.
tongli (Wed, 15 Nov 2017 16:53:00 GMT):
you can actually specify an org for orderers. so it is like a third party (company) is responsible for orderers, kakfa and zookeepers, other orgs are just participating in the network for transactions.
tongli (Wed, 15 Nov 2017 16:53:37 GMT):
but it is really your call. multiple orgs for orderers are fine. but zookeepers and kafka are not really having anything to do with an org.
MohammadObaid (Wed, 15 Nov 2017 16:53:40 GMT):
Alright. Thanks for helping me out :) Initially I modify that file considering fabric001 ,fabric002 as organizations .That's why I got everything wrong
tongli (Wed, 15 Nov 2017 16:53:53 GMT):
haha.
tongli (Wed, 15 Nov 2017 16:54:13 GMT):
fabric001, fabric002,etc are physical or virtual machines
MohammadObaid (Wed, 15 Nov 2017 16:54:48 GMT):
Yeah. Now I understand .
tongli (Wed, 15 Nov 2017 16:56:01 GMT):
great. let me know if you need any more help.
MohammadObaid (Wed, 15 Nov 2017 16:57:08 GMT):
Alright.
HubertYoung (Thu, 16 Nov 2017 07:58:15 GMT):
Has joined the channel.
HubertYoung (Thu, 16 Nov 2017 07:58:29 GMT):
Hi,guys.I cannot login with admin:pass.
HubertYoung (Thu, 16 Nov 2017 07:58:50 GMT):
dashboard | [2017-11-16 15:56:16,774] ERROR [modules.user.user] [user.py:99 get_by_username()] - get user exc User matching query does not exist.
dashboard | [2017-11-16 15:56:16,775] INFO [modules.user.auth.login] [login.py:70 post()] - error 'NoneType' object has no attribute 'password'
dashboard | 172.19.0.6 - - [16/Nov/2017 15:56:16] "POST /api/auth/login HTTP/1.0" 401 -
HubertYoung (Thu, 16 Nov 2017 08:06:05 GMT):
Got it.I didn't start dashboard service.
sravs (Thu, 16 Nov 2017 08:07:13 GMT):
Has joined the channel.
baohua (Thu, 16 Nov 2017 08:16:55 GMT):
:)
HubertYoung (Thu, 16 Nov 2017 08:36:59 GMT):
@baohua I'm new here.How can i start to setup fabric cluster on my computer?Mine is centos.
baohua (Thu, 16 Nov 2017 11:56:35 GMT):
welcome @HubertYoung feel free to look at cello's doc: http://cello.readthedocs.io/en/latest/
MohammadObaid (Thu, 16 Nov 2017 12:11:48 GMT):
Hey @tongli till yesterday everything works fine for me but today when I relaunch fabric network I am seeing my kafka containers not properly running and seems to be `restarting again and again` Initially I thought that it may be due to changes I done in ansible files however after fresh download of cello and ansible and using default configuration files I am still facing the same issue. Seems like root issue is kafka is unable to communicate with zookeeper as you can see in kafka logs below
MohammadObaid (Thu, 16 Nov 2017 12:12:51 GMT):
I am seeing this log on all three servers ```[2017-11-16 12:08:12,606] INFO Initiating client connection, connectString=zookeeper1st:2181,zookeeper2nd:2181,zookeeper3rd:2181 sessionTimeout=6000 watcher=org.I0Itec.zkclient.ZkClient@3c419631 (org.apache.zookeeper.ZooKeeper)
[2017-11-16 12:08:12,621] INFO Waiting for keeper state SyncConnected (org.I0Itec.zkclient.ZkClient)
[2017-11-16 12:08:12,627] INFO Opening socket connection to server ip-172-16-12-4.ec2.internal/172.16.12.4:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2017-11-16 12:08:12,632] INFO Socket connection established to ip-172-16-12-4.ec2.internal/172.16.12.4:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2017-11-16 12:08:12,634] INFO Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
[2017-11-16 12:08:12,932] INFO Opening socket connection to server ip-172-16-80-5.ec2.internal/172.16.80.5:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2017-11-16 12:08:14,934] INFO Client session timed out, have not heard from server in 2198ms for sessionid 0x0, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
[2017-11-16 12:08:15,885] INFO Opening socket connection to server ip-172-16-18-4.ec2.internal/172.16.18.4:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2017-11-16 12:08:17,887] INFO Client session timed out, have not heard from server in 2852ms for sessionid 0x0, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
[2017-11-16 12:08:18,622] INFO Terminate ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
[2017-11-16 12:08:19,215] INFO Opening socket connection to server ip-172-16-12-4.ec2.internal/172.16.12.4:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2017-11-16 12:08:19,216] INFO Socket connection established to ip-172-16-12-4.ec2.internal/172.16.12.4:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2017-11-16 12:08:19,318] INFO Session: 0x0 closed (org.apache.zookeeper.ZooKeeper)
[2017-11-16 12:08:19,318] INFO EventThread shut down (org.apache.zookeeper.ClientCnxn)
[2017-11-16 12:08:19,319] FATAL Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 6000
at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1223)
at org.I0Itec.zkclient.ZkClient.
MohammadObaid (Thu, 16 Nov 2017 12:14:56 GMT):
And here is the zookeeper logs ```2017-11-16 12:12:32,436 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2181:QuorumPeer$QuorumServer@149] - Resolved hostname: zookeeper1st to address: zookeeper1st/172.16.18.4
2017-11-16 12:12:37,441 [myid:2] - WARN [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@400] - Cannot open channel to 3 at election address zookeeper3rd/172.16.80.5:3888
java.net.SocketTimeoutException: connect timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:381)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:426)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:843)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:822)
2017-11-16 12:12:37,443 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2181:QuorumPeer$QuorumServer@149] - Resolved hostname: zookeeper3rd to address: zookeeper3rd/172.16.80.5
2017-11-16 12:12:37,443 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2181:FastLeaderElection@852] - Notification time out: 60000
```
MohammadObaid (Thu, 16 Nov 2017 12:31:50 GMT):
May be I need to increase timeout period of zookeeper but I am unsure how to do it or any alternative solution to solve this?
tongli (Thu, 16 Nov 2017 14:23:21 GMT):
@MohammadObaid mostly the network issues. nodes can not communicate with each other.
MohammadObaid (Thu, 16 Nov 2017 14:23:42 GMT):
Means an aws issue right?
tongli (Thu, 16 Nov 2017 14:24:09 GMT):
the best way to confirm that is to start a busybox container and ping other containers by ip and name
tongli (Thu, 16 Nov 2017 14:24:17 GMT):
not necessarily aws network issue.
tongli (Thu, 16 Nov 2017 14:25:02 GMT):
@MohammadObaid cello sets up an overlay network (using flanneld), which allows us sets up a small network on top of aws network.
tongli (Thu, 16 Nov 2017 14:25:19 GMT):
this way your network is not accessible from outside of your network.
tongli (Thu, 16 Nov 2017 14:25:39 GMT):
and you do not have problems with conflicting ports since each container gets their own private ip.
tongli (Thu, 16 Nov 2017 14:25:48 GMT):
in 172.16.x.x segement.
MohammadObaid (Thu, 16 Nov 2017 14:25:50 GMT):
Ideally from one server I should be able to ping every other container beacuse three server are connected to each other
tongli (Thu, 16 Nov 2017 14:26:01 GMT):
also we set up dns services.
tongli (Thu, 16 Nov 2017 14:26:19 GMT):
that is why peer1st.orga can see peer2nd.orgb container.
tongli (Thu, 16 Nov 2017 14:26:22 GMT):
by using names.
tongli (Thu, 16 Nov 2017 14:26:38 GMT):
@MohammadObaid that is exactly right.
tongli (Thu, 16 Nov 2017 14:27:03 GMT):
using IP addresses in 172.16.x.x segement which is part of the cello ansible agent configuration file.
tongli (Thu, 16 Nov 2017 14:27:17 GMT):
not the IP address that AWS signed to your VMs.
tongli (Thu, 16 Nov 2017 14:27:35 GMT):
if you log in to one of your machines, you should see your VM ip address.
MohammadObaid (Thu, 16 Nov 2017 14:28:01 GMT):
I will try to ping every other conatainer . Yeah ip address under docker0 tag in ifconfig section right?
tongli (Thu, 16 Nov 2017 14:28:10 GMT):
if you docker inspect fabric containers, you will see them having IP address like 172.16.x.x
tongli (Thu, 16 Nov 2017 14:28:38 GMT):
ifconfig to see your VM ip. or ip addr
tongli (Thu, 16 Nov 2017 14:29:18 GMT):
to see container IPs, you will need to run ``` docker inspect xxx```
tongli (Thu, 16 Nov 2017 14:29:26 GMT):
that way you will see the ip address.
ascatox (Thu, 16 Nov 2017 14:29:30 GMT):
Hi All! Someone can help me in configuring the worker host in order to use it with MacOs
ascatox (Thu, 16 Nov 2017 14:29:38 GMT):
I followed the guide
ascatox (Thu, 16 Nov 2017 14:29:44 GMT):
using the command
ascatox (Thu, 16 Nov 2017 14:30:05 GMT):
`$ docker run -d -v /var/run/docker.sock:/var/run/docker.sock -p 127.0.0.1:2375:2375 bobrik/socat TCP-LISTEN:2375,fork UNIX-CONNECT:/var/run/docker.sock
``
ascatox (Thu, 16 Nov 2017 14:30:05 GMT):
`$ docker run -d -v /var/run/docker.sock:/var/run/docker.sock -p 127.0.0.1:2375:2375 bobrik/socat TCP-LISTEN:2375,fork UNIX-CONNECT:/var/run/docker.sock``
ascatox (Thu, 16 Nov 2017 14:30:39 GMT):
but I can't connect to the worker from the host
tongli (Thu, 16 Nov 2017 14:31:56 GMT):
@MohammadObaid I would suggest to run the busybox container which gives you commands like nslookup and ping, so that you can test both dns services and container.
tongli (Thu, 16 Nov 2017 14:32:10 GMT):
you can also do these things to do a bit more tests.
tongli (Thu, 16 Nov 2017 14:32:32 GMT):
```http://tongworknotes.blogspot.com/2017/11/how-to-test-if-port-is-open-and-running.html```
tongli (Thu, 16 Nov 2017 14:32:35 GMT):
http://tongworknotes.blogspot.com/2017/11/how-to-test-if-port-is-open-and-running.html
tongli (Thu, 16 Nov 2017 14:33:03 GMT):
the little procedure can help you to test if a port is open or not.
tongli (Thu, 16 Nov 2017 14:33:30 GMT):
even if it is a secure port like 7050 when tls is on.
grapebaba (Thu, 16 Nov 2017 14:40:08 GMT):
@tongli
tongli (Thu, 16 Nov 2017 14:40:38 GMT):
@grapebaba hi, much better here for cello related discussions.
grapebaba (Thu, 16 Nov 2017 14:40:51 GMT):
Ok
tongli (Thu, 16 Nov 2017 14:41:47 GMT):
https://gerrit.hyperledger.org/r/#/c/15495/ that patch just got merged.
tongli (Thu, 16 Nov 2017 14:42:14 GMT):
now you can use k8s versions starting 1.7.0 all the way to 1.8.3 (which is the latest)
grapebaba (Thu, 16 Nov 2017 14:42:47 GMT):
OK
tongli (Thu, 16 Nov 2017 14:42:49 GMT):
if you have an AWS account, to provision, setup k8s and fabric, you only need to do the following
tongli (Thu, 16 Nov 2017 14:43:33 GMT):
```1. make changes to vars/aws.yml according to your aws cloud
2. export AWS_SECRET_KEY="your secret key of your aws account"
3. To stand up the fabric network::
ansible-playbook -e "mode=apply" awsk8s.yml
4. To tear down the fabric network::
ansible-playbook -e "mode=destroy" awsk8s.yml```
tongli (Thu, 16 Nov 2017 14:44:03 GMT):
notice the yml file to be awsk8s.yml
tongli (Thu, 16 Nov 2017 14:44:21 GMT):
that means you want to setup fabric network using k8s.
tongli (Thu, 16 Nov 2017 14:45:24 GMT):
if you want to setup fabric network using docker swarm like network, simply replace awsk8s.yml with aws.yml, exactly same commands.
tongli (Thu, 16 Nov 2017 14:45:32 GMT):
like the main agent doc says.
ascatox (Thu, 16 Nov 2017 14:47:40 GMT):
Sorry but I didn't understand a simple thing! Where can I configure the topology (peers, orderer etc) of my fabric blockchain by cello?
ascatox (Thu, 16 Nov 2017 14:47:53 GMT):
I have to modify some files
ascatox (Thu, 16 Nov 2017 14:47:55 GMT):
?
tongli (Thu, 16 Nov 2017 14:48:27 GMT):
@ascatox not sure how you stand up your fabric network. cello provides few different agent to create fabric network.
tongli (Thu, 16 Nov 2017 14:49:21 GMT):
the agent I have been talking about is the ansible agent. which can be found here. I am in the process of moving the doc to the doc directory. before that is done, you can find it here.
tongli (Thu, 16 Nov 2017 14:49:40 GMT):
https://github.com/hyperledger/cello/tree/master/src/agent/ansible
tongli (Thu, 16 Nov 2017 14:50:12 GMT):
if you really just have one server, you can also follow this article. https://github.com/hyperledger/cello/blob/master/src/agent/ansible/vars/All-in-one.md
tongli (Thu, 16 Nov 2017 14:50:43 GMT):
the all-in-one is a great procedure get you started if you do not have a lot of resources.
ascatox (Thu, 16 Nov 2017 14:51:42 GMT):
Thank you for the guide
MohammadObaid (Thu, 16 Nov 2017 14:51:56 GMT):
@tongli I will try those methods
tongli (Thu, 16 Nov 2017 14:52:01 GMT):
@ascatox seems to me that you are using mac os, to have multiple servers in a fabric network, you can also look at this doc https://github.com/hyperledger/cello/blob/master/src/agent/ansible/README.md
tongli (Thu, 16 Nov 2017 14:53:21 GMT):
like install virtualbox on your mac, and download this ubuntu image. https://ibm.box.com/s/xpvhmbozwjzn61vctyxp1eosku3bxx59
tongli (Thu, 16 Nov 2017 14:53:29 GMT):
then you can have a VM running in few minutes.
tongli (Thu, 16 Nov 2017 14:54:07 GMT):
look for virtualbox section to stand up 2 VM fabric network either using docker swarm like netwokr or k8s.
ascatox (Thu, 16 Nov 2017 14:54:22 GMT):
thanks a lot!
tongli (Thu, 16 Nov 2017 14:55:03 GMT):
@ascatox yeah, not a problem. let me know if you run into any problems. I would strongly recommend using virtual machines for your daily work.
ascatox (Thu, 16 Nov 2017 15:25:21 GMT):
[ ](https://chat.hyperledger.org/channel/cello?msg=uZR58awsbSAxzP9JF) @tongli Where can i find the password to access to the VM?
tongli (Thu, 16 Nov 2017 15:27:30 GMT):
@ascatox you downloaded my virtualbox image?
ascatox (Thu, 16 Nov 2017 15:27:35 GMT):
yes
tongli (Thu, 16 Nov 2017 15:27:41 GMT):
the username and password is ubuntu and ps
tongli (Thu, 16 Nov 2017 15:27:45 GMT):
very easy.
ascatox (Thu, 16 Nov 2017 15:27:45 GMT):
thanks
tongli (Thu, 16 Nov 2017 15:28:53 GMT):
@ascatox can you access this link? https://github.ibm.com/IBMCode/howtos/pull/78
tongli (Thu, 16 Nov 2017 15:29:33 GMT):
@ascatox probably not.
ascatox (Thu, 16 Nov 2017 15:29:34 GMT):
No I don't have IBM credentials
tongli (Thu, 16 Nov 2017 15:29:42 GMT):
it is an ibm internal site. sigh!
tongli (Thu, 16 Nov 2017 15:30:24 GMT):
@ascatox follow the all in one should be ok. if you have issues, let me know. I will help.
ascatox (Thu, 16 Nov 2017 15:30:39 GMT):
Thank you very much!!!
MohammadObaid (Thu, 16 Nov 2017 16:05:02 GMT):
@tongli From busybox you mean creating busybox container then from that container bash , ping other docker conatiners like zookeeper etc right?
tongli (Thu, 16 Nov 2017 16:05:27 GMT):
yes. using nslookup and ping to test.
tongli (Thu, 16 Nov 2017 16:05:58 GMT):
that way, you can see if your overlay network is working and dns services is fine.
MohammadObaid (Thu, 16 Nov 2017 16:15:49 GMT):
with busybox conatiner I am only able to ping localhost containers. Cannot ping those container which is running on other ubuntu server
tongli (Thu, 16 Nov 2017 16:23:42 GMT):
you can do this.
tongli (Thu, 16 Nov 2017 16:23:56 GMT):
```nslookup peer1st-orga```
tongli (Thu, 16 Nov 2017 16:25:32 GMT):
and see if you can get the ip address for the peer. or any other peer, or orderer
tongli (Thu, 16 Nov 2017 16:25:59 GMT):
also after you find the ip address by using docker inspect, you can ping the container using ip
tongli (Thu, 16 Nov 2017 16:26:20 GMT):
if you can not do either of these things, that means that the overlay network is probably gone down.
MohammadObaid (Thu, 16 Nov 2017 16:26:30 GMT):
when I did nslookup zookeeper2nd from fabric001 I get result `Address 1: 172.16.76.4 ip-172-16-76-4.ec2.internal`
tongli (Thu, 16 Nov 2017 16:26:41 GMT):
yeah.
tongli (Thu, 16 Nov 2017 16:27:15 GMT):
172.16.74.4 is the zookeeper2nd address, right?
MohammadObaid (Thu, 16 Nov 2017 16:27:19 GMT):
but I didnt see any reply when I ping to that ip
MohammadObaid (Thu, 16 Nov 2017 16:27:21 GMT):
yeah
tongli (Thu, 16 Nov 2017 16:27:21 GMT):
you can confirm that.
tongli (Thu, 16 Nov 2017 16:28:23 GMT):
right, if you can not ping by using IP address, that means that the flanneld overlay network has gone down for some reason.
tongli (Thu, 16 Nov 2017 16:28:34 GMT):
you can try to run commands like this to see the service
tongli (Thu, 16 Nov 2017 16:28:45 GMT):
```systemctl status flanneld```
tongli (Thu, 16 Nov 2017 16:29:03 GMT):
be able to do nslookup, means that your dns is fine.
tongli (Thu, 16 Nov 2017 16:29:17 GMT):
but it will be good to check that services as well.
tongli (Thu, 16 Nov 2017 16:29:33 GMT):
```systemctl status etcd```
tongli (Thu, 16 Nov 2017 16:29:42 GMT):
and also check dns container logs.
tongli (Thu, 16 Nov 2017 16:30:04 GMT):
worst case, do a destroy and apply again.
tongli (Thu, 16 Nov 2017 16:30:26 GMT):
and keep close eye on the network and see what made the overlay network going down.
MohammadObaid (Thu, 16 Nov 2017 16:31:50 GMT):
flanneld.png
MohammadObaid (Thu, 16 Nov 2017 16:31:53 GMT):
Seems like flanneld working correctly however I will try to do it again
tongli (Thu, 16 Nov 2017 16:59:56 GMT):
@MohammadObaid k.
rjones (Thu, 16 Nov 2017 21:58:13 GMT):
@tongli an invite has been sent to your IBM address. Please accept it to get your ACL created, welcome to Cello as a maintainer!
baohua (Thu, 16 Nov 2017 23:52:59 GMT):
Congrats to @tongli as new maintainer! look forward to seeing more active contributors!
blackdog1987 (Fri, 17 Nov 2017 01:14:15 GMT):
Has joined the channel.
vdods (Fri, 17 Nov 2017 02:36:15 GMT):
Has joined the channel.
ascatox (Fri, 17 Nov 2017 07:43:14 GMT):
Hi All! I'm using the README All-in-one.md but I can't find where to configure my fabric chain (number of peers, orderers ect)?
ascatox (Fri, 17 Nov 2017 07:43:14 GMT):
Hi All! I'm using the README All-in-one.md but I can't find where to configure my fabric chain (number of peers, orderers ect)
ascatox (Fri, 17 Nov 2017 07:43:29 GMT):
Someone can help me?
MohammadObaid (Fri, 17 Nov 2017 12:32:47 GMT):
@tongli I guess fabric require docker version 17 or above but network constructed by ansible installs docker 1.13.1 on ubuntu servers. Isnt this cause any problems?
tongli (Fri, 17 Nov 2017 14:50:15 GMT):
@MohammadObaid no. version 17 I believe is docker ce.
tongli (Fri, 17 Nov 2017 14:50:32 GMT):
docker.io latest I think is 1.13
MohammadObaid (Fri, 17 Nov 2017 16:53:03 GMT):
flannelport.png
MohammadObaid (Fri, 17 Nov 2017 16:53:41 GMT):
@tongli Alright. Flannel uses port 8285. I should see some rules related to it in my iptables right?
tongli (Fri, 17 Nov 2017 16:54:15 GMT):
I believe our aws ansible script already does that.
MohammadObaid (Fri, 17 Nov 2017 16:57:47 GMT):
hmm. I suspect my flannel network broken after sometime. From my busybox container I tried to ping flannel0 ip. ALthough ping is normal but tcpdump on other server shows nothing which lead me to believe that flannel network unable to communicate
tongli (Fri, 17 Nov 2017 16:58:11 GMT):
please take a look at the flanneld service logs
MohammadObaid (Fri, 17 Nov 2017 17:03:20 GMT):
flannellogs.png
tongli (Fri, 17 Nov 2017 17:04:37 GMT):
@MohammadObaid not seeing any errors there. Do not know why it fails.
KonstantinWallach (Sat, 18 Nov 2017 06:48:57 GMT):
Has joined the channel.
miguelcalero (Sat, 18 Nov 2017 06:56:57 GMT):
Has joined the channel.
MohammadObaid (Sat, 18 Nov 2017 15:03:28 GMT):
Finally fixed that issue. It was due to iptables setting which blocks icmp packets to be forwarded from flannel interface to docker interface. By adding iptable rules everything works fine for me . However everytime when ansible created Ubuntu instance I need to quickly ssh into my instances and need to add rules before ansible go for setting fabric architecture.
heidecke (Sun, 19 Nov 2017 09:25:33 GMT):
Has joined the channel.
saadmir (Mon, 20 Nov 2017 02:54:43 GMT):
Has joined the channel.
sravs (Mon, 20 Nov 2017 11:27:58 GMT):
Iam not able to add host in the dashboard, please anyone help me
MohammadObaid (Mon, 20 Nov 2017 14:35:40 GMT):
@tongli I observed that when I modify bc1.yml file as below , only organization1 being able to invoke transaction but when I moved all anchor peers to fabric001 , all organization can invoke transactions. Cant we define anchors peer in other server like one acnhor peer in each servers? Here is bc1.yml code ``` network: {
fabric001: {
cas: ["ca1st.orga"],
peers: ["anchor@peer1st.orga","worker@peer1st.orga","worker@peer2nd.orga"],
orderers: ["orderer1st.orgc", "orderer1st.orgd"],
zookeepers: ["zookeeper1st"],
kafkas: ["kafka1st"]
},
fabric002: {
cas: ["ca1st.orgb"],
peers: ["anchor@peer1st.orgb", "worker@peer1st.orgb","worker@peer2nd.orgb"],
orderers: ["orderer2nd.orgc", "orderer2nd.orgd"],
zookeepers: ["zookeeper2nd"],
kafkas: ["kafka2nd"]
},
fabric003: {
cas: ["ca1st.orgc", "ca1st.orgd"],
peers: [],
orderers: ["orderer3rth.orgc", "orderer3rth.orgd"],
zookeepers: ["zookeeper3rd"],
kafkas: ["kafka3rd"]
},
fabric004: {
cas: ["ca1st.orgf"],
peers: ["anchor@peer1st.orgf","worker@peer1st.orgf","worker@peer2nd.orgf"],
orderers: ["orderer4th.orgc", "orderer4th.orgd"],
zookeepers: [],
kafkas: []
}
} ```
MohammadObaid (Mon, 20 Nov 2017 14:35:40 GMT):
@tongli I observed that when I modify bc1.yml file as below , only organization1 being able to invoke transactions but when I moved all anchor peers to fabric001 server ,then all organization can invoke transactions. Cant we define anchors peer in other servers like one anchor peer in each server? Here is bc1.yml code ``` network: {
fabric001: {
cas: ["ca1st.orga"],
peers: ["anchor@peer1st.orga","worker@peer1st.orga","worker@peer2nd.orga"],
orderers: ["orderer1st.orgc", "orderer1st.orgd"],
zookeepers: ["zookeeper1st"],
kafkas: ["kafka1st"]
},
fabric002: {
cas: ["ca1st.orgb"],
peers: ["anchor@peer1st.orgb", "worker@peer1st.orgb","worker@peer2nd.orgb"],
orderers: ["orderer2nd.orgc", "orderer2nd.orgd"],
zookeepers: ["zookeeper2nd"],
kafkas: ["kafka2nd"]
},
fabric003: {
cas: ["ca1st.orgc", "ca1st.orgd"],
peers: [],
orderers: ["orderer3rth.orgc", "orderer3rth.orgd"],
zookeepers: ["zookeeper3rd"],
kafkas: ["kafka3rd"]
},
fabric004: {
cas: ["ca1st.orgf"],
peers: ["anchor@peer1st.orgf","worker@peer1st.orgf","worker@peer2nd.orgf"],
orderers: ["orderer4th.orgc", "orderer4th.orgd"],
zookeepers: [],
kafkas: []
}
} ```
baohua (Mon, 20 Nov 2017 14:39:18 GMT):
@sravs, u need to post the steps and logs, then the team can try to help give suggestions.
tongli (Mon, 20 Nov 2017 15:59:31 GMT):
@MohammadObaid I am not quite sure what you are trying to do. you did not have orgf when you setup the network, then you run the same script again after you added orgf? if that is the case, I am pretty sure something will be wrong.
MohammadObaid (Mon, 20 Nov 2017 16:00:25 GMT):
@tongli I edit this bc1st.yml file and re run everything from scratch
tongli (Mon, 20 Nov 2017 16:00:38 GMT):
the ansible agent as of now, stand up a brand new network and assumed that the configuration such as bc1st.yml not changing. if you change it , then you will need to destroy the old one and stand up a new.
tongli (Mon, 20 Nov 2017 16:01:06 GMT):
@MohammadObaid did you run the destroy to get rid of everything and stand up a new one?
tongli (Mon, 20 Nov 2017 16:03:38 GMT):
@MohammadObaid from your network layout file, I do not see any problems.
tongli (Mon, 20 Nov 2017 16:03:56 GMT):
each org can have a anchor peer
MohammadObaid (Mon, 20 Nov 2017 16:04:34 GMT):
Yup .I just want to know we can define anchor peers in any ubuntu insatnce right?
tongli (Mon, 20 Nov 2017 16:05:42 GMT):
@MohammadObaid yeah, anchor peer can be allocated on any node.
MohammadObaid (Mon, 20 Nov 2017 16:09:33 GMT):
Alright . Another thing I was looking for is to modify endorsment policy . For endorsment policy , I have to make changes only in dochannel.js located in `roles/deploy_compose/fabricsetup/tempelates/` right ?
MohammadObaid (Mon, 20 Nov 2017 16:09:33 GMT):
@tongli Alright . Another thing I was looking for is to modify endorsment policy . For endorsment policy , I have to make changes only in dochannel.js located in `roles/deploy_compose/fabricsetup/tempelates/` right ?
tongli (Mon, 20 Nov 2017 21:56:35 GMT):
@MohammadObaid you can do that only when you need to set it up initially. I think you can change the policy after the network is setup. but I am not the expert on the policy.
tongli (Mon, 20 Nov 2017 21:57:08 GMT):
@here, can any one help with policy change command to give @MohammadObaid some help? I like to hear that as well.
tongli (Mon, 20 Nov 2017 21:57:56 GMT):
@baohua @hightall can you review these two patch sets? https://gerrit.hyperledger.org/r/#/c/15611/
tongli (Mon, 20 Nov 2017 21:58:10 GMT):
https://gerrit.hyperledger.org/r/#/c/15623/
tongli (Mon, 20 Nov 2017 21:58:58 GMT):
the first one is to merge the ansible agent doc to the main doc directory which Baohua has asked me to do for awhile. the second is the one that is needed for the CI. Please review. thanks.
baohua (Tue, 21 Nov 2017 01:30:12 GMT):
@MohammadObaid the endorsement policy is usually given during instantiate. I do not know if we can change it after cc bootup.
MohammadObaid (Tue, 21 Nov 2017 06:17:55 GMT):
@baohua @tongli I tried to change endorsement policy mentioned in dochannel.j2 as follows but failed ```peer chaincode instantiate -o {{ cliorderer.name }}:7050 \
-C firstchannel -n firstchaincode -v 1.0 -c '{"Args":["initData","dumy", "dummy", "dummy", "1" , "dummy"]}' -P "OR ('orga.member','orgb.member')"
MohammadObaid (Tue, 21 Nov 2017 06:17:55 GMT):
@baohua @tongli I tried to change endorsement policy mentioned in dochannel.j2 as follows during bootup but failed ```peer chaincode instantiate -o {{ cliorderer.name }}:7050 \
-C firstchannel -n firstchaincode -v 1.0 -c '{"Args":["initData","dumy", "dummy", "dummy", "1" , "dummy"]}' -P "OR ('orga.member','orgb.member')"
sravs (Tue, 21 Nov 2017 06:20:33 GMT):
thankyou @baohua i am following this link https://github.com/hyperledger/cello/blob/master/docs/installation.md
sravs (Tue, 21 Nov 2017 06:22:48 GMT):
1.png
sravs (Tue, 21 Nov 2017 06:22:57 GMT):
2.png
sravs (Tue, 21 Nov 2017 06:23:07 GMT):
3.png
sravs (Tue, 21 Nov 2017 06:23:19 GMT):
4.png
qingsongGuo (Tue, 21 Nov 2017 08:10:16 GMT):
Has joined the channel.
sravs (Tue, 21 Nov 2017 11:25:26 GMT):
@baohua help me to go forward in cello, thankyou in advance.
tongli (Tue, 21 Nov 2017 14:33:21 GMT):
@hightall can you please review the two patches. we are kind waiting on the them especially the second one. https://gerrit.hyperledger.org/r/#/c/15611/ https://gerrit.hyperledger.org/r/#/c/15623/
tongli (Tue, 21 Nov 2017 14:42:41 GMT):
Thanks @baohua @hightall .
Raje (Tue, 21 Nov 2017 22:59:36 GMT):
Has joined the channel.
Raje (Tue, 21 Nov 2017 23:11:05 GMT): baohua (Wed, 22 Nov 2017 04:20:06 GMT): baohua (Wed, 22 Nov 2017 04:20:54 GMT): baohua (Wed, 22 Nov 2017 04:21:23 GMT): baohua (Wed, 22 Nov 2017 04:22:01 GMT): baohua (Wed, 22 Nov 2017 04:25:37 GMT): baohua (Wed, 22 Nov 2017 04:25:57 GMT): sravs (Wed, 22 Nov 2017 05:41:06 GMT): Manasabingi (Wed, 22 Nov 2017 07:28:47 GMT): atian15 (Wed, 22 Nov 2017 08:16:36 GMT): rajasekharpippalla (Wed, 22 Nov 2017 09:08:41 GMT): rajasekharpippalla (Wed, 22 Nov 2017 11:06:46 GMT): sravs (Wed, 22 Nov 2017 14:20:41 GMT): tongli (Wed, 22 Nov 2017 14:24:45 GMT): tongli (Wed, 22 Nov 2017 14:25:13 GMT): tongli (Wed, 22 Nov 2017 14:25:37 GMT): tongli (Wed, 22 Nov 2017 14:25:47 GMT): tongli (Wed, 22 Nov 2017 14:26:18 GMT): tongli (Wed, 22 Nov 2017 14:27:07 GMT): MohammadObaid (Wed, 22 Nov 2017 16:04:34 GMT): tongli (Wed, 22 Nov 2017 16:05:55 GMT): tongli (Wed, 22 Nov 2017 16:06:13 GMT): tongli (Wed, 22 Nov 2017 16:07:12 GMT): MohammadObaid (Wed, 22 Nov 2017 16:09:18 GMT): tongli (Wed, 22 Nov 2017 16:11:28 GMT): tongli (Wed, 22 Nov 2017 16:12:54 GMT): tongli (Wed, 22 Nov 2017 16:13:11 GMT): tongli (Wed, 22 Nov 2017 16:13:23 GMT): tongli (Wed, 22 Nov 2017 16:13:49 GMT): tongli (Wed, 22 Nov 2017 16:14:21 GMT): tongli (Wed, 22 Nov 2017 16:14:59 GMT): zonkedhazmat (Thu, 23 Nov 2017 05:18:19 GMT): zonkedhazmat (Thu, 23 Nov 2017 05:20:27 GMT): MoulaliMvg (Thu, 23 Nov 2017 05:28:40 GMT): baohua (Thu, 23 Nov 2017 05:42:39 GMT): harsha (Thu, 23 Nov 2017 06:39:03 GMT): MohammadObaid (Thu, 23 Nov 2017 17:49:05 GMT): MohammadObaid (Thu, 23 Nov 2017 17:49:05 GMT): MohammadObaid (Thu, 23 Nov 2017 17:49:43 GMT): MohammadObaid (Thu, 23 Nov 2017 17:52:45 GMT): baohua (Fri, 24 Nov 2017 02:00:30 GMT): baohua (Fri, 24 Nov 2017 02:00:48 GMT): harsha (Fri, 24 Nov 2017 02:01:10 GMT): kong-bo (Sat, 25 Nov 2017 06:47:56 GMT): baohua (Mon, 27 Nov 2017 01:20:27 GMT): ambatigaan (Mon, 27 Nov 2017 14:39:54 GMT): ambatigaan (Mon, 27 Nov 2017 14:47:42 GMT): tongli (Mon, 27 Nov 2017 14:58:39 GMT): MohammadObaid (Mon, 27 Nov 2017 15:39:17 GMT): MohammadObaid (Mon, 27 Nov 2017 15:39:17 GMT): Luke_Chen (Mon, 27 Nov 2017 15:42:31 GMT): Luke_Chen (Mon, 27 Nov 2017 15:44:13 GMT): ambatigaan (Mon, 27 Nov 2017 15:49:04 GMT): tongli (Mon, 27 Nov 2017 16:54:49 GMT): tongli (Mon, 27 Nov 2017 16:55:33 GMT): tongli (Mon, 27 Nov 2017 16:55:41 GMT): MohammadObaid (Mon, 27 Nov 2017 17:41:38 GMT): MohammadObaid (Mon, 27 Nov 2017 17:41:52 GMT): tongli (Mon, 27 Nov 2017 17:43:12 GMT): tongli (Mon, 27 Nov 2017 17:44:07 GMT): MohammadObaid (Mon, 27 Nov 2017 17:47:49 GMT): MohammadObaid (Mon, 27 Nov 2017 17:48:15 GMT): tongli (Mon, 27 Nov 2017 17:57:32 GMT): MohammadObaid (Mon, 27 Nov 2017 19:23:48 GMT): MohammadObaid (Mon, 27 Nov 2017 19:24:09 GMT): tongli (Mon, 27 Nov 2017 19:39:24 GMT): tongli (Mon, 27 Nov 2017 19:40:04 GMT): tongli (Mon, 27 Nov 2017 19:40:18 GMT): MohammadObaid (Mon, 27 Nov 2017 19:46:10 GMT): tongli (Mon, 27 Nov 2017 21:24:39 GMT): tongli (Mon, 27 Nov 2017 21:24:43 GMT): tongli (Mon, 27 Nov 2017 21:25:08 GMT): tongli (Mon, 27 Nov 2017 23:40:43 GMT): tongli (Mon, 27 Nov 2017 23:41:09 GMT): rjones (Mon, 27 Nov 2017 23:43:06 GMT): oujunketu (Tue, 28 Nov 2017 00:15:00 GMT): Luke_Chen (Tue, 28 Nov 2017 07:43:58 GMT): Luke_Chen (Tue, 28 Nov 2017 07:46:20 GMT): SimonOberzan (Tue, 28 Nov 2017 08:13:34 GMT): smallant (Tue, 28 Nov 2017 11:04:03 GMT): tongli (Tue, 28 Nov 2017 14:49:05 GMT): tongli (Tue, 28 Nov 2017 14:49:17 GMT): tongli (Tue, 28 Nov 2017 14:49:42 GMT): tongli (Tue, 28 Nov 2017 14:51:32 GMT): tongli (Tue, 28 Nov 2017 14:51:34 GMT): SimonOberzan (Tue, 28 Nov 2017 14:52:06 GMT): SimonOberzan (Tue, 28 Nov 2017 14:52:51 GMT): SimonOberzan (Tue, 28 Nov 2017 14:52:51 GMT): SimonOberzan (Tue, 28 Nov 2017 15:40:07 GMT): tongli (Tue, 28 Nov 2017 15:41:12 GMT): tongli (Tue, 28 Nov 2017 15:41:25 GMT): SimonOberzan (Tue, 28 Nov 2017 15:41:44 GMT): tongli (Tue, 28 Nov 2017 15:41:46 GMT): tongli (Tue, 28 Nov 2017 15:42:11 GMT): tongli (Tue, 28 Nov 2017 15:42:44 GMT): SimonOberzan (Tue, 28 Nov 2017 15:42:49 GMT): tongli (Tue, 28 Nov 2017 15:43:11 GMT): tongli (Tue, 28 Nov 2017 15:43:16 GMT): tongli (Tue, 28 Nov 2017 15:43:25 GMT): SimonOberzan (Tue, 28 Nov 2017 15:43:50 GMT): tongli (Tue, 28 Nov 2017 15:43:56 GMT): tongli (Tue, 28 Nov 2017 15:44:13 GMT): SimonOberzan (Tue, 28 Nov 2017 15:44:24 GMT): tongli (Tue, 28 Nov 2017 15:44:51 GMT): tongli (Tue, 28 Nov 2017 15:44:57 GMT): tongli (Tue, 28 Nov 2017 15:45:10 GMT): SimonOberzan (Tue, 28 Nov 2017 15:45:28 GMT): MohammadObaid (Tue, 28 Nov 2017 15:52:51 GMT): tongli (Tue, 28 Nov 2017 16:00:40 GMT): tongli (Tue, 28 Nov 2017 16:00:48 GMT): tongli (Tue, 28 Nov 2017 16:01:29 GMT): baohua (Wed, 29 Nov 2017 05:41:17 GMT): baohua (Wed, 29 Nov 2017 05:41:26 GMT): zeroest (Wed, 29 Nov 2017 07:50:13 GMT): tongli (Wed, 29 Nov 2017 13:40:37 GMT): SimonOberzan (Wed, 29 Nov 2017 14:38:55 GMT): SimonOberzan (Wed, 29 Nov 2017 14:38:55 GMT): SimonOberzan (Wed, 29 Nov 2017 14:38:55 GMT): SimonOberzan (Wed, 29 Nov 2017 14:38:55 GMT): SimonOberzan (Wed, 29 Nov 2017 14:38:55 GMT): SimonOberzan (Wed, 29 Nov 2017 14:38:55 GMT): SimonOberzan (Wed, 29 Nov 2017 14:38:55 GMT): SimonOberzan (Wed, 29 Nov 2017 14:38:55 GMT): SimonOberzan (Wed, 29 Nov 2017 14:38:55 GMT): SimonOberzan (Wed, 29 Nov 2017 14:43:35 GMT): SimonOberzan (Wed, 29 Nov 2017 15:13:30 GMT): tongli (Wed, 29 Nov 2017 15:19:04 GMT): SimonOberzan (Wed, 29 Nov 2017 15:19:18 GMT): tongli (Wed, 29 Nov 2017 15:20:04 GMT): SimonOberzan (Wed, 29 Nov 2017 15:20:23 GMT): SimonOberzan (Wed, 29 Nov 2017 15:20:23 GMT): tongli (Wed, 29 Nov 2017 15:22:36 GMT): SimonOberzan (Wed, 29 Nov 2017 15:24:34 GMT): SimonOberzan (Wed, 29 Nov 2017 15:24:34 GMT): tongli (Wed, 29 Nov 2017 15:26:36 GMT): tongli (Wed, 29 Nov 2017 15:27:27 GMT): tongli (Wed, 29 Nov 2017 15:27:45 GMT): tongli (Wed, 29 Nov 2017 15:28:06 GMT): SimonOberzan (Wed, 29 Nov 2017 15:28:43 GMT): SimonOberzan (Wed, 29 Nov 2017 15:28:59 GMT): SimonOberzan (Wed, 29 Nov 2017 15:30:07 GMT): SimonOberzan (Wed, 29 Nov 2017 15:30:07 GMT): SimonOberzan (Wed, 29 Nov 2017 15:30:07 GMT): SimonOberzan (Wed, 29 Nov 2017 15:30:07 GMT): MohammadObaid (Wed, 29 Nov 2017 15:38:28 GMT): MohammadObaid (Wed, 29 Nov 2017 15:38:28 GMT): MohammadObaid (Wed, 29 Nov 2017 15:38:28 GMT): tongli (Wed, 29 Nov 2017 15:38:44 GMT): tongli (Wed, 29 Nov 2017 15:39:46 GMT): SimonOberzan (Wed, 29 Nov 2017 15:40:16 GMT): tongli (Wed, 29 Nov 2017 15:40:29 GMT): tongli (Wed, 29 Nov 2017 15:41:22 GMT): tongli (Wed, 29 Nov 2017 15:42:05 GMT): tongli (Wed, 29 Nov 2017 15:42:37 GMT): SimonOberzan (Wed, 29 Nov 2017 15:42:55 GMT): tongli (Wed, 29 Nov 2017 15:43:39 GMT): SimonOberzan (Wed, 29 Nov 2017 15:43:59 GMT): tongli (Wed, 29 Nov 2017 15:44:23 GMT): MohammadObaid (Wed, 29 Nov 2017 15:44:35 GMT): tongli (Wed, 29 Nov 2017 15:45:03 GMT): tongli (Wed, 29 Nov 2017 15:45:18 GMT): tongli (Wed, 29 Nov 2017 15:45:23 GMT): tongli (Wed, 29 Nov 2017 15:45:33 GMT): tongli (Wed, 29 Nov 2017 15:45:52 GMT): tongli (Wed, 29 Nov 2017 15:46:24 GMT): tongli (Wed, 29 Nov 2017 15:46:31 GMT): sdarwin (Wed, 29 Nov 2017 15:46:36 GMT): MohammadObaid (Wed, 29 Nov 2017 15:46:53 GMT): sdarwin (Wed, 29 Nov 2017 15:47:04 GMT): tongli (Wed, 29 Nov 2017 15:47:04 GMT): sdarwin (Wed, 29 Nov 2017 15:47:13 GMT): sdarwin (Wed, 29 Nov 2017 15:47:27 GMT): tongli (Wed, 29 Nov 2017 15:47:34 GMT): tongli (Wed, 29 Nov 2017 15:47:52 GMT): sdarwin (Wed, 29 Nov 2017 15:48:05 GMT): tongli (Wed, 29 Nov 2017 15:48:15 GMT): sdarwin (Wed, 29 Nov 2017 15:48:19 GMT): tongli (Wed, 29 Nov 2017 15:48:25 GMT): SimonOberzan (Wed, 29 Nov 2017 15:48:37 GMT): sdarwin (Wed, 29 Nov 2017 15:48:38 GMT): tongli (Wed, 29 Nov 2017 15:48:52 GMT): sdarwin (Wed, 29 Nov 2017 15:49:24 GMT): sdarwin (Wed, 29 Nov 2017 15:49:43 GMT): SimonOberzan (Wed, 29 Nov 2017 15:49:56 GMT): tongli (Wed, 29 Nov 2017 15:50:49 GMT): tongli (Wed, 29 Nov 2017 15:51:34 GMT): SimonOberzan (Wed, 29 Nov 2017 15:52:07 GMT): tongli (Wed, 29 Nov 2017 15:52:18 GMT): SimonOberzan (Wed, 29 Nov 2017 15:52:19 GMT): SimonOberzan (Wed, 29 Nov 2017 15:52:33 GMT): tongli (Wed, 29 Nov 2017 15:52:42 GMT): SimonOberzan (Wed, 29 Nov 2017 15:53:00 GMT): tongli (Wed, 29 Nov 2017 15:53:08 GMT): SimonOberzan (Wed, 29 Nov 2017 15:53:19 GMT): tongli (Wed, 29 Nov 2017 15:53:32 GMT): SimonOberzan (Wed, 29 Nov 2017 15:53:57 GMT): SimonOberzan (Wed, 29 Nov 2017 15:54:36 GMT): tongli (Wed, 29 Nov 2017 15:54:54 GMT): tongli (Wed, 29 Nov 2017 15:55:25 GMT): tongli (Wed, 29 Nov 2017 15:55:43 GMT): tongli (Wed, 29 Nov 2017 15:55:48 GMT): tongli (Wed, 29 Nov 2017 15:56:42 GMT): tongli (Wed, 29 Nov 2017 15:57:15 GMT): tongli (Wed, 29 Nov 2017 15:57:28 GMT): tongli (Wed, 29 Nov 2017 15:57:35 GMT): tongli (Wed, 29 Nov 2017 15:57:46 GMT): tongli (Wed, 29 Nov 2017 15:58:12 GMT): SimonOberzan (Wed, 29 Nov 2017 15:58:21 GMT): tongli (Wed, 29 Nov 2017 15:58:31 GMT): tongli (Wed, 29 Nov 2017 15:58:54 GMT): SimonOberzan (Wed, 29 Nov 2017 15:59:21 GMT): tongli (Wed, 29 Nov 2017 16:00:33 GMT): SimonOberzan (Wed, 29 Nov 2017 16:00:50 GMT): tongli (Wed, 29 Nov 2017 16:00:55 GMT): tongli (Wed, 29 Nov 2017 16:00:58 GMT): tongli (Wed, 29 Nov 2017 16:01:18 GMT): SimonOberzan (Wed, 29 Nov 2017 16:01:18 GMT): tongli (Wed, 29 Nov 2017 16:01:36 GMT): SimonOberzan (Wed, 29 Nov 2017 16:02:24 GMT): tongli (Wed, 29 Nov 2017 16:02:47 GMT): SimonOberzan (Wed, 29 Nov 2017 16:02:58 GMT): tongli (Wed, 29 Nov 2017 16:03:15 GMT): tongli (Wed, 29 Nov 2017 16:03:48 GMT): SimonOberzan (Wed, 29 Nov 2017 16:03:48 GMT): tongli (Wed, 29 Nov 2017 16:04:06 GMT): tongli (Wed, 29 Nov 2017 16:04:56 GMT): tongli (Wed, 29 Nov 2017 16:05:06 GMT): tongli (Wed, 29 Nov 2017 16:05:14 GMT): tongli (Wed, 29 Nov 2017 16:05:23 GMT): tongli (Wed, 29 Nov 2017 16:05:46 GMT): SimonOberzan (Wed, 29 Nov 2017 16:05:58 GMT): tongli (Wed, 29 Nov 2017 16:06:03 GMT): tongli (Wed, 29 Nov 2017 16:06:23 GMT): SimonOberzan (Wed, 29 Nov 2017 16:06:31 GMT): SimonOberzan (Wed, 29 Nov 2017 16:06:51 GMT): tongli (Wed, 29 Nov 2017 16:07:41 GMT): tongli (Wed, 29 Nov 2017 16:08:01 GMT): SimonOberzan (Wed, 29 Nov 2017 16:08:14 GMT): SimonOberzan (Wed, 29 Nov 2017 16:09:05 GMT): tongli (Wed, 29 Nov 2017 16:10:56 GMT): tongli (Wed, 29 Nov 2017 16:11:05 GMT): tongli (Wed, 29 Nov 2017 16:11:32 GMT): tongli (Wed, 29 Nov 2017 16:11:43 GMT): tongli (Wed, 29 Nov 2017 16:12:04 GMT): SimonOberzan (Wed, 29 Nov 2017 16:12:12 GMT): tongli (Wed, 29 Nov 2017 16:12:32 GMT): tongli (Wed, 29 Nov 2017 16:13:21 GMT): tongli (Wed, 29 Nov 2017 16:14:35 GMT): tongli (Wed, 29 Nov 2017 16:15:07 GMT): SimonOberzan (Wed, 29 Nov 2017 16:17:23 GMT): tongli (Wed, 29 Nov 2017 16:21:29 GMT): tongli (Wed, 29 Nov 2017 16:22:17 GMT): tongli (Wed, 29 Nov 2017 16:22:48 GMT): compleatang (Wed, 29 Nov 2017 19:56:03 GMT): Russell-Columbia (Wed, 29 Nov 2017 23:00:18 GMT): tongli (Wed, 29 Nov 2017 23:41:46 GMT): tongli (Thu, 30 Nov 2017 01:53:19 GMT): miiiiiyoung (Thu, 30 Nov 2017 07:39:51 GMT): baohua (Thu, 30 Nov 2017 09:34:50 GMT): baohua (Thu, 30 Nov 2017 09:35:12 GMT): MohammadObaid (Thu, 30 Nov 2017 12:07:56 GMT): sdarwin (Thu, 30 Nov 2017 13:21:57 GMT): sdarwin (Thu, 30 Nov 2017 13:23:27 GMT): harsha (Thu, 30 Nov 2017 13:33:32 GMT): tongli (Thu, 30 Nov 2017 14:48:39 GMT): tongli (Thu, 30 Nov 2017 14:49:53 GMT): tongli (Thu, 30 Nov 2017 14:50:35 GMT): tongli (Thu, 30 Nov 2017 14:51:07 GMT): SimonOberzan (Thu, 30 Nov 2017 15:02:18 GMT): tongli (Thu, 30 Nov 2017 15:04:24 GMT): tongli (Thu, 30 Nov 2017 15:04:52 GMT): tongli (Thu, 30 Nov 2017 15:05:15 GMT): tongli (Thu, 30 Nov 2017 15:05:41 GMT): SimonOberzan (Thu, 30 Nov 2017 15:05:57 GMT): tongli (Thu, 30 Nov 2017 15:06:00 GMT): tongli (Thu, 30 Nov 2017 15:06:40 GMT): tongli (Thu, 30 Nov 2017 15:06:45 GMT): tongli (Thu, 30 Nov 2017 15:06:55 GMT): harsha (Thu, 30 Nov 2017 15:07:13 GMT): SimonOberzan (Thu, 30 Nov 2017 15:07:49 GMT): tongli (Thu, 30 Nov 2017 15:08:00 GMT): tongli (Thu, 30 Nov 2017 15:08:44 GMT): tongli (Thu, 30 Nov 2017 15:09:20 GMT): tongli (Thu, 30 Nov 2017 15:09:47 GMT): harsha (Thu, 30 Nov 2017 15:10:13 GMT): baohua (Thu, 30 Nov 2017 15:10:38 GMT): baohua (Thu, 30 Nov 2017 15:11:05 GMT): SimonOberzan (Thu, 30 Nov 2017 15:11:43 GMT): SimonOberzan (Thu, 30 Nov 2017 15:11:43 GMT): baohua (Thu, 30 Nov 2017 15:11:53 GMT): smallant (Thu, 30 Nov 2017 15:12:01 GMT): smallant (Thu, 30 Nov 2017 15:12:01 GMT): SimonOberzan (Thu, 30 Nov 2017 15:13:08 GMT): SimonOberzan (Thu, 30 Nov 2017 15:14:01 GMT): SimonOberzan (Thu, 30 Nov 2017 15:14:01 GMT): SimonOberzan (Thu, 30 Nov 2017 15:14:01 GMT): SimonOberzan (Thu, 30 Nov 2017 15:14:01 GMT): sdarwin (Thu, 30 Nov 2017 15:17:15 GMT): tongli (Thu, 30 Nov 2017 15:18:50 GMT): smallant (Thu, 30 Nov 2017 15:19:36 GMT): sdarwin (Thu, 30 Nov 2017 15:22:43 GMT): tongli (Thu, 30 Nov 2017 15:23:46 GMT): tongli (Thu, 30 Nov 2017 15:24:05 GMT): sdarwin (Thu, 30 Nov 2017 15:24:32 GMT): tongli (Thu, 30 Nov 2017 15:24:46 GMT): sdarwin (Thu, 30 Nov 2017 15:25:15 GMT): sdarwin (Thu, 30 Nov 2017 15:25:19 GMT): sdarwin (Thu, 30 Nov 2017 15:28:16 GMT): SimonOberzan (Thu, 30 Nov 2017 15:31:10 GMT): tongli (Thu, 30 Nov 2017 15:55:04 GMT): tongli (Thu, 30 Nov 2017 15:56:02 GMT): SimonOberzan (Thu, 30 Nov 2017 16:12:43 GMT): BlockchainMuggle (Thu, 30 Nov 2017 21:41:14 GMT): iamprem (Thu, 30 Nov 2017 21:54:01 GMT): baohua (Fri, 01 Dec 2017 01:42:39 GMT): harsha (Fri, 01 Dec 2017 02:39:41 GMT): qiushaoxi (Fri, 01 Dec 2017 03:17:18 GMT): qiushaoxi (Fri, 01 Dec 2017 03:18:10 GMT): baohua (Fri, 01 Dec 2017 03:18:51 GMT): qiushaoxi (Fri, 01 Dec 2017 03:19:56 GMT): qiushaoxi (Fri, 01 Dec 2017 03:20:05 GMT): baohua (Fri, 01 Dec 2017 03:25:11 GMT): qiushaoxi (Fri, 01 Dec 2017 03:27:59 GMT): SimonOberzan (Fri, 01 Dec 2017 12:27:56 GMT): SimonOberzan (Fri, 01 Dec 2017 15:43:24 GMT): tongli (Fri, 01 Dec 2017 16:00:32 GMT): tongli (Fri, 01 Dec 2017 16:01:24 GMT): tongli (Fri, 01 Dec 2017 16:01:53 GMT): SimonOberzan (Fri, 01 Dec 2017 17:44:58 GMT): harsha (Fri, 01 Dec 2017 17:48:02 GMT): harsha (Fri, 01 Dec 2017 17:53:37 GMT): SimonOberzan (Fri, 01 Dec 2017 18:05:49 GMT): SimonOberzan (Fri, 01 Dec 2017 19:08:40 GMT): SimonOberzan (Fri, 01 Dec 2017 19:08:40 GMT): SimonOberzan (Fri, 01 Dec 2017 19:08:40 GMT): yoyokeen (Sat, 02 Dec 2017 12:22:04 GMT): yoyokeen (Sat, 02 Dec 2017 12:22:55 GMT): baohua (Sat, 02 Dec 2017 14:38:58 GMT): baohua (Sat, 02 Dec 2017 14:55:17 GMT): baohua (Sat, 02 Dec 2017 14:55:37 GMT): baohua (Sat, 02 Dec 2017 14:56:01 GMT): yipcma (Sun, 03 Dec 2017 10:30:25 GMT): yoyokeen (Sun, 03 Dec 2017 11:32:18 GMT): tongli (Sun, 03 Dec 2017 21:57:29 GMT): baohua (Mon, 04 Dec 2017 01:08:33 GMT): baohua (Mon, 04 Dec 2017 01:10:06 GMT): NikkiNikki (Mon, 04 Dec 2017 07:31:03 GMT): liujingkun2017 (Mon, 04 Dec 2017 07:44:41 GMT): liujingkun2017 (Mon, 04 Dec 2017 07:45:05 GMT): liujingkun2017 (Mon, 04 Dec 2017 07:45:45 GMT): yipcma (Mon, 04 Dec 2017 08:16:43 GMT): yipcma (Mon, 04 Dec 2017 08:16:43 GMT): yipcma (Mon, 04 Dec 2017 08:17:13 GMT): yipcma (Mon, 04 Dec 2017 08:24:29 GMT): harsha (Mon, 04 Dec 2017 10:06:24 GMT): yipcma (Mon, 04 Dec 2017 10:06:45 GMT): yipcma (Mon, 04 Dec 2017 10:07:16 GMT): harsha (Mon, 04 Dec 2017 10:08:56 GMT): yipcma (Mon, 04 Dec 2017 10:09:50 GMT): yipcma (Mon, 04 Dec 2017 10:25:05 GMT): yipcma (Mon, 04 Dec 2017 10:25:14 GMT): harsha (Mon, 04 Dec 2017 10:29:18 GMT): harsha (Mon, 04 Dec 2017 10:29:18 GMT): yipcma (Mon, 04 Dec 2017 10:29:32 GMT): yipcma (Mon, 04 Dec 2017 10:30:15 GMT): yipcma (Mon, 04 Dec 2017 10:30:53 GMT): harsha (Mon, 04 Dec 2017 10:32:18 GMT): yipcma (Mon, 04 Dec 2017 10:32:41 GMT): yipcma (Mon, 04 Dec 2017 10:33:04 GMT): yipcma (Mon, 04 Dec 2017 10:33:04 GMT): harsha (Mon, 04 Dec 2017 10:33:28 GMT): yipcma (Mon, 04 Dec 2017 10:34:01 GMT): harsha (Mon, 04 Dec 2017 10:34:09 GMT): yipcma (Mon, 04 Dec 2017 10:34:10 GMT): yipcma (Mon, 04 Dec 2017 10:34:14 GMT): yipcma (Mon, 04 Dec 2017 10:34:27 GMT): harsha (Mon, 04 Dec 2017 10:34:37 GMT): yipcma (Mon, 04 Dec 2017 10:34:40 GMT): yipcma (Mon, 04 Dec 2017 10:34:45 GMT): yipcma (Mon, 04 Dec 2017 10:35:06 GMT): harsha (Mon, 04 Dec 2017 10:35:14 GMT): harsha (Mon, 04 Dec 2017 10:36:02 GMT): yipcma (Mon, 04 Dec 2017 10:37:13 GMT): yipcma (Mon, 04 Dec 2017 10:37:21 GMT): harsha (Mon, 04 Dec 2017 10:38:55 GMT): yipcma (Mon, 04 Dec 2017 10:46:12 GMT): yipcma (Mon, 04 Dec 2017 10:48:00 GMT): harsha (Mon, 04 Dec 2017 10:50:36 GMT): harsha (Mon, 04 Dec 2017 10:50:36 GMT): yipcma (Mon, 04 Dec 2017 10:52:05 GMT): yipcma (Mon, 04 Dec 2017 10:53:23 GMT): harsha (Mon, 04 Dec 2017 10:54:29 GMT): yipcma (Mon, 04 Dec 2017 10:58:22 GMT): yipcma (Mon, 04 Dec 2017 11:11:20 GMT): harsha (Mon, 04 Dec 2017 11:15:23 GMT): harsha (Mon, 04 Dec 2017 11:15:37 GMT): yipcma (Mon, 04 Dec 2017 11:15:54 GMT): yipcma (Mon, 04 Dec 2017 11:16:05 GMT): harsha (Mon, 04 Dec 2017 11:16:18 GMT): harsha (Mon, 04 Dec 2017 11:16:27 GMT): harsha (Mon, 04 Dec 2017 11:16:34 GMT): yipcma (Mon, 04 Dec 2017 11:17:33 GMT): yipcma (Mon, 04 Dec 2017 11:19:16 GMT): harsha (Mon, 04 Dec 2017 11:19:58 GMT): harsha (Mon, 04 Dec 2017 14:02:05 GMT): RamaKrishnaS (Mon, 04 Dec 2017 16:23:20 GMT): ambatigaan (Mon, 04 Dec 2017 16:28:38 GMT): yipcma (Mon, 04 Dec 2017 16:30:32 GMT): yipcma (Mon, 04 Dec 2017 16:47:19 GMT): yipcma (Mon, 04 Dec 2017 16:47:19 GMT): yipcma (Mon, 04 Dec 2017 18:48:39 GMT): sarifuddin (Mon, 04 Dec 2017 20:53:17 GMT): sarifuddin (Mon, 04 Dec 2017 20:54:44 GMT): sarifuddin (Mon, 04 Dec 2017 20:55:04 GMT): sarifuddin (Mon, 04 Dec 2017 20:55:13 GMT): sarifuddin (Mon, 04 Dec 2017 20:55:20 GMT): sarifuddin (Mon, 04 Dec 2017 20:55:22 GMT): sarifuddin (Mon, 04 Dec 2017 20:55:43 GMT): Luke_Chen (Tue, 05 Dec 2017 01:56:27 GMT): baohua (Tue, 05 Dec 2017 03:26:26 GMT): shreyanadvisory (Tue, 05 Dec 2017 04:45:20 GMT): harsha (Tue, 05 Dec 2017 08:46:31 GMT): harsha (Tue, 05 Dec 2017 08:46:31 GMT): harsha (Tue, 05 Dec 2017 08:46:31 GMT): harsha (Tue, 05 Dec 2017 08:46:31 GMT): harsha (Tue, 05 Dec 2017 19:26:05 GMT): harsha (Wed, 06 Dec 2017 04:43:35 GMT): baohua (Wed, 06 Dec 2017 04:56:14 GMT): baohua (Wed, 06 Dec 2017 05:11:24 GMT): baohua (Wed, 06 Dec 2017 05:15:41 GMT): MohammadObaid (Wed, 06 Dec 2017 06:26:23 GMT): MohammadObaid (Wed, 06 Dec 2017 06:27:10 GMT): baohua (Wed, 06 Dec 2017 07:07:02 GMT): MohammadObaid (Wed, 06 Dec 2017 07:18:49 GMT): yipcma (Wed, 06 Dec 2017 12:40:05 GMT): yipcma (Wed, 06 Dec 2017 12:46:08 GMT): Luke_Chen (Wed, 06 Dec 2017 14:50:53 GMT): yipcma (Wed, 06 Dec 2017 16:12:52 GMT): SomaNayak (Wed, 06 Dec 2017 18:03:08 GMT): maddyhopper (Wed, 06 Dec 2017 20:49:36 GMT): parags (Thu, 07 Dec 2017 10:40:43 GMT): baohua (Fri, 08 Dec 2017 01:50:55 GMT): Soundharya (Fri, 08 Dec 2017 05:27:13 GMT): sarifuddin (Fri, 08 Dec 2017 17:55:43 GMT): sarifuddin (Fri, 08 Dec 2017 17:56:02 GMT): sarifuddin (Fri, 08 Dec 2017 17:56:06 GMT): sarifuddin (Fri, 08 Dec 2017 17:56:16 GMT): sarifuddin (Fri, 08 Dec 2017 17:56:19 GMT): sarifuddin (Fri, 08 Dec 2017 17:56:39 GMT): MohammadObaid (Fri, 08 Dec 2017 18:10:27 GMT): MohammadObaid (Fri, 08 Dec 2017 18:10:27 GMT): MohammadObaid (Fri, 08 Dec 2017 18:10:54 GMT): MohammadObaid (Fri, 08 Dec 2017 18:10:54 GMT): tongli (Fri, 08 Dec 2017 18:37:03 GMT): sarifuddin (Fri, 08 Dec 2017 18:41:38 GMT): sarifuddin (Fri, 08 Dec 2017 18:41:59 GMT): sarifuddin (Fri, 08 Dec 2017 18:42:09 GMT): sarifuddin (Fri, 08 Dec 2017 18:42:50 GMT): tongli (Fri, 08 Dec 2017 18:42:53 GMT): sarifuddin (Fri, 08 Dec 2017 18:42:58 GMT): tongli (Fri, 08 Dec 2017 18:43:26 GMT): sarifuddin (Fri, 08 Dec 2017 18:43:47 GMT): sarifuddin (Fri, 08 Dec 2017 18:44:09 GMT): sarifuddin (Fri, 08 Dec 2017 18:44:22 GMT): sarifuddin (Fri, 08 Dec 2017 18:44:35 GMT): sarifuddin (Fri, 08 Dec 2017 18:44:49 GMT): sarifuddin (Fri, 08 Dec 2017 18:45:00 GMT): tongli (Fri, 08 Dec 2017 18:45:03 GMT): sarifuddin (Fri, 08 Dec 2017 18:45:15 GMT): tongli (Fri, 08 Dec 2017 18:45:17 GMT): tongli (Fri, 08 Dec 2017 18:46:09 GMT): sarifuddin (Fri, 08 Dec 2017 18:46:22 GMT): sarifuddin (Fri, 08 Dec 2017 18:46:37 GMT): tongli (Fri, 08 Dec 2017 18:47:24 GMT): tongli (Fri, 08 Dec 2017 18:48:08 GMT): tongli (Fri, 08 Dec 2017 18:48:28 GMT): tongli (Fri, 08 Dec 2017 18:48:34 GMT): sarifuddin (Fri, 08 Dec 2017 18:48:54 GMT): tongli (Fri, 08 Dec 2017 18:49:24 GMT): sarifuddin (Fri, 08 Dec 2017 18:49:45 GMT): tongli (Fri, 08 Dec 2017 18:49:53 GMT): sarifuddin (Fri, 08 Dec 2017 18:50:04 GMT): tongli (Fri, 08 Dec 2017 18:50:15 GMT): tongli (Fri, 08 Dec 2017 18:50:21 GMT): tongli (Fri, 08 Dec 2017 18:50:39 GMT): sarifuddin (Fri, 08 Dec 2017 18:52:32 GMT): tongli (Fri, 08 Dec 2017 18:52:35 GMT): sarifuddin (Fri, 08 Dec 2017 18:52:44 GMT): sarifuddin (Fri, 08 Dec 2017 18:54:39 GMT): sarifuddin (Fri, 08 Dec 2017 18:54:41 GMT): sarifuddin (Fri, 08 Dec 2017 18:54:47 GMT): sarifuddin (Fri, 08 Dec 2017 18:54:52 GMT): tongli (Fri, 08 Dec 2017 18:54:59 GMT): tongli (Fri, 08 Dec 2017 18:55:12 GMT): sarifuddin (Fri, 08 Dec 2017 18:55:34 GMT): tongli (Fri, 08 Dec 2017 18:55:49 GMT): sarifuddin (Fri, 08 Dec 2017 18:56:31 GMT): sarifuddin (Fri, 08 Dec 2017 18:56:59 GMT): sarifuddin (Fri, 08 Dec 2017 18:57:03 GMT): sarifuddin (Fri, 08 Dec 2017 18:57:20 GMT): sarifuddin (Fri, 08 Dec 2017 18:57:35 GMT): tongli (Fri, 08 Dec 2017 18:58:04 GMT): tongli (Fri, 08 Dec 2017 18:58:40 GMT): sarifuddin (Fri, 08 Dec 2017 19:00:16 GMT): tongli (Fri, 08 Dec 2017 19:00:39 GMT): MohammadObaid (Fri, 08 Dec 2017 19:22:24 GMT): tongli (Fri, 08 Dec 2017 19:22:45 GMT): MohammadObaid (Fri, 08 Dec 2017 19:24:58 GMT): tongli (Fri, 08 Dec 2017 19:40:41 GMT): tongli (Fri, 08 Dec 2017 19:41:10 GMT): tongli (Fri, 08 Dec 2017 19:41:35 GMT): tongli (Fri, 08 Dec 2017 19:48:26 GMT): sarifuddin (Fri, 08 Dec 2017 19:58:54 GMT): sarifuddin (Fri, 08 Dec 2017 19:59:02 GMT): sarifuddin (Fri, 08 Dec 2017 19:59:15 GMT): sarifuddin (Fri, 08 Dec 2017 20:02:00 GMT): sarifuddin (Fri, 08 Dec 2017 20:02:02 GMT): sarifuddin (Fri, 08 Dec 2017 20:02:16 GMT): sarifuddin (Fri, 08 Dec 2017 20:02:20 GMT): sarifuddin (Fri, 08 Dec 2017 20:02:36 GMT): sarifuddin (Fri, 08 Dec 2017 20:02:38 GMT): sarifuddin (Fri, 08 Dec 2017 20:04:14 GMT): sarifuddin (Fri, 08 Dec 2017 20:04:15 GMT): sarifuddin (Fri, 08 Dec 2017 20:22:07 GMT): sarifuddin (Fri, 08 Dec 2017 20:22:09 GMT): sarifuddin (Fri, 08 Dec 2017 20:22:34 GMT): MohammadObaid (Fri, 08 Dec 2017 20:41:51 GMT): mogamboizer (Fri, 08 Dec 2017 21:05:16 GMT): kvskmech (Sat, 09 Dec 2017 12:57:55 GMT): tongli (Sat, 09 Dec 2017 20:36:05 GMT): tongli (Sat, 09 Dec 2017 20:36:26 GMT): tongli (Sat, 09 Dec 2017 20:37:24 GMT): tongli (Sat, 09 Dec 2017 20:37:55 GMT): baohua (Sun, 10 Dec 2017 11:13:37 GMT): hightall (Sun, 10 Dec 2017 14:53:50 GMT): sarifuddin (Sun, 10 Dec 2017 18:04:40 GMT): sarifuddin (Sun, 10 Dec 2017 18:06:41 GMT): sarifuddin (Sun, 10 Dec 2017 18:07:49 GMT): baohua (Mon, 11 Dec 2017 01:40:52 GMT): baohua (Mon, 11 Dec 2017 01:40:57 GMT): baohua (Mon, 11 Dec 2017 01:41:03 GMT): tongli (Mon, 11 Dec 2017 14:40:08 GMT): tongli (Mon, 11 Dec 2017 14:40:32 GMT): tongli (Mon, 11 Dec 2017 14:41:14 GMT): tongli (Mon, 11 Dec 2017 14:43:37 GMT): tongli (Mon, 11 Dec 2017 14:43:57 GMT): tongli (Mon, 11 Dec 2017 14:44:07 GMT): baohua (Mon, 11 Dec 2017 14:45:27 GMT): tongli (Mon, 11 Dec 2017 14:47:28 GMT): tongli (Mon, 11 Dec 2017 14:47:59 GMT): tongli (Mon, 11 Dec 2017 14:50:04 GMT): tongli (Mon, 11 Dec 2017 14:50:40 GMT): tongli (Mon, 11 Dec 2017 14:51:17 GMT): tongli (Mon, 11 Dec 2017 14:51:21 GMT): tongli (Mon, 11 Dec 2017 14:51:33 GMT): tongli (Mon, 11 Dec 2017 14:51:37 GMT): tongli (Mon, 11 Dec 2017 14:52:39 GMT): tongli (Mon, 11 Dec 2017 14:53:10 GMT): tongli (Mon, 11 Dec 2017 14:53:19 GMT): tongli (Mon, 11 Dec 2017 14:53:22 GMT): harsha (Mon, 11 Dec 2017 18:34:16 GMT): MohammadObaid (Mon, 11 Dec 2017 19:05:33 GMT): hendry19901990 (Mon, 11 Dec 2017 20:10:25 GMT): tongli (Mon, 11 Dec 2017 20:44:39 GMT): tongli (Mon, 11 Dec 2017 20:44:52 GMT): tongli (Mon, 11 Dec 2017 20:45:02 GMT): tongli (Mon, 11 Dec 2017 20:45:14 GMT): tongli (Mon, 11 Dec 2017 20:45:29 GMT): zonked 1 (Mon, 11 Dec 2017 21:55:30 GMT): zonked 1 (Mon, 11 Dec 2017 22:29:13 GMT): zonked 1 (Mon, 11 Dec 2017 22:29:29 GMT): zonked 1 (Mon, 11 Dec 2017 22:54:12 GMT): zonked 1 (Mon, 11 Dec 2017 22:54:15 GMT): harsha (Tue, 12 Dec 2017 02:09:06 GMT): zonked 1 (Tue, 12 Dec 2017 02:47:59 GMT): zonked 1 (Tue, 12 Dec 2017 02:48:46 GMT): zonked 1 (Tue, 12 Dec 2017 02:49:24 GMT): baohua (Tue, 12 Dec 2017 02:50:30 GMT): KSLee (Tue, 12 Dec 2017 05:23:45 GMT): KSLee (Tue, 12 Dec 2017 05:27:05 GMT): KSLee (Tue, 12 Dec 2017 05:38:55 GMT): KSLee (Tue, 12 Dec 2017 05:38:55 GMT): KSLee (Tue, 12 Dec 2017 05:38:55 GMT): KSLee (Tue, 12 Dec 2017 05:38:55 GMT): KSLee (Tue, 12 Dec 2017 05:38:55 GMT): harsha (Tue, 12 Dec 2017 07:13:51 GMT): harsha (Tue, 12 Dec 2017 07:13:51 GMT): KSLee (Tue, 12 Dec 2017 07:16:00 GMT): harsha (Tue, 12 Dec 2017 07:17:51 GMT): harsha (Tue, 12 Dec 2017 07:17:51 GMT): harsha (Tue, 12 Dec 2017 07:17:51 GMT): harsha (Tue, 12 Dec 2017 07:18:24 GMT): baohua (Tue, 12 Dec 2017 07:32:53 GMT): baohua (Tue, 12 Dec 2017 07:33:14 GMT): KSLee (Tue, 12 Dec 2017 08:07:09 GMT): baohua (Tue, 12 Dec 2017 08:23:31 GMT): baohua (Tue, 12 Dec 2017 08:23:38 GMT): baohua (Tue, 12 Dec 2017 08:23:55 GMT): baohua (Tue, 12 Dec 2017 08:24:09 GMT): baohua (Tue, 12 Dec 2017 08:25:42 GMT): baohua (Tue, 12 Dec 2017 08:25:43 GMT): baohua (Tue, 12 Dec 2017 08:25:43 GMT): baohua (Tue, 12 Dec 2017 08:25:46 GMT): harsha (Tue, 12 Dec 2017 08:33:01 GMT): baohua (Tue, 12 Dec 2017 08:34:48 GMT): KSLee (Tue, 12 Dec 2017 08:54:39 GMT): KSLee (Tue, 12 Dec 2017 08:54:39 GMT): KSLee (Tue, 12 Dec 2017 08:54:39 GMT): KSLee (Tue, 12 Dec 2017 08:54:39 GMT): KSLee (Tue, 12 Dec 2017 08:54:39 GMT): baohua (Tue, 12 Dec 2017 08:58:06 GMT): KSLee (Tue, 12 Dec 2017 09:30:59 GMT): nmpacheco@gmail.com (Tue, 12 Dec 2017 11:08:15 GMT): harsha (Tue, 12 Dec 2017 11:09:24 GMT): harsha (Tue, 12 Dec 2017 11:09:24 GMT): tongli (Tue, 12 Dec 2017 16:20:19 GMT): baohua (Wed, 13 Dec 2017 00:41:52 GMT): tongli (Wed, 13 Dec 2017 01:10:14 GMT): tongli (Wed, 13 Dec 2017 01:10:16 GMT): baohua (Wed, 13 Dec 2017 01:23:49 GMT): baohua (Wed, 13 Dec 2017 01:24:58 GMT): tongli (Wed, 13 Dec 2017 01:25:50 GMT): tongli (Wed, 13 Dec 2017 01:26:56 GMT): tongli (Wed, 13 Dec 2017 01:27:04 GMT): KSLee (Wed, 13 Dec 2017 01:29:54 GMT): baohua (Wed, 13 Dec 2017 02:39:16 GMT): harsha (Wed, 13 Dec 2017 04:45:31 GMT): mvaibhavshah (Wed, 13 Dec 2017 05:15:28 GMT): mvaibhavshah (Wed, 13 Dec 2017 05:16:46 GMT): harsha (Wed, 13 Dec 2017 05:37:10 GMT): mvaibhavshah (Wed, 13 Dec 2017 05:38:10 GMT): harsha (Wed, 13 Dec 2017 05:39:11 GMT): KSLee (Wed, 13 Dec 2017 10:33:19 GMT): harsha (Wed, 13 Dec 2017 10:36:04 GMT): KSLee (Wed, 13 Dec 2017 10:44:37 GMT): harsha (Wed, 13 Dec 2017 10:46:06 GMT): Sarvesh (Wed, 13 Dec 2017 13:39:17 GMT): Sarvesh (Wed, 13 Dec 2017 13:39:51 GMT): rameshthoomu (Wed, 13 Dec 2017 14:10:23 GMT): hightall (Wed, 13 Dec 2017 14:35:01 GMT): baohua (Wed, 13 Dec 2017 14:51:06 GMT): tongli (Wed, 13 Dec 2017 14:58:44 GMT): tongli (Wed, 13 Dec 2017 14:58:54 GMT): baohua (Wed, 13 Dec 2017 15:01:47 GMT): baohua (Wed, 13 Dec 2017 15:01:54 GMT): tongli (Wed, 13 Dec 2017 15:30:40 GMT): cweiers (Wed, 13 Dec 2017 17:00:56 GMT): cweiers (Wed, 13 Dec 2017 18:39:57 GMT): cweiers (Wed, 13 Dec 2017 18:41:43 GMT): cweiers (Wed, 13 Dec 2017 18:43:00 GMT): cweiers (Wed, 13 Dec 2017 18:45:43 GMT): cweiers (Wed, 13 Dec 2017 18:46:13 GMT): gen_el (Wed, 13 Dec 2017 19:40:26 GMT): ysim (Wed, 13 Dec 2017 20:27:10 GMT): KSLee (Thu, 14 Dec 2017 04:14:39 GMT): baohua (Thu, 14 Dec 2017 04:28:40 GMT): maris.c (Thu, 14 Dec 2017 06:16:04 GMT): maris.c (Thu, 14 Dec 2017 06:19:47 GMT): baohua (Thu, 14 Dec 2017 06:48:04 GMT): maris.c (Thu, 14 Dec 2017 07:27:07 GMT): yingmsky (Thu, 14 Dec 2017 07:50:13 GMT): Luke_Chen (Thu, 14 Dec 2017 08:50:15 GMT): baohua (Thu, 14 Dec 2017 08:53:07 GMT): baohua (Thu, 14 Dec 2017 08:53:11 GMT): baohua (Thu, 14 Dec 2017 08:56:02 GMT): baohua (Thu, 14 Dec 2017 08:56:15 GMT): Luke_Chen (Thu, 14 Dec 2017 09:01:56 GMT): baohua (Thu, 14 Dec 2017 09:02:24 GMT): baohua (Thu, 14 Dec 2017 09:02:46 GMT): Luke_Chen (Thu, 14 Dec 2017 09:03:49 GMT): Luke_Chen (Thu, 14 Dec 2017 09:04:27 GMT): Luke_Chen (Thu, 14 Dec 2017 09:04:27 GMT): baohua (Thu, 14 Dec 2017 09:05:37 GMT): Luke_Chen (Thu, 14 Dec 2017 09:06:26 GMT): baohua (Thu, 14 Dec 2017 09:06:56 GMT): maris.c (Thu, 14 Dec 2017 09:07:41 GMT): maris.c (Thu, 14 Dec 2017 09:08:04 GMT): baohua (Thu, 14 Dec 2017 09:08:18 GMT): harsha (Thu, 14 Dec 2017 09:11:10 GMT): maris.c (Thu, 14 Dec 2017 09:11:27 GMT): maris.c (Thu, 14 Dec 2017 09:29:27 GMT): baohua (Thu, 14 Dec 2017 09:33:02 GMT): maris.c (Thu, 14 Dec 2017 09:35:07 GMT): baohua (Thu, 14 Dec 2017 09:44:54 GMT): maris.c (Thu, 14 Dec 2017 09:52:29 GMT): luminance (Thu, 14 Dec 2017 10:13:21 GMT): maris.c (Thu, 14 Dec 2017 10:23:40 GMT): maris.c (Thu, 14 Dec 2017 10:24:04 GMT): luminance (Thu, 14 Dec 2017 10:30:32 GMT): luminance (Thu, 14 Dec 2017 10:53:34 GMT): luminance (Thu, 14 Dec 2017 10:53:34 GMT): luminance (Thu, 14 Dec 2017 11:32:46 GMT): cweiers (Thu, 14 Dec 2017 12:44:35 GMT): cweiers (Thu, 14 Dec 2017 12:45:40 GMT): harsha (Thu, 14 Dec 2017 13:45:46 GMT): harsha (Thu, 14 Dec 2017 13:51:29 GMT): harsha (Thu, 14 Dec 2017 13:51:29 GMT): luminance (Thu, 14 Dec 2017 13:51:42 GMT): baohua (Thu, 14 Dec 2017 14:24:13 GMT): tonym1 (Thu, 14 Dec 2017 14:27:12 GMT): harsha (Thu, 14 Dec 2017 14:28:21 GMT): maris.c (Thu, 14 Dec 2017 14:36:00 GMT): maris.c (Thu, 14 Dec 2017 14:37:18 GMT): maris.c (Thu, 14 Dec 2017 14:38:02 GMT): maris.c (Thu, 14 Dec 2017 14:38:02 GMT): harsha (Thu, 14 Dec 2017 14:38:36 GMT): maris.c (Thu, 14 Dec 2017 14:38:46 GMT): harsha (Thu, 14 Dec 2017 14:39:49 GMT): maris.c (Thu, 14 Dec 2017 14:40:23 GMT): maris.c (Thu, 14 Dec 2017 14:40:24 GMT): maris.c (Thu, 14 Dec 2017 15:12:14 GMT): maris.c (Thu, 14 Dec 2017 15:14:08 GMT): luminance (Thu, 14 Dec 2017 15:35:05 GMT): baohua (Thu, 14 Dec 2017 15:51:03 GMT): baohua (Thu, 14 Dec 2017 15:58:49 GMT): baohua (Thu, 14 Dec 2017 15:58:56 GMT): baohua (Thu, 14 Dec 2017 16:06:50 GMT): Luke_Chen (Thu, 14 Dec 2017 17:08:09 GMT): Luke_Chen (Thu, 14 Dec 2017 17:08:09 GMT): Luke_Chen (Thu, 14 Dec 2017 17:08:09 GMT): Luke_Chen (Thu, 14 Dec 2017 17:08:33 GMT): Luke_Chen (Thu, 14 Dec 2017 17:09:05 GMT): Luke_Chen (Thu, 14 Dec 2017 17:10:05 GMT): Luke_Chen (Thu, 14 Dec 2017 17:12:00 GMT): Luke_Chen (Thu, 14 Dec 2017 17:12:32 GMT): Luke_Chen (Thu, 14 Dec 2017 17:16:37 GMT): Luke_Chen (Thu, 14 Dec 2017 17:22:38 GMT): baohua (Fri, 15 Dec 2017 04:07:24 GMT): maris.c (Fri, 15 Dec 2017 07:56:54 GMT): harsha (Fri, 15 Dec 2017 07:59:33 GMT): maris.c (Fri, 15 Dec 2017 09:02:54 GMT): baohua (Fri, 15 Dec 2017 09:19:24 GMT): maris.c (Fri, 15 Dec 2017 09:29:00 GMT): maris.c (Fri, 15 Dec 2017 09:31:53 GMT): Luke_Chen (Fri, 15 Dec 2017 11:29:33 GMT): Luke_Chen (Fri, 15 Dec 2017 11:29:40 GMT): Luke_Chen (Fri, 15 Dec 2017 11:30:17 GMT): Luke_Chen (Fri, 15 Dec 2017 11:30:31 GMT): harsha (Fri, 15 Dec 2017 12:00:17 GMT): maris.c (Fri, 15 Dec 2017 12:02:58 GMT): maris.c (Fri, 15 Dec 2017 12:05:00 GMT): Layne (Fri, 15 Dec 2017 12:19:09 GMT): harsha (Fri, 15 Dec 2017 12:28:48 GMT): Layne (Fri, 15 Dec 2017 12:39:56 GMT): Layne (Fri, 15 Dec 2017 12:42:48 GMT): maris.c (Fri, 15 Dec 2017 12:46:41 GMT): Luke_Chen (Fri, 15 Dec 2017 14:55:40 GMT): Luke_Chen (Fri, 15 Dec 2017 14:55:41 GMT): baohua (Sat, 16 Dec 2017 07:41:49 GMT): Luke_Chen (Sat, 16 Dec 2017 15:51:04 GMT): Luke_Chen (Sat, 16 Dec 2017 15:51:04 GMT): hightall (Sun, 17 Dec 2017 08:13:10 GMT): hightall (Sun, 17 Dec 2017 10:05:33 GMT): baohua (Sun, 17 Dec 2017 10:22:39 GMT): baohua (Sun, 17 Dec 2017 10:23:26 GMT): harsha (Sun, 17 Dec 2017 10:25:09 GMT): harsha (Sun, 17 Dec 2017 10:25:59 GMT): rhansharief (Sun, 17 Dec 2017 23:19:07 GMT): rhansharief (Sun, 17 Dec 2017 23:25:14 GMT): hightall (Mon, 18 Dec 2017 01:56:37 GMT): baohua (Mon, 18 Dec 2017 03:08:20 GMT): rhansharief (Mon, 18 Dec 2017 03:10:09 GMT): harsha (Mon, 18 Dec 2017 05:34:30 GMT): harsha (Mon, 18 Dec 2017 05:34:30 GMT): harsha (Mon, 18 Dec 2017 05:34:30 GMT): harsha (Mon, 18 Dec 2017 05:34:30 GMT): harsha (Mon, 18 Dec 2017 05:34:43 GMT): harsha (Mon, 18 Dec 2017 05:34:43 GMT): harsha (Mon, 18 Dec 2017 05:34:43 GMT): harsha (Tue, 19 Dec 2017 04:24:25 GMT): harsha (Tue, 19 Dec 2017 04:24:25 GMT): harsha (Tue, 19 Dec 2017 04:24:25 GMT): tallharish (Wed, 20 Dec 2017 01:35:47 GMT): PabloBascunana (Wed, 20 Dec 2017 08:49:46 GMT): Varun2887 (Wed, 20 Dec 2017 09:43:26 GMT): Varun2887 (Wed, 20 Dec 2017 09:43:33 GMT): tongli (Thu, 21 Dec 2017 00:48:37 GMT): harsha (Thu, 21 Dec 2017 05:02:02 GMT): yoyokeen (Fri, 22 Dec 2017 05:31:57 GMT): yoyokeen (Fri, 22 Dec 2017 05:46:16 GMT): yoyokeen (Fri, 22 Dec 2017 06:29:16 GMT): harsha (Fri, 22 Dec 2017 07:54:43 GMT): smallant (Fri, 22 Dec 2017 11:19:37 GMT): hendry19901990 (Fri, 22 Dec 2017 18:03:28 GMT): hendry19901990 (Fri, 22 Dec 2017 18:03:28 GMT): hendry19901990 (Fri, 22 Dec 2017 18:04:58 GMT): harsha (Fri, 22 Dec 2017 19:00:36 GMT): harsha (Fri, 22 Dec 2017 19:01:00 GMT): hendry19901990 (Fri, 22 Dec 2017 19:25:04 GMT): hendry19901990 (Fri, 22 Dec 2017 20:13:19 GMT): hendry19901990 (Fri, 22 Dec 2017 20:13:22 GMT): hendry19901990 (Fri, 22 Dec 2017 20:13:23 GMT): harsha (Sat, 23 Dec 2017 03:42:37 GMT): yoyokeen (Sat, 23 Dec 2017 09:04:37 GMT): yoyokeen (Sat, 23 Dec 2017 09:17:45 GMT): yoyokeen (Sat, 23 Dec 2017 09:23:00 GMT): harsha (Sat, 23 Dec 2017 11:03:52 GMT): harsha (Sat, 23 Dec 2017 11:04:24 GMT): yoyokeen (Sat, 23 Dec 2017 12:40:29 GMT): yoyokeen (Sat, 23 Dec 2017 12:40:48 GMT): yoyokeen (Sat, 23 Dec 2017 12:41:21 GMT): yoyokeen (Sat, 23 Dec 2017 12:43:14 GMT): harsha (Sat, 23 Dec 2017 19:04:00 GMT): harsha (Sat, 23 Dec 2017 19:04:00 GMT): yoyokeen (Sun, 24 Dec 2017 09:30:01 GMT): yoyokeen (Sun, 24 Dec 2017 09:30:10 GMT): yoyokeen (Sun, 24 Dec 2017 09:30:37 GMT): yoyokeen (Sun, 24 Dec 2017 09:30:54 GMT): baohua (Mon, 25 Dec 2017 01:16:20 GMT): harsha (Mon, 25 Dec 2017 15:51:42 GMT): harsha (Mon, 25 Dec 2017 15:51:42 GMT): MohammadObaid (Mon, 25 Dec 2017 17:41:34 GMT): rohanagarwal (Mon, 25 Dec 2017 21:31:13 GMT): baohua (Tue, 26 Dec 2017 05:18:03 GMT): hendry19901990 (Tue, 26 Dec 2017 14:28:16 GMT): hendry19901990 (Tue, 26 Dec 2017 14:28:20 GMT): hendry19901990 (Tue, 26 Dec 2017 14:28:34 GMT): hightall (Tue, 26 Dec 2017 14:47:36 GMT): davux (Tue, 26 Dec 2017 19:38:29 GMT): baohua (Wed, 27 Dec 2017 01:26:42 GMT): baohua (Wed, 27 Dec 2017 01:31:56 GMT): DannyWong (Wed, 27 Dec 2017 02:17:14 GMT): sb2407 (Wed, 27 Dec 2017 12:36:48 GMT): nasserfci (Wed, 27 Dec 2017 14:30:20 GMT): nasserfci (Wed, 27 Dec 2017 14:30:30 GMT): nasserfci (Wed, 27 Dec 2017 14:31:02 GMT): nasserfci (Wed, 27 Dec 2017 14:31:36 GMT): nasserfci (Wed, 27 Dec 2017 14:31:46 GMT): nasserfci (Wed, 27 Dec 2017 14:31:46 GMT): nasserfci (Wed, 27 Dec 2017 14:31:53 GMT): nasserfci (Wed, 27 Dec 2017 14:32:06 GMT): Luke_Chen (Wed, 27 Dec 2017 15:59:58 GMT): MohammadObaid (Thu, 28 Dec 2017 08:36:07 GMT): nasserfci (Fri, 29 Dec 2017 20:56:07 GMT): elias_p (Sat, 30 Dec 2017 10:28:22 GMT): nasserfci (Sat, 30 Dec 2017 13:20:56 GMT): nasserfci (Sat, 30 Dec 2017 13:20:56 GMT): nasserfci (Sat, 30 Dec 2017 13:20:56 GMT): marc0o (Sat, 30 Dec 2017 19:26:35 GMT): MohammadObaid (Sun, 31 Dec 2017 17:02:31 GMT): nazeem92 (Mon, 01 Jan 2018 10:39:00 GMT): nazeem92 (Mon, 01 Jan 2018 10:42:03 GMT): baohua (Tue, 02 Jan 2018 00:50:11 GMT): saeedi (Tue, 02 Jan 2018 06:21:08 GMT): nazeem92 (Tue, 02 Jan 2018 06:24:29 GMT): luminance (Tue, 02 Jan 2018 10:06:50 GMT): tongli (Tue, 02 Jan 2018 15:12:26 GMT): nshah 1 (Tue, 02 Jan 2018 16:02:47 GMT): baohua (Wed, 03 Jan 2018 00:39:57 GMT): asaningmaxchain (Wed, 03 Jan 2018 05:44:24 GMT): asaningmaxchain (Wed, 03 Jan 2018 05:47:51 GMT): asaningmaxchain (Wed, 03 Jan 2018 05:48:04 GMT): asaningmaxchain (Wed, 03 Jan 2018 05:48:04 GMT): baohua (Wed, 03 Jan 2018 08:46:56 GMT): nasserfci (Wed, 03 Jan 2018 14:57:07 GMT): asaningmaxchain (Wed, 03 Jan 2018 16:29:45 GMT): baohua (Thu, 04 Jan 2018 04:56:20 GMT): harsha (Thu, 04 Jan 2018 05:35:47 GMT): harsha (Thu, 04 Jan 2018 05:35:47 GMT): harsha (Thu, 04 Jan 2018 05:35:47 GMT): baohua (Thu, 04 Jan 2018 12:59:12 GMT): baohua (Thu, 04 Jan 2018 12:59:12 GMT): harsha (Thu, 04 Jan 2018 14:53:58 GMT): baohua (Fri, 05 Jan 2018 02:18:50 GMT): hightall (Sat, 06 Jan 2018 09:03:13 GMT): baohua (Sun, 07 Jan 2018 03:53:45 GMT): Rishabh-Sharma (Sun, 07 Jan 2018 06:03:52 GMT): baohua (Tue, 09 Jan 2018 03:23:16 GMT): DarshanBc (Tue, 09 Jan 2018 04:38:51 GMT): DarshanBc (Tue, 09 Jan 2018 04:45:18 GMT): AnandVijay (Tue, 09 Jan 2018 10:45:16 GMT): harsha (Tue, 09 Jan 2018 15:03:02 GMT): baohua (Tue, 09 Jan 2018 16:17:37 GMT): PetrVlasekCA (Tue, 09 Jan 2018 16:35:11 GMT): baohua (Wed, 10 Jan 2018 00:27:54 GMT): username343 (Wed, 10 Jan 2018 06:24:45 GMT): username343 (Wed, 10 Jan 2018 06:24:53 GMT): baohua (Wed, 10 Jan 2018 07:26:14 GMT): baohua (Wed, 10 Jan 2018 07:31:03 GMT): asaningmaxchain (Wed, 10 Jan 2018 12:23:58 GMT): asaningmaxchain (Wed, 10 Jan 2018 12:24:26 GMT): DarshanBc (Wed, 10 Jan 2018 12:48:50 GMT): harsha (Wed, 10 Jan 2018 13:15:28 GMT): DarshanBc (Wed, 10 Jan 2018 13:18:05 GMT): DarshanBc (Wed, 10 Jan 2018 13:18:05 GMT): harsha (Wed, 10 Jan 2018 13:55:11 GMT): harsha (Wed, 10 Jan 2018 13:55:14 GMT): DarshanBc (Wed, 10 Jan 2018 13:57:44 GMT): harsha (Wed, 10 Jan 2018 13:58:44 GMT): DarshanBc (Wed, 10 Jan 2018 14:00:03 GMT): asaningmaxchain (Wed, 10 Jan 2018 14:03:58 GMT): asaningmaxchain (Wed, 10 Jan 2018 14:04:05 GMT): harsha (Wed, 10 Jan 2018 14:04:06 GMT): harsha (Wed, 10 Jan 2018 14:06:01 GMT): asaningmaxchain (Wed, 10 Jan 2018 14:06:16 GMT): harsha (Wed, 10 Jan 2018 14:06:41 GMT): asaningmaxchain (Wed, 10 Jan 2018 14:08:44 GMT): tongli (Wed, 10 Jan 2018 14:18:02 GMT): tongli (Wed, 10 Jan 2018 14:18:11 GMT): tongli (Wed, 10 Jan 2018 14:18:32 GMT): tongli (Wed, 10 Jan 2018 14:18:48 GMT): asaningmaxchain (Wed, 10 Jan 2018 14:20:52 GMT): tongli (Wed, 10 Jan 2018 14:22:37 GMT): asaningmaxchain (Wed, 10 Jan 2018 14:23:29 GMT): SimonOberzan (Wed, 10 Jan 2018 14:59:25 GMT): SimonOberzan (Wed, 10 Jan 2018 14:59:25 GMT): baohua (Wed, 10 Jan 2018 16:02:13 GMT): baohua (Wed, 10 Jan 2018 16:02:49 GMT): harsha (Wed, 10 Jan 2018 16:06:40 GMT): harsha (Wed, 10 Jan 2018 16:06:40 GMT): harsha (Wed, 10 Jan 2018 16:07:35 GMT): harsha (Wed, 10 Jan 2018 16:08:01 GMT): asaningmaxchain (Wed, 10 Jan 2018 16:17:15 GMT): harsha (Wed, 10 Jan 2018 16:22:49 GMT): asaningmaxchain (Wed, 10 Jan 2018 16:23:40 GMT): asaningmaxchain (Wed, 10 Jan 2018 16:24:32 GMT): asaningmaxchain (Wed, 10 Jan 2018 16:24:40 GMT): baohua (Wed, 10 Jan 2018 18:01:28 GMT): baohua (Wed, 10 Jan 2018 18:02:22 GMT): harsha (Wed, 10 Jan 2018 18:02:25 GMT): harsha (Wed, 10 Jan 2018 18:02:25 GMT): baohua (Wed, 10 Jan 2018 18:03:32 GMT): baohua (Wed, 10 Jan 2018 18:03:48 GMT): baohua (Wed, 10 Jan 2018 18:03:48 GMT): baohua (Wed, 10 Jan 2018 18:04:49 GMT): harsha (Wed, 10 Jan 2018 18:05:55 GMT): harsha (Wed, 10 Jan 2018 18:05:55 GMT): baohua (Wed, 10 Jan 2018 18:06:20 GMT): harsha (Wed, 10 Jan 2018 18:06:30 GMT): baohua (Wed, 10 Jan 2018 18:06:39 GMT): baohua (Wed, 10 Jan 2018 18:07:03 GMT): baohua (Wed, 10 Jan 2018 18:07:18 GMT): rameshthoomu (Wed, 10 Jan 2018 18:08:34 GMT): harsha (Wed, 10 Jan 2018 18:09:58 GMT): harsha (Wed, 10 Jan 2018 18:11:24 GMT): baohua (Wed, 10 Jan 2018 18:11:48 GMT): baohua (Wed, 10 Jan 2018 18:12:53 GMT): baohua (Wed, 10 Jan 2018 19:48:32 GMT): baohua (Wed, 10 Jan 2018 19:48:47 GMT): rameshthoomu (Wed, 10 Jan 2018 21:44:34 GMT): harsha (Thu, 11 Jan 2018 04:42:50 GMT): baohua (Thu, 11 Jan 2018 05:07:07 GMT): rhansharief (Thu, 11 Jan 2018 06:52:11 GMT): Lavanya5896 (Thu, 11 Jan 2018 09:43:17 GMT): fabcan (Thu, 11 Jan 2018 11:20:20 GMT): fabcan (Thu, 11 Jan 2018 11:22:58 GMT): fabcan (Thu, 11 Jan 2018 11:22:58 GMT): Lavanya5896 (Thu, 11 Jan 2018 11:48:23 GMT): baohua (Thu, 11 Jan 2018 14:30:47 GMT): baohua (Thu, 11 Jan 2018 14:31:11 GMT): baohua (Thu, 11 Jan 2018 14:31:51 GMT): baohua (Thu, 11 Jan 2018 14:54:39 GMT): fabcan (Thu, 11 Jan 2018 14:54:52 GMT): baohua (Thu, 11 Jan 2018 14:55:37 GMT): baohua (Thu, 11 Jan 2018 14:55:56 GMT): fabcan (Thu, 11 Jan 2018 15:19:17 GMT): fabcan (Thu, 11 Jan 2018 15:19:17 GMT): fabcan (Thu, 11 Jan 2018 15:19:17 GMT): fabcan (Thu, 11 Jan 2018 15:19:17 GMT): baohua (Thu, 11 Jan 2018 15:23:57 GMT): baohua (Thu, 11 Jan 2018 15:24:11 GMT): baohua (Thu, 11 Jan 2018 15:24:33 GMT): fabcan (Thu, 11 Jan 2018 15:25:46 GMT): baohua (Thu, 11 Jan 2018 15:26:23 GMT): fabcan (Thu, 11 Jan 2018 15:36:01 GMT): YinxiaZhang (Thu, 11 Jan 2018 16:02:23 GMT): harsha (Thu, 11 Jan 2018 16:29:17 GMT): harsha (Thu, 11 Jan 2018 16:29:17 GMT): rameshthoomu (Thu, 11 Jan 2018 16:39:53 GMT): rameshthoomu (Thu, 11 Jan 2018 16:43:37 GMT): nkl199 (Thu, 11 Jan 2018 17:14:26 GMT): baohua (Thu, 11 Jan 2018 17:17:38 GMT): baohua (Thu, 11 Jan 2018 22:55:28 GMT): Sufbin (Thu, 11 Jan 2018 22:55:28 GMT): baohua (Thu, 11 Jan 2018 22:57:14 GMT): qqbxclboy (Fri, 12 Jan 2018 03:29:54 GMT): qqbxclboy (Fri, 12 Jan 2018 03:29:59 GMT): qqbxclboy (Fri, 12 Jan 2018 03:30:36 GMT): qqbxclboy (Fri, 12 Jan 2018 03:31:52 GMT): qqbxclboy (Fri, 12 Jan 2018 03:32:11 GMT): qqbxclboy (Fri, 12 Jan 2018 03:34:16 GMT): Luke_Chen (Fri, 12 Jan 2018 03:34:43 GMT): DarshanBc (Fri, 12 Jan 2018 06:21:03 GMT): DarshanBc (Fri, 12 Jan 2018 06:21:51 GMT): DarshanBc (Fri, 12 Jan 2018 06:21:51 GMT): rhansharief (Fri, 12 Jan 2018 09:01:53 GMT): DarshanBc (Fri, 12 Jan 2018 11:26:56 GMT): DarshanBc (Fri, 12 Jan 2018 11:26:56 GMT): DarshanBc (Fri, 12 Jan 2018 11:28:55 GMT): tongli (Fri, 12 Jan 2018 13:02:22 GMT): tongli (Fri, 12 Jan 2018 13:47:48 GMT): tongli (Fri, 12 Jan 2018 13:49:03 GMT): baohua (Fri, 12 Jan 2018 19:40:25 GMT): baohua (Sat, 13 Jan 2018 15:11:10 GMT): rbole (Sun, 14 Jan 2018 18:27:08 GMT): baohua (Mon, 15 Jan 2018 01:31:42 GMT): harsha (Mon, 15 Jan 2018 09:17:37 GMT): harsha (Mon, 15 Jan 2018 09:17:37 GMT): yash-ibm (Mon, 15 Jan 2018 12:04:35 GMT): tasmiya_n (Mon, 15 Jan 2018 12:06:11 GMT): rhansharief (Tue, 16 Jan 2018 10:25:01 GMT): hightall (Tue, 16 Jan 2018 14:17:30 GMT): SimonOberzan (Tue, 16 Jan 2018 16:25:30 GMT): SimonOberzan (Tue, 16 Jan 2018 16:25:30 GMT): ascatox (Tue, 16 Jan 2018 17:49:07 GMT): ascatox (Tue, 16 Jan 2018 17:49:07 GMT): ascatox (Tue, 16 Jan 2018 17:49:07 GMT): ascatox (Tue, 16 Jan 2018 17:49:07 GMT): tongli (Tue, 16 Jan 2018 18:24:15 GMT): Amir (Tue, 16 Jan 2018 20:17:16 GMT): SimonOberzan (Tue, 16 Jan 2018 20:19:10 GMT): tongli (Tue, 16 Jan 2018 21:08:06 GMT): baohua (Wed, 17 Jan 2018 00:44:53 GMT): baohua (Wed, 17 Jan 2018 01:14:38 GMT): baohua (Wed, 17 Jan 2018 01:58:05 GMT): harsha (Wed, 17 Jan 2018 02:26:57 GMT): harsha (Wed, 17 Jan 2018 02:29:48 GMT): baohua (Wed, 17 Jan 2018 02:56:41 GMT): DarshanBc (Wed, 17 Jan 2018 04:40:59 GMT): baohua (Wed, 17 Jan 2018 05:55:28 GMT): baohua (Wed, 17 Jan 2018 06:01:13 GMT): baohua (Wed, 17 Jan 2018 06:42:28 GMT): ascatox (Wed, 17 Jan 2018 06:59:16 GMT): ascatox (Wed, 17 Jan 2018 06:59:16 GMT): ascatox (Wed, 17 Jan 2018 07:01:52 GMT): ascatox (Wed, 17 Jan 2018 08:16:36 GMT): baohua (Wed, 17 Jan 2018 08:17:38 GMT): baohua (Wed, 17 Jan 2018 08:17:59 GMT): ascatox (Wed, 17 Jan 2018 08:18:42 GMT): ascatox (Wed, 17 Jan 2018 08:19:45 GMT): ascatox (Wed, 17 Jan 2018 08:19:45 GMT): baohua (Wed, 17 Jan 2018 08:40:48 GMT): baohua (Wed, 17 Jan 2018 08:41:04 GMT): DarshanBc (Wed, 17 Jan 2018 14:36:38 GMT): baohua (Wed, 17 Jan 2018 14:46:52 GMT): AshishMishra 1 (Wed, 17 Jan 2018 16:53:14 GMT): tongli (Wed, 17 Jan 2018 16:56:21 GMT): tongli (Wed, 17 Jan 2018 16:57:05 GMT): DarshanBc (Wed, 17 Jan 2018 18:17:49 GMT): tongli (Wed, 17 Jan 2018 19:04:42 GMT): tongli (Wed, 17 Jan 2018 19:05:21 GMT): erikpupo (Wed, 17 Jan 2018 23:22:01 GMT): DarshanBc (Thu, 18 Jan 2018 04:01:11 GMT): baohua (Thu, 18 Jan 2018 05:29:09 GMT): zhmz1326 (Thu, 18 Jan 2018 05:44:48 GMT): huiyingGong (Thu, 18 Jan 2018 06:27:05 GMT): huiyingGong (Thu, 18 Jan 2018 06:27:21 GMT): yatheesan (Thu, 18 Jan 2018 07:36:21 GMT): hightall (Thu, 18 Jan 2018 08:32:10 GMT): baohua (Thu, 18 Jan 2018 08:36:21 GMT): varun-raj (Thu, 18 Jan 2018 09:09:05 GMT): SunilMahamuni (Thu, 18 Jan 2018 09:24:43 GMT): SunilMahamuni (Thu, 18 Jan 2018 09:26:16 GMT): SunilMahamuni (Thu, 18 Jan 2018 09:26:46 GMT): harsha (Thu, 18 Jan 2018 09:27:56 GMT): harsha (Thu, 18 Jan 2018 09:27:59 GMT): wbwangk (Thu, 18 Jan 2018 11:56:08 GMT): bryancan (Thu, 18 Jan 2018 17:56:24 GMT): douglasthiel (Thu, 18 Jan 2018 18:31:36 GMT): hightall (Fri, 19 Jan 2018 01:52:06 GMT): baohua (Fri, 19 Jan 2018 02:56:01 GMT): fiendster (Fri, 19 Jan 2018 03:19:58 GMT): huiyingGong (Fri, 19 Jan 2018 03:42:02 GMT): baohua (Fri, 19 Jan 2018 06:12:37 GMT): baohua (Fri, 19 Jan 2018 06:12:38 GMT): huiyingGong (Fri, 19 Jan 2018 06:31:43 GMT): baohua (Fri, 19 Jan 2018 07:11:10 GMT): baohua (Fri, 19 Jan 2018 07:11:14 GMT): B2BProgrammer (Fri, 19 Jan 2018 07:56:07 GMT): joaquimpedrooliveira (Fri, 19 Jan 2018 20:06:38 GMT): ctyu (Sat, 20 Jan 2018 05:27:28 GMT): ctyu (Sat, 20 Jan 2018 05:27:37 GMT): ctyu (Sat, 20 Jan 2018 05:30:43 GMT): harsha (Sat, 20 Jan 2018 10:12:20 GMT): baohua (Sat, 20 Jan 2018 12:57:38 GMT): hightall (Sat, 20 Jan 2018 13:06:02 GMT): lujiang (Sat, 20 Jan 2018 15:29:47 GMT): ctyu (Sun, 21 Jan 2018 06:37:52 GMT): DarshanBc (Sun, 21 Jan 2018 12:26:05 GMT): DarshanBc (Sun, 21 Jan 2018 13:10:24 GMT): tongli (Sun, 21 Jan 2018 13:58:00 GMT): DarshanBc (Sun, 21 Jan 2018 14:19:49 GMT): tongli (Sun, 21 Jan 2018 14:21:22 GMT): tiknoh (Sun, 21 Jan 2018 14:22:34 GMT): DarshanBc (Sun, 21 Jan 2018 14:30:34 GMT): DarshanBc (Sun, 21 Jan 2018 14:30:54 GMT): DarshanBc (Sun, 21 Jan 2018 15:32:11 GMT): tongli (Sun, 21 Jan 2018 19:27:07 GMT): baohua (Mon, 22 Jan 2018 02:00:21 GMT): tongli (Mon, 22 Jan 2018 02:01:56 GMT): tongli (Mon, 22 Jan 2018 02:02:23 GMT): baohua (Mon, 22 Jan 2018 02:02:54 GMT): tongli (Mon, 22 Jan 2018 02:02:57 GMT): baohua (Mon, 22 Jan 2018 02:03:31 GMT): tongli (Mon, 22 Jan 2018 02:03:31 GMT): baohua (Mon, 22 Jan 2018 02:03:46 GMT): baohua (Mon, 22 Jan 2018 02:04:40 GMT): tongli (Mon, 22 Jan 2018 02:04:57 GMT): baohua (Mon, 22 Jan 2018 02:05:07 GMT): tongli (Mon, 22 Jan 2018 02:06:46 GMT): baohua (Mon, 22 Jan 2018 02:13:05 GMT): baohua (Mon, 22 Jan 2018 02:13:20 GMT): baohua (Mon, 22 Jan 2018 02:13:31 GMT): tongli (Mon, 22 Jan 2018 02:14:06 GMT): tongli (Mon, 22 Jan 2018 02:14:46 GMT): tongli (Mon, 22 Jan 2018 02:15:08 GMT): tongli (Mon, 22 Jan 2018 02:15:12 GMT): baohua (Mon, 22 Jan 2018 02:15:14 GMT): tongli (Mon, 22 Jan 2018 02:15:47 GMT): baohua (Mon, 22 Jan 2018 02:15:54 GMT): tongli (Mon, 22 Jan 2018 02:15:56 GMT): baohua (Mon, 22 Jan 2018 02:16:04 GMT): baohua (Mon, 22 Jan 2018 02:17:01 GMT): tongli (Mon, 22 Jan 2018 02:17:41 GMT): baohua (Mon, 22 Jan 2018 02:17:44 GMT): tongli (Mon, 22 Jan 2018 02:18:11 GMT): baohua (Mon, 22 Jan 2018 02:18:38 GMT): baohua (Mon, 22 Jan 2018 02:18:56 GMT): harsha (Mon, 22 Jan 2018 02:29:44 GMT): harsha (Mon, 22 Jan 2018 02:33:27 GMT): baohua (Mon, 22 Jan 2018 02:35:27 GMT): baohua (Mon, 22 Jan 2018 02:37:31 GMT): baohua (Mon, 22 Jan 2018 02:37:54 GMT): baohua (Mon, 22 Jan 2018 02:38:04 GMT): harsha (Mon, 22 Jan 2018 02:40:10 GMT): baohua (Mon, 22 Jan 2018 02:40:50 GMT): ctyu (Mon, 22 Jan 2018 02:57:24 GMT): ctyu (Mon, 22 Jan 2018 02:58:34 GMT): ctyu (Mon, 22 Jan 2018 02:58:57 GMT): ctyu (Mon, 22 Jan 2018 02:59:02 GMT): ctyu (Mon, 22 Jan 2018 02:59:05 GMT): baohua (Mon, 22 Jan 2018 03:01:16 GMT): tongli (Mon, 22 Jan 2018 03:01:40 GMT): tongli (Mon, 22 Jan 2018 03:03:57 GMT): ctyu (Mon, 22 Jan 2018 03:05:19 GMT): baohua (Mon, 22 Jan 2018 03:06:24 GMT): harsha (Mon, 22 Jan 2018 03:06:36 GMT): baohua (Mon, 22 Jan 2018 03:06:39 GMT): tongli (Mon, 22 Jan 2018 03:06:59 GMT): tongli (Mon, 22 Jan 2018 03:07:24 GMT): ctyu (Mon, 22 Jan 2018 03:08:28 GMT): tongli (Mon, 22 Jan 2018 03:08:52 GMT): tongli (Mon, 22 Jan 2018 03:08:57 GMT): ctyu (Mon, 22 Jan 2018 03:10:00 GMT): baohua (Mon, 22 Jan 2018 03:10:40 GMT): tongli (Mon, 22 Jan 2018 03:10:59 GMT): ctyu (Mon, 22 Jan 2018 03:11:19 GMT): tongli (Mon, 22 Jan 2018 03:11:23 GMT): baohua (Mon, 22 Jan 2018 03:11:43 GMT): baohua (Mon, 22 Jan 2018 03:11:57 GMT): tongli (Mon, 22 Jan 2018 03:12:37 GMT): tongli (Mon, 22 Jan 2018 03:12:43 GMT): tongli (Mon, 22 Jan 2018 03:13:07 GMT): tongli (Mon, 22 Jan 2018 03:13:38 GMT): ctyu (Mon, 22 Jan 2018 03:14:44 GMT): ctyu (Mon, 22 Jan 2018 03:14:46 GMT): ctyu (Mon, 22 Jan 2018 03:15:34 GMT): tongli (Mon, 22 Jan 2018 03:16:04 GMT): ctyu (Mon, 22 Jan 2018 03:16:22 GMT): tongli (Mon, 22 Jan 2018 03:16:31 GMT): ctyu (Mon, 22 Jan 2018 03:16:39 GMT): ctyu (Mon, 22 Jan 2018 03:17:35 GMT): ctyu (Mon, 22 Jan 2018 03:17:36 GMT): ctyu (Mon, 22 Jan 2018 03:17:41 GMT): tongli (Mon, 22 Jan 2018 03:18:25 GMT): tongli (Mon, 22 Jan 2018 03:18:47 GMT): tongli (Mon, 22 Jan 2018 03:19:02 GMT): ctyu (Mon, 22 Jan 2018 03:19:08 GMT): tongli (Mon, 22 Jan 2018 03:19:13 GMT): tongli (Mon, 22 Jan 2018 03:19:21 GMT): tongli (Mon, 22 Jan 2018 03:19:42 GMT): tongli (Mon, 22 Jan 2018 03:20:55 GMT): tongli (Mon, 22 Jan 2018 03:21:00 GMT): tongli (Mon, 22 Jan 2018 03:21:06 GMT): tongli (Mon, 22 Jan 2018 03:21:34 GMT): tongli (Mon, 22 Jan 2018 03:21:50 GMT): ctyu (Mon, 22 Jan 2018 03:22:49 GMT): ctyu (Mon, 22 Jan 2018 03:23:06 GMT): ctyu (Mon, 22 Jan 2018 03:23:48 GMT): ctyu (Mon, 22 Jan 2018 03:23:49 GMT): ctyu (Mon, 22 Jan 2018 03:24:09 GMT): ctyu (Mon, 22 Jan 2018 03:35:09 GMT): ctyu (Mon, 22 Jan 2018 03:35:14 GMT): ctyu (Mon, 22 Jan 2018 03:55:07 GMT): ctyu (Mon, 22 Jan 2018 03:55:18 GMT): DarshanBc (Mon, 22 Jan 2018 05:51:18 GMT): DarshanBc (Mon, 22 Jan 2018 06:08:55 GMT): DarshanBc (Mon, 22 Jan 2018 06:19:43 GMT): DarshanBc (Mon, 22 Jan 2018 06:19:43 GMT): DarshanBc (Mon, 22 Jan 2018 06:19:43 GMT): DarshanBc (Mon, 22 Jan 2018 06:19:43 GMT): DarshanBc (Mon, 22 Jan 2018 06:19:43 GMT): DarshanBc (Mon, 22 Jan 2018 06:19:43 GMT): DarshanBc (Mon, 22 Jan 2018 06:19:43 GMT): DarshanBc (Mon, 22 Jan 2018 06:19:43 GMT): DarshanBc (Mon, 22 Jan 2018 06:19:43 GMT): DarshanBc (Mon, 22 Jan 2018 06:19:43 GMT): DarshanBc (Mon, 22 Jan 2018 06:19:43 GMT): DarshanBc (Mon, 22 Jan 2018 06:33:03 GMT): ctyu (Mon, 22 Jan 2018 06:54:44 GMT): ctyu (Mon, 22 Jan 2018 06:54:52 GMT): rajivgandhi2010 (Mon, 22 Jan 2018 07:21:04 GMT): rajivgandhi2010 (Mon, 22 Jan 2018 07:47:13 GMT): rajivgandhi2010 (Mon, 22 Jan 2018 07:47:28 GMT): ctyu (Mon, 22 Jan 2018 07:48:22 GMT): rajivgandhi2010 (Mon, 22 Jan 2018 07:56:18 GMT): prithveesh (Mon, 22 Jan 2018 11:44:06 GMT): DarshanBc (Mon, 22 Jan 2018 12:11:01 GMT): bingimanasa (Mon, 22 Jan 2018 12:36:03 GMT): rameshthoomu (Mon, 22 Jan 2018 13:11:33 GMT): rameshthoomu (Mon, 22 Jan 2018 13:12:34 GMT): rameshthoomu (Mon, 22 Jan 2018 13:13:06 GMT): harsha (Mon, 22 Jan 2018 13:43:10 GMT): baohua (Mon, 22 Jan 2018 14:21:42 GMT): tongli (Mon, 22 Jan 2018 14:25:13 GMT): DarshanBc (Mon, 22 Jan 2018 14:30:31 GMT): ctyu (Mon, 22 Jan 2018 14:46:34 GMT): ctyu (Mon, 22 Jan 2018 14:47:04 GMT): harsha (Mon, 22 Jan 2018 15:08:19 GMT): harsha (Mon, 22 Jan 2018 15:08:19 GMT): rameshthoomu (Mon, 22 Jan 2018 15:12:29 GMT): harsha (Mon, 22 Jan 2018 15:14:17 GMT): HL (Mon, 22 Jan 2018 16:31:23 GMT): HL (Mon, 22 Jan 2018 16:36:04 GMT): HL (Mon, 22 Jan 2018 16:36:04 GMT): harsha (Mon, 22 Jan 2018 19:16:57 GMT): harsha (Tue, 23 Jan 2018 02:06:49 GMT): baohua (Tue, 23 Jan 2018 02:25:48 GMT): baohua (Tue, 23 Jan 2018 02:25:54 GMT): ctyu (Tue, 23 Jan 2018 02:41:14 GMT): ctyu (Tue, 23 Jan 2018 02:41:18 GMT): HL (Tue, 23 Jan 2018 02:56:52 GMT): harsha (Tue, 23 Jan 2018 03:18:19 GMT): baohua (Tue, 23 Jan 2018 08:13:46 GMT): rhansharief (Tue, 23 Jan 2018 08:28:14 GMT): WHATISOOP (Tue, 23 Jan 2018 08:42:06 GMT): sampath06 (Tue, 23 Jan 2018 13:03:06 GMT): harsha (Tue, 23 Jan 2018 15:19:49 GMT): harsha (Tue, 23 Jan 2018 15:19:49 GMT): vkblue (Tue, 23 Jan 2018 16:46:05 GMT): baohua (Wed, 24 Jan 2018 02:15:05 GMT): baohua (Wed, 24 Jan 2018 02:15:33 GMT): singh.amitkumar (Wed, 24 Jan 2018 07:02:36 GMT): DarshanBc (Wed, 24 Jan 2018 09:54:13 GMT): DarshanBc (Wed, 24 Jan 2018 09:54:24 GMT): DarshanBc (Wed, 24 Jan 2018 09:54:59 GMT): DarshanBc (Wed, 24 Jan 2018 09:55:47 GMT): DarshanBc (Wed, 24 Jan 2018 09:56:29 GMT): harsha (Wed, 24 Jan 2018 13:13:11 GMT): baohua (Wed, 24 Jan 2018 13:26:52 GMT): baohua (Wed, 24 Jan 2018 13:27:39 GMT): harsha (Wed, 24 Jan 2018 13:32:31 GMT): harsha (Wed, 24 Jan 2018 13:54:33 GMT): gesaleh (Wed, 24 Jan 2018 19:35:40 GMT): gesaleh (Wed, 24 Jan 2018 19:35:55 GMT): gesaleh (Wed, 24 Jan 2018 19:36:10 GMT): gesaleh (Wed, 24 Jan 2018 19:36:21 GMT): gesaleh (Wed, 24 Jan 2018 19:36:36 GMT): gesaleh (Wed, 24 Jan 2018 19:37:30 GMT): gesaleh (Wed, 24 Jan 2018 19:38:12 GMT): gesaleh (Wed, 24 Jan 2018 19:38:20 GMT): gesaleh (Wed, 24 Jan 2018 19:39:07 GMT): gesaleh (Wed, 24 Jan 2018 19:39:15 GMT): gesaleh (Wed, 24 Jan 2018 19:40:57 GMT): gesaleh (Wed, 24 Jan 2018 20:33:47 GMT): gesaleh (Wed, 24 Jan 2018 20:33:48 GMT): gesaleh (Wed, 24 Jan 2018 20:33:48 GMT): gesaleh (Wed, 24 Jan 2018 22:41:10 GMT): gesaleh (Wed, 24 Jan 2018 22:43:00 GMT): gesaleh (Wed, 24 Jan 2018 22:43:13 GMT): gesaleh (Wed, 24 Jan 2018 23:19:28 GMT): gesaleh (Wed, 24 Jan 2018 23:19:38 GMT): baohua (Thu, 25 Jan 2018 06:20:39 GMT): chandrairawan (Thu, 25 Jan 2018 07:30:03 GMT): DarshanBc (Thu, 25 Jan 2018 07:48:03 GMT): DarshanBc (Thu, 25 Jan 2018 07:48:22 GMT): lujiang (Thu, 25 Jan 2018 08:17:42 GMT): DarshanBc (Thu, 25 Jan 2018 08:26:02 GMT): baohua (Thu, 25 Jan 2018 08:30:47 GMT): gesaleh (Thu, 25 Jan 2018 09:09:14 GMT): gesaleh (Thu, 25 Jan 2018 09:09:14 GMT): harsha (Thu, 25 Jan 2018 10:14:51 GMT): gesaleh (Thu, 25 Jan 2018 10:56:15 GMT): gesaleh (Thu, 25 Jan 2018 10:56:32 GMT): gesaleh (Thu, 25 Jan 2018 11:25:26 GMT): gesaleh (Thu, 25 Jan 2018 11:26:02 GMT): rajivgandhi2010 (Thu, 25 Jan 2018 11:26:13 GMT): DarshanBc (Thu, 25 Jan 2018 11:51:39 GMT): DarshanBc (Thu, 25 Jan 2018 11:53:47 GMT): reyaansh (Thu, 25 Jan 2018 11:55:40 GMT): ascatox (Thu, 25 Jan 2018 13:24:49 GMT): ascatox (Thu, 25 Jan 2018 13:24:49 GMT): ascatox (Thu, 25 Jan 2018 13:24:49 GMT): gesaleh (Thu, 25 Jan 2018 13:39:55 GMT): gesaleh (Thu, 25 Jan 2018 13:40:06 GMT): harsha (Thu, 25 Jan 2018 13:56:54 GMT): gesaleh (Thu, 25 Jan 2018 14:04:45 GMT): gesaleh (Thu, 25 Jan 2018 14:05:16 GMT): gesaleh (Thu, 25 Jan 2018 14:05:32 GMT): harsha (Thu, 25 Jan 2018 14:14:19 GMT): harsha (Thu, 25 Jan 2018 14:15:48 GMT): harsha (Thu, 25 Jan 2018 14:15:48 GMT): gesaleh (Thu, 25 Jan 2018 14:25:55 GMT): gesaleh (Thu, 25 Jan 2018 14:26:51 GMT): gesaleh (Thu, 25 Jan 2018 14:27:14 GMT): harsha (Thu, 25 Jan 2018 14:28:00 GMT): harsha (Thu, 25 Jan 2018 14:31:13 GMT): gesaleh (Thu, 25 Jan 2018 14:35:57 GMT): gesaleh (Thu, 25 Jan 2018 14:36:08 GMT): gesaleh (Thu, 25 Jan 2018 14:36:28 GMT): harsha (Thu, 25 Jan 2018 14:36:44 GMT): gesaleh (Thu, 25 Jan 2018 14:37:28 GMT): gesaleh (Thu, 25 Jan 2018 14:37:41 GMT): harsha (Thu, 25 Jan 2018 14:40:44 GMT): gesaleh (Thu, 25 Jan 2018 14:41:35 GMT): gesaleh (Thu, 25 Jan 2018 14:41:42 GMT): gesaleh (Thu, 25 Jan 2018 14:41:53 GMT): gesaleh (Thu, 25 Jan 2018 14:42:50 GMT): gesaleh (Thu, 25 Jan 2018 14:43:09 GMT): gesaleh (Thu, 25 Jan 2018 14:44:09 GMT): harsha (Thu, 25 Jan 2018 14:46:37 GMT): harsha (Thu, 25 Jan 2018 14:46:39 GMT): gesaleh (Thu, 25 Jan 2018 15:00:28 GMT): harsha (Thu, 25 Jan 2018 15:04:54 GMT): tongli (Thu, 25 Jan 2018 15:16:01 GMT): tongli (Thu, 25 Jan 2018 15:17:29 GMT): tongli (Thu, 25 Jan 2018 15:20:46 GMT): tongli (Thu, 25 Jan 2018 15:21:13 GMT): tongli (Thu, 25 Jan 2018 15:25:30 GMT): baohua (Thu, 25 Jan 2018 15:26:51 GMT): tongli (Thu, 25 Jan 2018 15:27:37 GMT): tongli (Thu, 25 Jan 2018 15:27:46 GMT): tongli (Thu, 25 Jan 2018 15:28:29 GMT): tongli (Thu, 25 Jan 2018 15:28:35 GMT): gesaleh (Thu, 25 Jan 2018 15:28:43 GMT): baohua (Thu, 25 Jan 2018 15:28:49 GMT): tongli (Thu, 25 Jan 2018 15:29:14 GMT): tongli (Thu, 25 Jan 2018 15:30:00 GMT): baohua (Thu, 25 Jan 2018 15:32:28 GMT): tongli (Thu, 25 Jan 2018 15:33:42 GMT): tongli (Thu, 25 Jan 2018 15:33:49 GMT): tongli (Thu, 25 Jan 2018 15:33:56 GMT): baohua (Thu, 25 Jan 2018 15:42:23 GMT): tongli (Thu, 25 Jan 2018 18:31:53 GMT): DarshanBc (Thu, 25 Jan 2018 19:11:43 GMT): gesaleh (Thu, 25 Jan 2018 19:48:04 GMT): kerokhin (Fri, 26 Jan 2018 06:09:31 GMT): ascatox (Fri, 26 Jan 2018 08:41:23 GMT): tongli (Fri, 26 Jan 2018 21:16:25 GMT): gesaleh (Fri, 26 Jan 2018 21:22:06 GMT): tongli (Fri, 26 Jan 2018 22:22:19 GMT): tongli (Fri, 26 Jan 2018 22:22:45 GMT): gesaleh (Fri, 26 Jan 2018 22:37:11 GMT): gesaleh (Fri, 26 Jan 2018 22:37:39 GMT): gesaleh (Fri, 26 Jan 2018 22:38:03 GMT): tongli (Fri, 26 Jan 2018 22:38:35 GMT): tongli (Fri, 26 Jan 2018 22:38:50 GMT): tongli (Fri, 26 Jan 2018 22:39:22 GMT): tongli (Fri, 26 Jan 2018 22:39:33 GMT): gesaleh (Fri, 26 Jan 2018 22:39:58 GMT): gesaleh (Fri, 26 Jan 2018 22:40:16 GMT): gesaleh (Fri, 26 Jan 2018 22:40:31 GMT): gesaleh (Fri, 26 Jan 2018 22:40:48 GMT): tongli (Fri, 26 Jan 2018 22:49:31 GMT): Amir (Fri, 26 Jan 2018 23:01:39 GMT): tongli (Fri, 26 Jan 2018 23:02:54 GMT): tongli (Fri, 26 Jan 2018 23:03:47 GMT): tongli (Fri, 26 Jan 2018 23:03:55 GMT): tongli (Fri, 26 Jan 2018 23:06:29 GMT): Amir (Fri, 26 Jan 2018 23:06:35 GMT): Amir (Fri, 26 Jan 2018 23:07:33 GMT): Amir (Fri, 26 Jan 2018 23:10:02 GMT): tongli (Fri, 26 Jan 2018 23:41:21 GMT): Amir (Fri, 26 Jan 2018 23:47:25 GMT): Amir (Fri, 26 Jan 2018 23:48:44 GMT): Amir (Sat, 27 Jan 2018 00:07:24 GMT): gen_el (Sat, 27 Jan 2018 12:02:31 GMT): gen_el (Sat, 27 Jan 2018 12:03:14 GMT): gen_el (Sat, 27 Jan 2018 12:03:20 GMT): gen_el (Sat, 27 Jan 2018 12:04:22 GMT): jack21 (Sat, 27 Jan 2018 12:37:51 GMT): gen_el (Sat, 27 Jan 2018 13:58:49 GMT): gen_el (Sat, 27 Jan 2018 15:27:49 GMT): gen_el (Sat, 27 Jan 2018 15:28:12 GMT): gen_el (Sat, 27 Jan 2018 15:39:02 GMT): baohua (Sun, 28 Jan 2018 01:45:53 GMT): SjirNijssen (Sun, 28 Jan 2018 09:39:18 GMT): gesaleh (Sun, 28 Jan 2018 11:15:28 GMT): gen_el (Sun, 28 Jan 2018 12:11:33 GMT): baohua (Sun, 28 Jan 2018 12:49:40 GMT): gen_el (Sun, 28 Jan 2018 13:08:55 GMT): gen_el (Sun, 28 Jan 2018 13:09:05 GMT): gen_el (Sun, 28 Jan 2018 13:09:33 GMT): gen_el (Sun, 28 Jan 2018 13:10:52 GMT): gen_el (Sun, 28 Jan 2018 13:10:52 GMT): tongli (Sun, 28 Jan 2018 14:00:01 GMT): tongli (Sun, 28 Jan 2018 15:42:43 GMT): baohua (Mon, 29 Jan 2018 01:20:01 GMT): yoyokeen (Mon, 29 Jan 2018 02:13:11 GMT): yoyokeen (Mon, 29 Jan 2018 02:13:39 GMT): ctyu (Mon, 29 Jan 2018 03:25:55 GMT): ctyu (Mon, 29 Jan 2018 03:26:14 GMT): tongli (Mon, 29 Jan 2018 03:49:28 GMT): baohua (Mon, 29 Jan 2018 03:56:11 GMT): baohua (Mon, 29 Jan 2018 03:56:33 GMT): ctyu (Mon, 29 Jan 2018 04:11:40 GMT): ctyu (Mon, 29 Jan 2018 04:12:05 GMT): ctyu (Mon, 29 Jan 2018 04:12:17 GMT): ctyu (Mon, 29 Jan 2018 04:12:57 GMT): ctyu (Mon, 29 Jan 2018 04:12:58 GMT): baohua (Mon, 29 Jan 2018 04:36:26 GMT): vinayak mahajan (Mon, 29 Jan 2018 06:21:00 GMT): ctyu (Mon, 29 Jan 2018 07:14:44 GMT): ctyu (Mon, 29 Jan 2018 07:31:26 GMT): tritone11 (Mon, 29 Jan 2018 11:43:51 GMT): tritone11 (Mon, 29 Jan 2018 11:44:11 GMT): tritone11 (Mon, 29 Jan 2018 12:06:14 GMT): tritone11 (Mon, 29 Jan 2018 12:07:26 GMT): harsha (Mon, 29 Jan 2018 15:39:46 GMT): tongli (Mon, 29 Jan 2018 21:32:42 GMT): Amir (Mon, 29 Jan 2018 23:25:55 GMT): zajanksk (Tue, 30 Jan 2018 01:51:03 GMT): baohua (Tue, 30 Jan 2018 01:54:37 GMT): baohua (Tue, 30 Jan 2018 01:55:22 GMT): hightall (Tue, 30 Jan 2018 01:58:55 GMT): baohua (Tue, 30 Jan 2018 02:47:55 GMT): hightall (Tue, 30 Jan 2018 02:48:58 GMT): baohua (Tue, 30 Jan 2018 02:49:18 GMT): baohua (Tue, 30 Jan 2018 02:52:44 GMT): baohua (Tue, 30 Jan 2018 02:52:44 GMT): roblav96 (Tue, 30 Jan 2018 03:45:01 GMT): hightall (Tue, 30 Jan 2018 03:45:27 GMT): Amir (Tue, 30 Jan 2018 05:46:46 GMT): Amir (Tue, 30 Jan 2018 05:49:09 GMT): Luke_Chen (Tue, 30 Jan 2018 14:09:49 GMT): Luke_Chen (Tue, 30 Jan 2018 14:09:49 GMT): Amir (Tue, 30 Jan 2018 22:37:03 GMT): tkueda (Wed, 31 Jan 2018 00:15:59 GMT): tkueda (Wed, 31 Jan 2018 00:40:42 GMT): baohua (Wed, 31 Jan 2018 01:15:59 GMT): tkueda (Wed, 31 Jan 2018 01:23:37 GMT): DarshanBc (Wed, 31 Jan 2018 03:48:17 GMT): DarshanBc (Wed, 31 Jan 2018 03:48:17 GMT): DarshanBc (Wed, 31 Jan 2018 04:04:51 GMT): tongli (Wed, 31 Jan 2018 04:05:43 GMT): tongli (Wed, 31 Jan 2018 04:05:57 GMT): tongli (Wed, 31 Jan 2018 04:06:23 GMT): tongli (Wed, 31 Jan 2018 04:06:34 GMT): tongli (Wed, 31 Jan 2018 04:06:40 GMT): tongli (Wed, 31 Jan 2018 04:07:00 GMT): tongli (Wed, 31 Jan 2018 04:07:09 GMT): tongli (Wed, 31 Jan 2018 04:07:15 GMT): tongli (Wed, 31 Jan 2018 04:07:59 GMT): tongli (Wed, 31 Jan 2018 04:08:12 GMT): tongli (Wed, 31 Jan 2018 04:08:23 GMT): tongli (Wed, 31 Jan 2018 04:08:36 GMT): FiratSertgoz (Wed, 31 Jan 2018 13:52:38 GMT): william123 (Wed, 31 Jan 2018 15:16:32 GMT): MohammadObaid (Wed, 31 Jan 2018 17:02:40 GMT): tongli (Wed, 31 Jan 2018 17:03:32 GMT): tongli (Wed, 31 Jan 2018 17:03:57 GMT): tongli (Wed, 31 Jan 2018 17:04:27 GMT): tongli (Wed, 31 Jan 2018 17:04:41 GMT): tongli (Wed, 31 Jan 2018 17:04:55 GMT): tongli (Wed, 31 Jan 2018 17:05:18 GMT): tongli (Wed, 31 Jan 2018 17:09:45 GMT): saeedi (Wed, 31 Jan 2018 17:12:08 GMT): tongli (Wed, 31 Jan 2018 17:13:35 GMT): tongli (Wed, 31 Jan 2018 17:13:58 GMT): tongli (Wed, 31 Jan 2018 17:15:26 GMT): saeedi (Wed, 31 Jan 2018 17:18:22 GMT): tongli (Wed, 31 Jan 2018 17:20:48 GMT): tongli (Wed, 31 Jan 2018 17:20:58 GMT): saeedi (Wed, 31 Jan 2018 17:22:13 GMT): tongli (Wed, 31 Jan 2018 17:22:28 GMT): tongli (Wed, 31 Jan 2018 17:22:54 GMT): tongli (Wed, 31 Jan 2018 17:23:01 GMT): tongli (Wed, 31 Jan 2018 17:23:23 GMT): tongli (Wed, 31 Jan 2018 17:23:39 GMT): saeedi (Wed, 31 Jan 2018 17:24:38 GMT): MohammadObaid (Wed, 31 Jan 2018 17:25:13 GMT): saeedi (Wed, 31 Jan 2018 17:25:57 GMT): saeedi (Wed, 31 Jan 2018 17:26:56 GMT): MohammadObaid (Wed, 31 Jan 2018 17:28:20 GMT): tongli (Wed, 31 Jan 2018 17:28:34 GMT): tongli (Wed, 31 Jan 2018 17:29:09 GMT): tongli (Wed, 31 Jan 2018 17:29:14 GMT): tongli (Wed, 31 Jan 2018 17:30:17 GMT): saeedi (Wed, 31 Jan 2018 17:30:32 GMT): tongli (Wed, 31 Jan 2018 17:30:49 GMT): tongli (Wed, 31 Jan 2018 17:31:14 GMT): tongli (Wed, 31 Jan 2018 17:31:49 GMT): tongli (Wed, 31 Jan 2018 17:32:33 GMT): tongli (Wed, 31 Jan 2018 17:33:05 GMT): tongli (Wed, 31 Jan 2018 17:33:21 GMT): saeedi (Wed, 31 Jan 2018 17:36:55 GMT): tongli (Wed, 31 Jan 2018 17:37:54 GMT): MohammadObaid (Wed, 31 Jan 2018 17:38:06 GMT): MohammadObaid (Wed, 31 Jan 2018 17:38:37 GMT): tongli (Wed, 31 Jan 2018 17:38:38 GMT): tongli (Wed, 31 Jan 2018 17:39:28 GMT): MohammadObaid (Wed, 31 Jan 2018 17:39:30 GMT): MohammadObaid (Wed, 31 Jan 2018 17:39:46 GMT): tongli (Wed, 31 Jan 2018 17:40:35 GMT): MohammadObaid (Wed, 31 Jan 2018 17:41:27 GMT): tongli (Wed, 31 Jan 2018 17:41:31 GMT): tongli (Wed, 31 Jan 2018 17:43:54 GMT): saeedi (Wed, 31 Jan 2018 17:44:46 GMT): saeedi (Wed, 31 Jan 2018 17:44:46 GMT): tongli (Wed, 31 Jan 2018 17:44:55 GMT): tongli (Wed, 31 Jan 2018 17:45:00 GMT): saeedi (Wed, 31 Jan 2018 17:45:08 GMT): saeedi (Wed, 31 Jan 2018 17:47:22 GMT): tongli (Wed, 31 Jan 2018 17:49:17 GMT): tongli (Wed, 31 Jan 2018 17:49:35 GMT): saeedi (Wed, 31 Jan 2018 17:51:11 GMT): tongli (Wed, 31 Jan 2018 17:51:18 GMT): saeedi (Wed, 31 Jan 2018 17:52:44 GMT): saeedi (Wed, 31 Jan 2018 18:01:12 GMT): tongli (Wed, 31 Jan 2018 18:25:14 GMT): ajayatgit (Wed, 31 Jan 2018 21:21:40 GMT): Amir (Wed, 31 Jan 2018 23:19:16 GMT): Norberthu (Thu, 01 Feb 2018 03:09:57 GMT): tongli (Thu, 01 Feb 2018 03:39:49 GMT): tongli (Thu, 01 Feb 2018 03:41:19 GMT): tongli (Thu, 01 Feb 2018 03:41:44 GMT): vchengsong (Thu, 01 Feb 2018 03:52:00 GMT): LeDang (Thu, 01 Feb 2018 04:00:28 GMT): baohua (Thu, 01 Feb 2018 06:36:46 GMT): jackyLee (Thu, 01 Feb 2018 08:55:03 GMT): tongli (Thu, 01 Feb 2018 13:14:49 GMT): william123 (Thu, 01 Feb 2018 13:27:43 GMT): Darumada (Thu, 01 Feb 2018 13:41:01 GMT): Darumada (Thu, 01 Feb 2018 13:46:02 GMT): Darumada (Thu, 01 Feb 2018 14:07:22 GMT): tongli (Thu, 01 Feb 2018 14:34:38 GMT): tongli (Thu, 01 Feb 2018 14:35:43 GMT): tongli (Thu, 01 Feb 2018 14:43:27 GMT): Darumada (Thu, 01 Feb 2018 14:52:06 GMT): baohua (Thu, 01 Feb 2018 15:05:27 GMT): baohua (Thu, 01 Feb 2018 15:08:34 GMT): tongli (Thu, 01 Feb 2018 15:09:00 GMT): tongli (Thu, 01 Feb 2018 15:09:12 GMT): baohua (Thu, 01 Feb 2018 15:09:30 GMT): tongli (Thu, 01 Feb 2018 15:09:53 GMT): baohua (Thu, 01 Feb 2018 15:10:47 GMT): baohua (Thu, 01 Feb 2018 15:10:47 GMT): baohua (Thu, 01 Feb 2018 15:12:52 GMT): baohua (Thu, 01 Feb 2018 15:13:26 GMT): tongli (Thu, 01 Feb 2018 15:18:20 GMT): tongli (Thu, 01 Feb 2018 15:19:37 GMT): baohua (Thu, 01 Feb 2018 15:21:08 GMT): baohua (Thu, 01 Feb 2018 15:24:17 GMT): FiratSertgoz (Thu, 01 Feb 2018 15:35:41 GMT): oleg.borovyk (Thu, 01 Feb 2018 17:23:28 GMT): william123 (Thu, 01 Feb 2018 19:24:41 GMT): tongli (Fri, 02 Feb 2018 00:31:48 GMT): tongli (Fri, 02 Feb 2018 00:32:33 GMT): tongli (Fri, 02 Feb 2018 00:33:48 GMT): tongli (Fri, 02 Feb 2018 00:34:22 GMT): Amir (Fri, 02 Feb 2018 00:43:48 GMT): Amir (Fri, 02 Feb 2018 00:44:05 GMT): Amir (Fri, 02 Feb 2018 00:44:18 GMT): Amir (Fri, 02 Feb 2018 00:44:29 GMT): Amir (Fri, 02 Feb 2018 00:44:51 GMT): Amir (Fri, 02 Feb 2018 00:47:31 GMT): Amir (Fri, 02 Feb 2018 00:48:29 GMT): Amir (Fri, 02 Feb 2018 00:48:46 GMT): baohua (Fri, 02 Feb 2018 01:29:55 GMT): fiendster (Fri, 02 Feb 2018 03:48:43 GMT): fiendster (Fri, 02 Feb 2018 04:52:46 GMT): DarshanBc (Fri, 02 Feb 2018 10:39:02 GMT): DarshanBc (Fri, 02 Feb 2018 10:39:02 GMT): DarshanBc (Fri, 02 Feb 2018 10:48:19 GMT): DarshanBc (Fri, 02 Feb 2018 10:48:49 GMT): Darumada (Fri, 02 Feb 2018 11:39:47 GMT): harsha (Fri, 02 Feb 2018 13:13:39 GMT): cuijianlong (Fri, 02 Feb 2018 15:12:58 GMT): tongli (Fri, 02 Feb 2018 15:31:49 GMT): tongli (Fri, 02 Feb 2018 15:32:09 GMT): baohua (Sun, 04 Feb 2018 08:17:25 GMT): Darumada (Sun, 04 Feb 2018 09:45:26 GMT): Darumada (Sun, 04 Feb 2018 09:47:09 GMT): Darumada (Sun, 04 Feb 2018 09:47:17 GMT): Darumada (Sun, 04 Feb 2018 09:47:24 GMT): Darumada (Sun, 04 Feb 2018 09:47:44 GMT): Darumada (Sun, 04 Feb 2018 09:47:48 GMT): Darumada (Sun, 04 Feb 2018 09:47:48 GMT): Darumada (Sun, 04 Feb 2018 09:49:06 GMT): Amir (Sun, 04 Feb 2018 10:58:31 GMT): Amir (Sun, 04 Feb 2018 10:58:57 GMT): Amir (Sun, 04 Feb 2018 10:59:06 GMT): Norberthu (Sun, 04 Feb 2018 11:51:16 GMT): Norberthu (Sun, 04 Feb 2018 11:53:26 GMT): Norberthu (Sun, 04 Feb 2018 11:54:44 GMT): Norberthu (Sun, 04 Feb 2018 11:55:25 GMT): Norberthu (Sun, 04 Feb 2018 11:59:11 GMT): baohua (Sun, 04 Feb 2018 13:55:51 GMT): baohua (Sun, 04 Feb 2018 15:10:51 GMT): MohammadObaid (Sun, 04 Feb 2018 17:43:55 GMT): MohammadObaid (Sun, 04 Feb 2018 17:44:54 GMT): Amir (Sun, 04 Feb 2018 21:13:53 GMT): Darumada (Sun, 04 Feb 2018 23:48:14 GMT): baohua (Mon, 05 Feb 2018 01:41:52 GMT): Norberthu (Mon, 05 Feb 2018 02:52:45 GMT): Norberthu (Mon, 05 Feb 2018 02:53:00 GMT): tkueda (Mon, 05 Feb 2018 03:02:50 GMT): baohua (Mon, 05 Feb 2018 03:22:49 GMT): baohua (Mon, 05 Feb 2018 03:22:53 GMT): baohua (Mon, 05 Feb 2018 03:23:01 GMT): tkueda (Mon, 05 Feb 2018 04:49:34 GMT): Amir (Mon, 05 Feb 2018 05:03:56 GMT): baohua (Mon, 05 Feb 2018 05:04:18 GMT): baohua (Mon, 05 Feb 2018 05:04:18 GMT): baohua (Mon, 05 Feb 2018 05:04:37 GMT): baohua (Mon, 05 Feb 2018 05:05:29 GMT): Norberthu (Mon, 05 Feb 2018 06:31:18 GMT): Amir (Mon, 05 Feb 2018 07:33:40 GMT): Darumada (Mon, 05 Feb 2018 10:43:31 GMT): berestet (Mon, 05 Feb 2018 18:52:22 GMT): berestet (Mon, 05 Feb 2018 18:53:16 GMT): baohua (Tue, 06 Feb 2018 05:53:21 GMT): harsha (Tue, 06 Feb 2018 08:23:53 GMT): harsha (Tue, 06 Feb 2018 08:32:20 GMT): harsha (Tue, 06 Feb 2018 08:32:20 GMT): lujiang (Tue, 06 Feb 2018 12:40:08 GMT): lujiang (Tue, 06 Feb 2018 13:17:22 GMT): lujiang (Tue, 06 Feb 2018 13:28:14 GMT): lujiang (Tue, 06 Feb 2018 13:43:29 GMT): baohua (Tue, 06 Feb 2018 14:28:36 GMT): harsha (Tue, 06 Feb 2018 14:29:24 GMT): baohua (Tue, 06 Feb 2018 14:30:59 GMT): baohua (Tue, 06 Feb 2018 14:33:34 GMT): baohua (Tue, 06 Feb 2018 14:34:09 GMT): harsha (Tue, 06 Feb 2018 14:34:36 GMT): lujiang (Tue, 06 Feb 2018 14:47:42 GMT): harsha (Tue, 06 Feb 2018 14:51:29 GMT): harsha (Tue, 06 Feb 2018 14:51:29 GMT): lujiang (Tue, 06 Feb 2018 14:57:51 GMT): harsha (Tue, 06 Feb 2018 15:02:37 GMT): harsha (Tue, 06 Feb 2018 15:02:37 GMT): pacoard (Tue, 06 Feb 2018 19:03:25 GMT): pacoard (Tue, 06 Feb 2018 19:05:10 GMT): Amir (Tue, 06 Feb 2018 19:12:40 GMT): tongli (Tue, 06 Feb 2018 21:43:45 GMT): tongli (Tue, 06 Feb 2018 21:44:22 GMT): baohua (Wed, 07 Feb 2018 00:16:51 GMT): baohua (Wed, 07 Feb 2018 00:17:40 GMT): tongli (Wed, 07 Feb 2018 01:30:53 GMT): baohua (Wed, 07 Feb 2018 01:42:08 GMT): tongli (Wed, 07 Feb 2018 02:24:59 GMT): Norberthu (Wed, 07 Feb 2018 10:03:49 GMT): lujiang (Wed, 07 Feb 2018 10:17:11 GMT): lujiang (Wed, 07 Feb 2018 10:17:17 GMT): harsha (Wed, 07 Feb 2018 13:13:16 GMT): paul.sitoh (Wed, 07 Feb 2018 16:52:14 GMT): paul.sitoh (Wed, 07 Feb 2018 16:52:14 GMT): paul.sitoh (Wed, 07 Feb 2018 16:52:14 GMT): harsha (Wed, 07 Feb 2018 17:11:39 GMT): tongli (Wed, 07 Feb 2018 19:14:33 GMT): tongli (Wed, 07 Feb 2018 19:15:12 GMT): yoyokeen (Thu, 08 Feb 2018 01:32:03 GMT): yoyokeen (Thu, 08 Feb 2018 01:32:08 GMT): yoyokeen (Thu, 08 Feb 2018 01:32:49 GMT): yoyokeen (Thu, 08 Feb 2018 01:34:18 GMT): baohua (Thu, 08 Feb 2018 01:44:39 GMT): yoyokeen (Thu, 08 Feb 2018 01:46:51 GMT): yoyokeen (Thu, 08 Feb 2018 01:47:01 GMT): yoyokeen (Thu, 08 Feb 2018 01:47:24 GMT): baohua (Thu, 08 Feb 2018 01:48:20 GMT): baohua (Thu, 08 Feb 2018 01:48:44 GMT): yoyokeen (Thu, 08 Feb 2018 01:49:34 GMT): baohua (Thu, 08 Feb 2018 01:50:15 GMT): yoyokeen (Thu, 08 Feb 2018 01:56:49 GMT): yoyokeen (Thu, 08 Feb 2018 01:56:57 GMT): baohua (Thu, 08 Feb 2018 01:59:37 GMT): yoyokeen (Thu, 08 Feb 2018 02:00:09 GMT): baohua (Thu, 08 Feb 2018 02:01:04 GMT): Amir (Thu, 08 Feb 2018 02:36:15 GMT): Amir (Thu, 08 Feb 2018 02:36:23 GMT): Amir (Thu, 08 Feb 2018 02:36:57 GMT): Amir (Thu, 08 Feb 2018 02:37:20 GMT): Amir (Thu, 08 Feb 2018 02:41:16 GMT): baohua (Thu, 08 Feb 2018 03:25:28 GMT): yoyokeen (Thu, 08 Feb 2018 06:48:40 GMT): lc_wxk (Thu, 08 Feb 2018 06:55:31 GMT): lc_wxk (Thu, 08 Feb 2018 06:55:50 GMT): baohua (Thu, 08 Feb 2018 07:02:13 GMT): lc_wxk (Thu, 08 Feb 2018 07:07:20 GMT): baohua (Thu, 08 Feb 2018 07:08:24 GMT): yoyokeen (Thu, 08 Feb 2018 07:50:51 GMT): yoyokeen (Thu, 08 Feb 2018 07:50:57 GMT): yoyokeen (Thu, 08 Feb 2018 07:58:44 GMT): lujiang (Thu, 08 Feb 2018 08:30:42 GMT): baohua (Thu, 08 Feb 2018 09:24:02 GMT): harsha (Thu, 08 Feb 2018 13:59:47 GMT): harsha (Thu, 08 Feb 2018 13:59:47 GMT): bipinu (Thu, 08 Feb 2018 14:55:43 GMT): MohammadObaid (Thu, 08 Feb 2018 15:41:20 GMT): tongli (Thu, 08 Feb 2018 15:42:16 GMT): tongli (Thu, 08 Feb 2018 15:42:52 GMT): tongli (Thu, 08 Feb 2018 15:42:53 GMT): tongli (Thu, 08 Feb 2018 15:43:06 GMT): tongli (Thu, 08 Feb 2018 15:43:28 GMT): tongli (Thu, 08 Feb 2018 15:43:53 GMT): tongli (Thu, 08 Feb 2018 15:43:54 GMT): tongli (Thu, 08 Feb 2018 15:44:29 GMT): MohammadObaid (Thu, 08 Feb 2018 15:48:34 GMT): MohammadObaid (Thu, 08 Feb 2018 15:48:34 GMT): MohammadObaid (Thu, 08 Feb 2018 15:56:21 GMT): bleggett (Thu, 08 Feb 2018 18:40:21 GMT): bleggett (Thu, 08 Feb 2018 18:40:32 GMT): bleggett (Thu, 08 Feb 2018 19:09:25 GMT): tongli (Thu, 08 Feb 2018 19:43:43 GMT): tongli (Thu, 08 Feb 2018 19:43:58 GMT): Amir (Thu, 08 Feb 2018 20:52:54 GMT): Amir (Thu, 08 Feb 2018 20:54:37 GMT): Amir (Thu, 08 Feb 2018 22:05:23 GMT): Amir (Thu, 08 Feb 2018 22:09:12 GMT): baohua (Fri, 09 Feb 2018 01:11:45 GMT): Amir (Fri, 09 Feb 2018 01:31:23 GMT): lujiang (Fri, 09 Feb 2018 02:55:29 GMT): lujiang (Fri, 09 Feb 2018 02:56:41 GMT): lujiang (Fri, 09 Feb 2018 02:59:37 GMT): lujiang (Fri, 09 Feb 2018 02:59:46 GMT): baohua (Fri, 09 Feb 2018 03:01:07 GMT): Brandon (Fri, 09 Feb 2018 09:46:15 GMT): Brandon (Fri, 09 Feb 2018 09:46:50 GMT): yoyokeen (Fri, 09 Feb 2018 11:47:24 GMT): yoyokeen (Fri, 09 Feb 2018 14:19:29 GMT): harsha (Fri, 09 Feb 2018 14:50:34 GMT): Amir (Fri, 09 Feb 2018 17:36:31 GMT): Amir (Fri, 09 Feb 2018 19:56:31 GMT): zsmartc (Sat, 10 Feb 2018 08:49:15 GMT): zsmartc (Sat, 10 Feb 2018 09:00:09 GMT): Amir (Sat, 10 Feb 2018 09:41:15 GMT): baohua (Sat, 10 Feb 2018 10:04:14 GMT): hightall (Sat, 10 Feb 2018 14:50:11 GMT): harsha (Sat, 10 Feb 2018 17:21:02 GMT): pacoard (Sat, 10 Feb 2018 18:21:44 GMT): pacoard (Sat, 10 Feb 2018 18:21:46 GMT): pacoard (Sat, 10 Feb 2018 18:22:29 GMT): baohua (Sun, 11 Feb 2018 01:44:28 GMT): baohua (Sun, 11 Feb 2018 01:44:28 GMT): baohua (Sun, 11 Feb 2018 01:45:39 GMT): baohua (Sun, 11 Feb 2018 01:57:46 GMT): Brandon (Sun, 11 Feb 2018 02:47:53 GMT): Brandon (Sun, 11 Feb 2018 02:48:28 GMT): Brandon (Sun, 11 Feb 2018 02:49:13 GMT): Brandon (Sun, 11 Feb 2018 02:49:56 GMT): zsmartc (Sun, 11 Feb 2018 02:54:09 GMT): MohammadObaid (Sun, 11 Feb 2018 05:56:24 GMT): baohua (Sun, 11 Feb 2018 05:57:12 GMT): MohammadObaid (Sun, 11 Feb 2018 05:59:26 GMT): MohammadObaid (Sun, 11 Feb 2018 05:59:45 GMT): Luke_Chen (Sun, 11 Feb 2018 06:40:49 GMT): Luke_Chen (Sun, 11 Feb 2018 06:44:25 GMT): Brandon (Sun, 11 Feb 2018 06:56:19 GMT): Luke_Chen (Sun, 11 Feb 2018 06:57:36 GMT): Luke_Chen (Sun, 11 Feb 2018 06:57:58 GMT): Luke_Chen (Sun, 11 Feb 2018 07:03:56 GMT): Luke_Chen (Sun, 11 Feb 2018 07:03:56 GMT): Luke_Chen (Sun, 11 Feb 2018 07:05:23 GMT): Randyshu2018 (Sun, 11 Feb 2018 07:56:48 GMT): zsmartc (Sun, 11 Feb 2018 13:18:00 GMT): Amir (Sun, 11 Feb 2018 19:17:00 GMT): tongli (Sun, 11 Feb 2018 22:04:19 GMT): baohua (Mon, 12 Feb 2018 01:35:27 GMT): baohua (Mon, 12 Feb 2018 01:43:07 GMT): lujiang (Mon, 12 Feb 2018 06:41:16 GMT): baohua (Mon, 12 Feb 2018 06:42:23 GMT): lujiang (Mon, 12 Feb 2018 06:46:13 GMT): lujiang (Mon, 12 Feb 2018 06:46:58 GMT): baohua (Mon, 12 Feb 2018 06:51:03 GMT): hightall (Mon, 12 Feb 2018 07:11:52 GMT): lujiang (Mon, 12 Feb 2018 07:12:19 GMT): Luke_Chen (Mon, 12 Feb 2018 07:36:10 GMT): Luke_Chen (Mon, 12 Feb 2018 07:36:10 GMT): zsmartc (Mon, 12 Feb 2018 07:39:05 GMT): hightall (Mon, 12 Feb 2018 07:39:06 GMT): Amir (Mon, 12 Feb 2018 07:50:05 GMT): Luke_Chen (Mon, 12 Feb 2018 07:52:27 GMT): Luke_Chen (Mon, 12 Feb 2018 07:54:06 GMT): Luke_Chen (Mon, 12 Feb 2018 07:54:58 GMT): lujiang (Mon, 12 Feb 2018 07:55:38 GMT): lujiang (Mon, 12 Feb 2018 07:56:04 GMT): Amir (Mon, 12 Feb 2018 07:58:37 GMT): Amir (Mon, 12 Feb 2018 08:04:58 GMT): Amir (Mon, 12 Feb 2018 08:05:04 GMT): lujiang (Mon, 12 Feb 2018 08:21:36 GMT): zsmartc (Mon, 12 Feb 2018 09:27:12 GMT): harsha (Mon, 12 Feb 2018 09:39:43 GMT): harsha (Mon, 12 Feb 2018 10:09:22 GMT): baohua (Mon, 12 Feb 2018 10:41:02 GMT): tongli (Mon, 12 Feb 2018 12:34:33 GMT): baohua (Mon, 12 Feb 2018 12:37:05 GMT): baohua (Mon, 12 Feb 2018 12:37:48 GMT): baohua (Mon, 12 Feb 2018 13:35:30 GMT): tongli (Mon, 12 Feb 2018 14:27:22 GMT): tongli (Mon, 12 Feb 2018 14:27:34 GMT): baohua (Mon, 12 Feb 2018 14:28:04 GMT): baohua (Mon, 12 Feb 2018 14:28:17 GMT): tongli (Mon, 12 Feb 2018 14:28:46 GMT): tongli (Mon, 12 Feb 2018 14:28:51 GMT): baohua (Mon, 12 Feb 2018 14:28:53 GMT): baohua (Mon, 12 Feb 2018 14:29:04 GMT): baohua (Mon, 12 Feb 2018 14:29:36 GMT): tongli (Mon, 12 Feb 2018 14:30:14 GMT): tongli (Mon, 12 Feb 2018 14:30:56 GMT): tongli (Mon, 12 Feb 2018 14:31:27 GMT): baohua (Mon, 12 Feb 2018 14:32:01 GMT): tongli (Mon, 12 Feb 2018 14:32:32 GMT): tongli (Mon, 12 Feb 2018 14:32:39 GMT): baohua (Mon, 12 Feb 2018 14:33:19 GMT): tongli (Mon, 12 Feb 2018 14:34:35 GMT): baohua (Mon, 12 Feb 2018 14:34:51 GMT): baohua (Mon, 12 Feb 2018 14:35:07 GMT): tongli (Mon, 12 Feb 2018 14:35:10 GMT): tongli (Mon, 12 Feb 2018 14:35:30 GMT): tongli (Mon, 12 Feb 2018 14:35:42 GMT): baohua (Mon, 12 Feb 2018 14:36:06 GMT): baohua (Mon, 12 Feb 2018 14:36:27 GMT): baohua (Mon, 12 Feb 2018 14:36:53 GMT): baohua (Mon, 12 Feb 2018 14:36:53 GMT): tongli (Mon, 12 Feb 2018 14:37:33 GMT): tongli (Mon, 12 Feb 2018 14:37:53 GMT): tongli (Mon, 12 Feb 2018 14:38:30 GMT): baohua (Mon, 12 Feb 2018 14:40:14 GMT): baohua (Mon, 12 Feb 2018 14:40:51 GMT): tongli (Mon, 12 Feb 2018 14:40:52 GMT): baohua (Mon, 12 Feb 2018 14:41:14 GMT): baohua (Mon, 12 Feb 2018 14:41:44 GMT): tongli (Mon, 12 Feb 2018 14:43:01 GMT): baohua (Mon, 12 Feb 2018 14:46:45 GMT): tongli (Mon, 12 Feb 2018 14:49:15 GMT): tongli (Mon, 12 Feb 2018 14:49:38 GMT): baohua (Mon, 12 Feb 2018 14:49:56 GMT): baohua (Mon, 12 Feb 2018 14:50:23 GMT): baohua (Mon, 12 Feb 2018 14:50:32 GMT): tongli (Mon, 12 Feb 2018 14:50:43 GMT): hightall (Mon, 12 Feb 2018 15:04:22 GMT): baohua (Mon, 12 Feb 2018 15:13:34 GMT): tongli (Mon, 12 Feb 2018 17:01:56 GMT): krabradosty (Mon, 12 Feb 2018 21:36:30 GMT): krabradosty (Mon, 12 Feb 2018 21:37:47 GMT): krabradosty (Mon, 12 Feb 2018 21:37:47 GMT): bleggett (Mon, 12 Feb 2018 21:46:40 GMT): bleggett (Mon, 12 Feb 2018 21:46:40 GMT): livespotty (Mon, 12 Feb 2018 21:46:57 GMT): krabradosty (Mon, 12 Feb 2018 21:47:23 GMT): krabradosty (Mon, 12 Feb 2018 21:56:16 GMT): baohua (Tue, 13 Feb 2018 02:07:34 GMT): Khan92 (Tue, 13 Feb 2018 03:09:23 GMT): Khan92 (Tue, 13 Feb 2018 03:09:51 GMT): Khan92 (Tue, 13 Feb 2018 03:10:11 GMT): Khan92 (Tue, 13 Feb 2018 03:10:22 GMT): Khan92 (Tue, 13 Feb 2018 03:10:47 GMT): Khan92 (Tue, 13 Feb 2018 03:10:49 GMT): tongli (Tue, 13 Feb 2018 03:48:37 GMT): baohua (Tue, 13 Feb 2018 06:29:52 GMT): zsmartc (Tue, 13 Feb 2018 09:18:51 GMT): Amir (Tue, 13 Feb 2018 09:24:21 GMT): Amir (Tue, 13 Feb 2018 09:27:04 GMT): baohua (Tue, 13 Feb 2018 09:27:16 GMT): zsmartc (Tue, 13 Feb 2018 09:38:45 GMT): tongli (Tue, 13 Feb 2018 14:38:23 GMT): tongli (Tue, 13 Feb 2018 15:13:53 GMT): FiratSertgoz (Tue, 13 Feb 2018 15:23:07 GMT): tongli (Tue, 13 Feb 2018 15:41:50 GMT): tongli (Tue, 13 Feb 2018 15:42:24 GMT): vladyslavmunin (Tue, 13 Feb 2018 18:08:02 GMT): vladyslavmunin (Tue, 13 Feb 2018 18:09:08 GMT): vladyslavmunin (Tue, 13 Feb 2018 18:09:58 GMT): tongli (Tue, 13 Feb 2018 18:10:51 GMT): tongli (Tue, 13 Feb 2018 18:10:55 GMT): RodneySa (Tue, 13 Feb 2018 19:00:18 GMT): evandro.franco (Tue, 13 Feb 2018 19:05:47 GMT): Amir (Tue, 13 Feb 2018 19:35:27 GMT): Amir (Tue, 13 Feb 2018 19:36:33 GMT): krabradosty (Tue, 13 Feb 2018 19:39:02 GMT): tongli (Tue, 13 Feb 2018 20:51:04 GMT): krabradosty (Tue, 13 Feb 2018 20:52:15 GMT): tongli (Tue, 13 Feb 2018 20:53:00 GMT): tongli (Tue, 13 Feb 2018 20:53:15 GMT): tongli (Tue, 13 Feb 2018 20:53:28 GMT): tongli (Tue, 13 Feb 2018 20:53:59 GMT): krabradosty (Tue, 13 Feb 2018 20:59:31 GMT): krabradosty (Tue, 13 Feb 2018 21:00:50 GMT): bleggett (Tue, 13 Feb 2018 22:21:39 GMT): bleggett (Tue, 13 Feb 2018 22:21:39 GMT): bleggett (Tue, 13 Feb 2018 22:21:39 GMT): bleggett (Tue, 13 Feb 2018 22:21:39 GMT): bleggett (Tue, 13 Feb 2018 22:21:39 GMT): bleggett (Tue, 13 Feb 2018 22:21:39 GMT): bleggett (Tue, 13 Feb 2018 22:21:39 GMT): bleggett (Tue, 13 Feb 2018 22:23:41 GMT): baohua (Wed, 14 Feb 2018 02:11:08 GMT): baohua (Wed, 14 Feb 2018 08:32:26 GMT): FiratSertgoz (Wed, 14 Feb 2018 09:19:57 GMT): AshishMishra 1 (Wed, 14 Feb 2018 14:06:28 GMT): AshishMishra 1 (Wed, 14 Feb 2018 14:07:07 GMT): baohua (Thu, 15 Feb 2018 03:53:10 GMT): AshishMishra 1 (Thu, 15 Feb 2018 06:50:30 GMT): baohua (Thu, 15 Feb 2018 06:52:35 GMT): baohua (Thu, 15 Feb 2018 06:52:35 GMT): AshishMishra 1 (Thu, 15 Feb 2018 06:58:19 GMT): baohua (Thu, 15 Feb 2018 06:59:15 GMT): DarshanBc (Thu, 15 Feb 2018 06:59:35 GMT): DarshanBc (Thu, 15 Feb 2018 06:59:35 GMT): baohua (Thu, 15 Feb 2018 07:00:19 GMT): DarshanBc (Thu, 15 Feb 2018 07:01:42 GMT): DarshanBc (Thu, 15 Feb 2018 07:02:08 GMT): baohua (Thu, 15 Feb 2018 07:02:14 GMT): DarshanBc (Thu, 15 Feb 2018 07:02:41 GMT): DarshanBc (Thu, 15 Feb 2018 07:46:01 GMT): baohua (Thu, 15 Feb 2018 07:47:24 GMT): DarshanBc (Thu, 15 Feb 2018 07:49:12 GMT): DarshanBc (Thu, 15 Feb 2018 07:49:47 GMT): DarshanBc (Thu, 15 Feb 2018 07:49:47 GMT): DarshanBc (Thu, 15 Feb 2018 07:49:47 GMT): DarshanBc (Thu, 15 Feb 2018 09:14:15 GMT): FiratSertgoz (Thu, 15 Feb 2018 09:26:38 GMT): dinesh.rivankar (Thu, 15 Feb 2018 10:30:25 GMT): dinesh.rivankar (Thu, 15 Feb 2018 10:34:03 GMT): baohua (Thu, 15 Feb 2018 10:34:33 GMT): DarshanBc (Thu, 15 Feb 2018 10:39:59 GMT): baohua (Thu, 15 Feb 2018 11:41:38 GMT): DarshanBc (Thu, 15 Feb 2018 12:26:55 GMT): DarshanBc (Thu, 15 Feb 2018 12:27:43 GMT): DarshanBc (Thu, 15 Feb 2018 12:31:24 GMT): DarshanBc (Thu, 15 Feb 2018 12:39:00 GMT): DarshanBc (Thu, 15 Feb 2018 12:41:14 GMT): bleggett (Thu, 15 Feb 2018 20:02:30 GMT): tongli (Thu, 15 Feb 2018 22:16:09 GMT): sampath06 (Fri, 16 Feb 2018 01:10:45 GMT): DarshanBc (Fri, 16 Feb 2018 06:03:43 GMT): tongli (Fri, 16 Feb 2018 06:30:27 GMT): tongli (Fri, 16 Feb 2018 06:30:43 GMT): tongli (Fri, 16 Feb 2018 06:31:41 GMT): tongli (Fri, 16 Feb 2018 06:32:20 GMT): tongli (Fri, 16 Feb 2018 06:32:47 GMT): dinesh.rivankar (Fri, 16 Feb 2018 07:05:15 GMT): dinesh.rivankar (Fri, 16 Feb 2018 07:05:48 GMT): FiratSertgoz (Fri, 16 Feb 2018 09:14:45 GMT): DarshanBc (Fri, 16 Feb 2018 10:01:16 GMT): DarshanBc (Fri, 16 Feb 2018 10:01:16 GMT): fabcan (Fri, 16 Feb 2018 11:04:12 GMT): tongli (Fri, 16 Feb 2018 18:22:32 GMT): baohua (Sat, 17 Feb 2018 02:23:07 GMT): baohua (Sat, 17 Feb 2018 02:23:07 GMT): ublubu (Sat, 17 Feb 2018 19:43:48 GMT): Amir (Sun, 18 Feb 2018 22:22:06 GMT): Amir (Sun, 18 Feb 2018 22:24:47 GMT): Amir (Sun, 18 Feb 2018 22:39:09 GMT): dinesh.rivankar (Mon, 19 Feb 2018 05:06:59 GMT): FiratSertgoz (Mon, 19 Feb 2018 08:51:45 GMT): tongli (Mon, 19 Feb 2018 16:31:22 GMT): ublubu (Mon, 19 Feb 2018 18:19:09 GMT): ublubu (Mon, 19 Feb 2018 18:19:53 GMT): ublubu (Mon, 19 Feb 2018 18:20:36 GMT): ublubu (Mon, 19 Feb 2018 18:26:21 GMT): baohua (Tue, 20 Feb 2018 07:03:45 GMT): jack21 (Tue, 20 Feb 2018 09:29:59 GMT): sampath06 (Tue, 20 Feb 2018 12:13:12 GMT): sampath06 (Tue, 20 Feb 2018 16:54:18 GMT): sampath06 (Wed, 21 Feb 2018 00:42:13 GMT): Menniti (Wed, 21 Feb 2018 03:11:16 GMT): toddinpal (Wed, 21 Feb 2018 04:57:23 GMT): toddinpal (Wed, 21 Feb 2018 04:58:05 GMT): toddinpal (Wed, 21 Feb 2018 04:58:16 GMT): harsha (Wed, 21 Feb 2018 06:54:19 GMT): AshishMishra 1 (Wed, 21 Feb 2018 06:58:51 GMT): sbh (Wed, 21 Feb 2018 08:26:46 GMT): tongli (Wed, 21 Feb 2018 11:12:53 GMT): tongli (Wed, 21 Feb 2018 11:13:37 GMT): Menniti (Wed, 21 Feb 2018 12:16:03 GMT): Menniti (Wed, 21 Feb 2018 12:16:24 GMT): Menniti (Wed, 21 Feb 2018 12:16:37 GMT): Menniti (Wed, 21 Feb 2018 12:16:51 GMT): tongli (Wed, 21 Feb 2018 12:31:02 GMT): Menniti (Wed, 21 Feb 2018 12:31:16 GMT): tongli (Wed, 21 Feb 2018 12:32:00 GMT): tongli (Wed, 21 Feb 2018 12:32:11 GMT): tongli (Wed, 21 Feb 2018 12:32:28 GMT): Menniti (Wed, 21 Feb 2018 12:43:45 GMT): Menniti (Wed, 21 Feb 2018 12:44:05 GMT): Menniti (Wed, 21 Feb 2018 12:44:08 GMT): tongli (Wed, 21 Feb 2018 12:44:08 GMT): tongli (Wed, 21 Feb 2018 12:44:19 GMT): tongli (Wed, 21 Feb 2018 12:45:01 GMT): tongli (Wed, 21 Feb 2018 12:45:19 GMT): Menniti (Wed, 21 Feb 2018 12:45:52 GMT): Menniti (Wed, 21 Feb 2018 12:46:44 GMT): Menniti (Wed, 21 Feb 2018 12:47:05 GMT): baohua (Wed, 21 Feb 2018 12:47:52 GMT): baohua (Wed, 21 Feb 2018 12:48:01 GMT): baohua (Wed, 21 Feb 2018 12:48:25 GMT): tongli (Wed, 21 Feb 2018 12:48:32 GMT): tongli (Wed, 21 Feb 2018 12:49:13 GMT): baohua (Wed, 21 Feb 2018 12:49:23 GMT): baohua (Wed, 21 Feb 2018 12:49:54 GMT): baohua (Wed, 21 Feb 2018 12:50:15 GMT): Menniti (Wed, 21 Feb 2018 12:50:30 GMT): Menniti (Wed, 21 Feb 2018 12:50:45 GMT): Menniti (Wed, 21 Feb 2018 12:50:57 GMT): tongli (Wed, 21 Feb 2018 12:51:40 GMT): tongli (Wed, 21 Feb 2018 12:51:57 GMT): tongli (Wed, 21 Feb 2018 12:52:12 GMT): tongli (Wed, 21 Feb 2018 12:52:22 GMT): baohua (Wed, 21 Feb 2018 12:53:39 GMT): baohua (Wed, 21 Feb 2018 12:53:39 GMT): tongli (Wed, 21 Feb 2018 12:54:13 GMT): Menniti (Wed, 21 Feb 2018 13:33:14 GMT): Menniti (Wed, 21 Feb 2018 13:33:26 GMT): toddinpal (Wed, 21 Feb 2018 15:00:05 GMT): harsha (Wed, 21 Feb 2018 15:18:57 GMT): harsha (Wed, 21 Feb 2018 15:19:31 GMT): toddinpal (Wed, 21 Feb 2018 15:50:28 GMT): Amir (Wed, 21 Feb 2018 19:13:12 GMT): Amir (Wed, 21 Feb 2018 19:16:46 GMT): Amir (Wed, 21 Feb 2018 19:16:46 GMT): Amir (Wed, 21 Feb 2018 19:16:46 GMT): Amir (Wed, 21 Feb 2018 19:20:12 GMT): toddinpal (Wed, 21 Feb 2018 19:33:06 GMT): Amir (Wed, 21 Feb 2018 19:34:48 GMT): toddinpal (Wed, 21 Feb 2018 19:37:02 GMT): Amir (Wed, 21 Feb 2018 21:28:02 GMT): Amir (Wed, 21 Feb 2018 21:28:02 GMT): Amir (Wed, 21 Feb 2018 21:29:39 GMT): Er1ck (Wed, 21 Feb 2018 21:45:17 GMT): baohua (Thu, 22 Feb 2018 01:49:25 GMT): Amir (Thu, 22 Feb 2018 02:45:09 GMT): baohua (Thu, 22 Feb 2018 03:09:31 GMT): baohua (Thu, 22 Feb 2018 03:09:31 GMT): paul.sitoh (Thu, 22 Feb 2018 09:32:29 GMT): krabradosty (Thu, 22 Feb 2018 10:52:09 GMT): baohua (Thu, 22 Feb 2018 13:44:35 GMT): paul.sitoh (Thu, 22 Feb 2018 13:46:11 GMT): baohua (Thu, 22 Feb 2018 14:10:19 GMT): paul.sitoh (Thu, 22 Feb 2018 14:30:56 GMT): paul.sitoh (Thu, 22 Feb 2018 14:31:51 GMT): paul.sitoh (Thu, 22 Feb 2018 14:32:47 GMT): Amir (Thu, 22 Feb 2018 19:03:40 GMT): Amir (Thu, 22 Feb 2018 19:03:46 GMT): Amir (Thu, 22 Feb 2018 20:44:50 GMT): bleggett (Thu, 22 Feb 2018 21:50:27 GMT): baohua (Fri, 23 Feb 2018 01:39:39 GMT): paul.sitoh (Fri, 23 Feb 2018 10:14:10 GMT): paul.sitoh (Fri, 23 Feb 2018 10:14:10 GMT): paul.sitoh (Fri, 23 Feb 2018 10:14:10 GMT): paul.sitoh (Fri, 23 Feb 2018 10:19:17 GMT): paul.sitoh (Fri, 23 Feb 2018 10:20:56 GMT): paul.sitoh (Fri, 23 Feb 2018 10:20:56 GMT): paul.sitoh (Fri, 23 Feb 2018 10:20:56 GMT): bleggett (Fri, 23 Feb 2018 16:38:00 GMT): bleggett (Fri, 23 Feb 2018 16:38:00 GMT): bleggett (Fri, 23 Feb 2018 16:38:00 GMT): Menniti (Sat, 24 Feb 2018 01:24:04 GMT): Menniti (Sat, 24 Feb 2018 01:24:48 GMT): Menniti (Sat, 24 Feb 2018 01:25:12 GMT): baohua (Sat, 24 Feb 2018 01:30:05 GMT): baohua (Sat, 24 Feb 2018 01:30:39 GMT): Menniti (Sat, 24 Feb 2018 01:31:01 GMT): Menniti (Sat, 24 Feb 2018 01:31:23 GMT): Menniti (Sat, 24 Feb 2018 01:31:25 GMT): Menniti (Sat, 24 Feb 2018 01:31:28 GMT): Menniti (Sat, 24 Feb 2018 01:32:07 GMT): Menniti (Sat, 24 Feb 2018 01:32:19 GMT): Menniti (Sat, 24 Feb 2018 01:33:17 GMT): baohua (Sat, 24 Feb 2018 01:39:39 GMT): Menniti (Sat, 24 Feb 2018 01:57:43 GMT): baohua (Sat, 24 Feb 2018 01:58:19 GMT): baohua (Sat, 24 Feb 2018 01:58:39 GMT): Menniti (Sat, 24 Feb 2018 01:59:09 GMT): Menniti (Sat, 24 Feb 2018 02:14:03 GMT): Menniti (Sat, 24 Feb 2018 02:14:07 GMT): Menniti (Sat, 24 Feb 2018 02:14:55 GMT): Menniti (Sat, 24 Feb 2018 03:00:51 GMT): Menniti (Sat, 24 Feb 2018 03:02:19 GMT): Menniti (Sat, 24 Feb 2018 03:05:02 GMT): baohua (Sat, 24 Feb 2018 06:04:10 GMT): baohua (Sat, 24 Feb 2018 06:04:42 GMT): baohua (Sat, 24 Feb 2018 06:04:42 GMT): baohua (Sat, 24 Feb 2018 06:58:26 GMT): toddinpal (Sat, 24 Feb 2018 14:53:13 GMT): Menniti (Sat, 24 Feb 2018 16:00:51 GMT): baohua (Sun, 25 Feb 2018 01:51:21 GMT): baohua (Sun, 25 Feb 2018 01:52:30 GMT): jeffprestes (Sun, 25 Feb 2018 16:12:36 GMT): nolimitkun (Sun, 25 Feb 2018 20:38:04 GMT): Amir (Sun, 25 Feb 2018 22:25:08 GMT): kakuzu (Sun, 25 Feb 2018 23:50:51 GMT): Dilip 25 (Mon, 26 Feb 2018 00:55:26 GMT): Dilip 25 (Mon, 26 Feb 2018 00:55:35 GMT): Dilip 25 (Mon, 26 Feb 2018 01:02:24 GMT): Dilip 25 (Mon, 26 Feb 2018 01:02:24 GMT): Dilip 25 (Mon, 26 Feb 2018 01:03:22 GMT): Dilip 25 (Mon, 26 Feb 2018 01:03:42 GMT): Amir (Mon, 26 Feb 2018 01:11:07 GMT): Dilip 25 (Mon, 26 Feb 2018 01:39:48 GMT): Dilip 25 (Mon, 26 Feb 2018 01:41:10 GMT): Dilip 25 (Mon, 26 Feb 2018 01:41:40 GMT): Dilip 25 (Mon, 26 Feb 2018 01:46:05 GMT): Dilip 25 (Mon, 26 Feb 2018 01:46:05 GMT): Dilip 25 (Mon, 26 Feb 2018 01:55:05 GMT): Amir (Mon, 26 Feb 2018 02:29:39 GMT): Dilip 25 (Mon, 26 Feb 2018 02:31:44 GMT): Dilip 25 (Mon, 26 Feb 2018 02:32:04 GMT): bittom (Mon, 26 Feb 2018 03:20:55 GMT): tongli (Mon, 26 Feb 2018 03:45:44 GMT): Dilip 25 (Mon, 26 Feb 2018 14:54:21 GMT): Dilip 25 (Mon, 26 Feb 2018 14:54:41 GMT): Dilip 25 (Mon, 26 Feb 2018 14:54:41 GMT): Dilip 25 (Mon, 26 Feb 2018 14:55:45 GMT): Dilip 25 (Mon, 26 Feb 2018 14:56:06 GMT): Dilip 25 (Mon, 26 Feb 2018 14:58:53 GMT): tongli (Mon, 26 Feb 2018 16:44:31 GMT): tongli (Mon, 26 Feb 2018 16:45:02 GMT): Dilip 25 (Mon, 26 Feb 2018 17:33:46 GMT): Dilip 25 (Mon, 26 Feb 2018 17:35:06 GMT): tongli (Mon, 26 Feb 2018 17:52:09 GMT): tongli (Mon, 26 Feb 2018 17:52:51 GMT): Dilip 25 (Mon, 26 Feb 2018 18:01:42 GMT): tongli (Mon, 26 Feb 2018 18:18:44 GMT): Dilip 25 (Mon, 26 Feb 2018 18:26:12 GMT): Dilip 25 (Mon, 26 Feb 2018 18:26:24 GMT): Dilip 25 (Mon, 26 Feb 2018 18:26:42 GMT): Dilip 25 (Mon, 26 Feb 2018 18:28:35 GMT): Dilip 25 (Mon, 26 Feb 2018 18:28:57 GMT): Dilip 25 (Mon, 26 Feb 2018 18:30:06 GMT): Dilip 25 (Mon, 26 Feb 2018 18:31:03 GMT): Dilip 25 (Mon, 26 Feb 2018 18:31:44 GMT): Dilip 25 (Mon, 26 Feb 2018 18:31:55 GMT): tongli (Mon, 26 Feb 2018 18:32:40 GMT): Dilip 25 (Mon, 26 Feb 2018 18:32:59 GMT): tongli (Mon, 26 Feb 2018 18:33:06 GMT): Dilip 25 (Mon, 26 Feb 2018 18:33:24 GMT): Dilip 25 (Mon, 26 Feb 2018 18:33:33 GMT): tongli (Mon, 26 Feb 2018 18:33:37 GMT): tongli (Mon, 26 Feb 2018 18:33:56 GMT): Dilip 25 (Mon, 26 Feb 2018 18:34:13 GMT): Dilip 25 (Mon, 26 Feb 2018 18:34:33 GMT): tongli (Mon, 26 Feb 2018 18:34:39 GMT): tongli (Mon, 26 Feb 2018 18:34:49 GMT): tongli (Mon, 26 Feb 2018 18:34:59 GMT): Dilip 25 (Mon, 26 Feb 2018 18:35:05 GMT): tongli (Mon, 26 Feb 2018 18:35:05 GMT): Dilip 25 (Mon, 26 Feb 2018 18:35:30 GMT): tongli (Mon, 26 Feb 2018 18:35:44 GMT): tongli (Mon, 26 Feb 2018 18:35:47 GMT): tongli (Mon, 26 Feb 2018 18:36:05 GMT): Dilip 25 (Mon, 26 Feb 2018 18:36:30 GMT): tongli (Mon, 26 Feb 2018 18:36:32 GMT): Dilip 25 (Mon, 26 Feb 2018 18:36:43 GMT): tongli (Mon, 26 Feb 2018 18:37:49 GMT): tongli (Mon, 26 Feb 2018 18:37:52 GMT): tongli (Mon, 26 Feb 2018 18:38:23 GMT): tongli (Mon, 26 Feb 2018 18:39:05 GMT): tongli (Mon, 26 Feb 2018 18:39:24 GMT): tongli (Mon, 26 Feb 2018 18:39:39 GMT): tongli (Mon, 26 Feb 2018 18:39:58 GMT): tongli (Mon, 26 Feb 2018 18:40:05 GMT): tongli (Mon, 26 Feb 2018 18:40:14 GMT): tongli (Mon, 26 Feb 2018 18:40:21 GMT): Dilip 25 (Mon, 26 Feb 2018 18:40:43 GMT): Dilip 25 (Mon, 26 Feb 2018 18:40:54 GMT): tongli (Mon, 26 Feb 2018 18:41:04 GMT): Dilip 25 (Mon, 26 Feb 2018 18:41:13 GMT): tongli (Mon, 26 Feb 2018 18:41:31 GMT): tongli (Mon, 26 Feb 2018 18:41:59 GMT): Dilip 25 (Mon, 26 Feb 2018 18:42:04 GMT): tongli (Mon, 26 Feb 2018 18:42:35 GMT): tongli (Mon, 26 Feb 2018 18:42:39 GMT): Dilip 25 (Mon, 26 Feb 2018 18:42:40 GMT): tongli (Mon, 26 Feb 2018 18:43:16 GMT): tongli (Mon, 26 Feb 2018 18:43:27 GMT): tongli (Mon, 26 Feb 2018 18:43:37 GMT): Dilip 25 (Mon, 26 Feb 2018 18:43:53 GMT): Dilip 25 (Mon, 26 Feb 2018 18:44:00 GMT): Dilip 25 (Mon, 26 Feb 2018 18:44:17 GMT): tongli (Mon, 26 Feb 2018 18:44:19 GMT): tongli (Mon, 26 Feb 2018 18:44:59 GMT): Dilip 25 (Mon, 26 Feb 2018 18:45:58 GMT): Dilip 25 (Mon, 26 Feb 2018 18:46:09 GMT): tongli (Mon, 26 Feb 2018 18:49:07 GMT): tongli (Mon, 26 Feb 2018 18:49:18 GMT): tongli (Mon, 26 Feb 2018 18:49:47 GMT): Dilip 25 (Mon, 26 Feb 2018 18:51:12 GMT): Dilip 25 (Mon, 26 Feb 2018 18:51:21 GMT): tongli (Mon, 26 Feb 2018 18:51:57 GMT): tongli (Mon, 26 Feb 2018 18:52:07 GMT): tongli (Mon, 26 Feb 2018 18:52:31 GMT): Dilip 25 (Mon, 26 Feb 2018 19:29:30 GMT): Dilip 25 (Mon, 26 Feb 2018 19:29:40 GMT): Dilip 25 (Mon, 26 Feb 2018 19:29:42 GMT): Dilip 25 (Mon, 26 Feb 2018 19:30:23 GMT): Dilip 25 (Mon, 26 Feb 2018 19:30:36 GMT): Dilip 25 (Mon, 26 Feb 2018 19:31:17 GMT): Dilip 25 (Mon, 26 Feb 2018 19:31:31 GMT): Dilip 25 (Mon, 26 Feb 2018 19:32:27 GMT): tongli (Mon, 26 Feb 2018 19:34:03 GMT): tongli (Mon, 26 Feb 2018 19:34:28 GMT): tongli (Mon, 26 Feb 2018 19:35:00 GMT): tongli (Mon, 26 Feb 2018 19:35:04 GMT): tongli (Mon, 26 Feb 2018 19:35:22 GMT): tongli (Mon, 26 Feb 2018 19:35:43 GMT): tongli (Mon, 26 Feb 2018 19:35:58 GMT): tongli (Mon, 26 Feb 2018 19:36:11 GMT): Dilip 25 (Mon, 26 Feb 2018 19:39:56 GMT): Dilip 25 (Mon, 26 Feb 2018 19:39:59 GMT): tongli (Mon, 26 Feb 2018 19:45:34 GMT): tongli (Mon, 26 Feb 2018 19:45:47 GMT): tongli (Mon, 26 Feb 2018 19:46:12 GMT): tongli (Mon, 26 Feb 2018 19:46:20 GMT): Dilip 25 (Mon, 26 Feb 2018 19:47:36 GMT): Dilip 25 (Mon, 26 Feb 2018 19:47:43 GMT): tongli (Mon, 26 Feb 2018 19:50:55 GMT): Dilip 25 (Mon, 26 Feb 2018 19:51:14 GMT): Dilip 25 (Mon, 26 Feb 2018 19:51:51 GMT): tongli (Mon, 26 Feb 2018 19:52:05 GMT): tongli (Mon, 26 Feb 2018 19:52:19 GMT): Dilip 25 (Mon, 26 Feb 2018 19:52:28 GMT): tongli (Mon, 26 Feb 2018 19:52:38 GMT): Dilip 25 (Mon, 26 Feb 2018 19:52:38 GMT): Dilip 25 (Mon, 26 Feb 2018 19:52:47 GMT): tongli (Mon, 26 Feb 2018 19:53:34 GMT): tongli (Mon, 26 Feb 2018 19:53:58 GMT): tongli (Mon, 26 Feb 2018 19:54:36 GMT): tongli (Mon, 26 Feb 2018 19:54:43 GMT): Dilip 25 (Mon, 26 Feb 2018 20:35:25 GMT): Dilip 25 (Mon, 26 Feb 2018 20:35:43 GMT): Dilip 25 (Mon, 26 Feb 2018 20:35:49 GMT): Dilip 25 (Mon, 26 Feb 2018 20:37:08 GMT): tongli (Mon, 26 Feb 2018 20:40:00 GMT): tongli (Mon, 26 Feb 2018 20:40:11 GMT): Dilip 25 (Mon, 26 Feb 2018 20:50:59 GMT): Dilip 25 (Mon, 26 Feb 2018 20:51:09 GMT): Dilip 25 (Mon, 26 Feb 2018 20:53:16 GMT): Dilip 25 (Mon, 26 Feb 2018 20:56:46 GMT): Dilip 25 (Mon, 26 Feb 2018 21:15:03 GMT): tongli (Mon, 26 Feb 2018 22:38:29 GMT): Dilip 25 (Mon, 26 Feb 2018 22:59:51 GMT): Dilip 25 (Mon, 26 Feb 2018 23:00:06 GMT): Dilip 25 (Mon, 26 Feb 2018 23:00:13 GMT): Dilip 25 (Mon, 26 Feb 2018 23:00:50 GMT): Dilip 25 (Mon, 26 Feb 2018 23:01:11 GMT): tongli (Tue, 27 Feb 2018 01:02:12 GMT): tongli (Tue, 27 Feb 2018 01:02:30 GMT): Dilip 25 (Tue, 27 Feb 2018 01:23:24 GMT): Dilip 25 (Tue, 27 Feb 2018 01:23:33 GMT): Dilip 25 (Tue, 27 Feb 2018 01:24:00 GMT): Randyshu2018 (Tue, 27 Feb 2018 02:16:12 GMT): tongli (Tue, 27 Feb 2018 05:21:27 GMT): tongli (Tue, 27 Feb 2018 05:21:38 GMT): Dilip 25 (Tue, 27 Feb 2018 05:27:28 GMT): tongli (Tue, 27 Feb 2018 05:32:42 GMT): Dilip 25 (Tue, 27 Feb 2018 05:34:29 GMT): Dilip 25 (Tue, 27 Feb 2018 05:34:29 GMT): DarshanBc (Tue, 27 Feb 2018 06:22:03 GMT): baohua (Tue, 27 Feb 2018 06:24:37 GMT): DarshanBc (Tue, 27 Feb 2018 06:33:59 GMT): Luke_Chen (Tue, 27 Feb 2018 09:00:24 GMT): baohua (Tue, 27 Feb 2018 09:00:45 GMT): Luke_Chen (Tue, 27 Feb 2018 09:00:53 GMT): baohua (Tue, 27 Feb 2018 09:01:30 GMT): tongli (Tue, 27 Feb 2018 12:30:47 GMT): tongli (Tue, 27 Feb 2018 12:49:52 GMT): VikasJakhar (Tue, 27 Feb 2018 20:04:14 GMT): baohua (Wed, 28 Feb 2018 00:45:03 GMT): tongli (Wed, 28 Feb 2018 00:45:48 GMT): tongli (Wed, 28 Feb 2018 00:50:38 GMT): baohua (Wed, 28 Feb 2018 01:00:55 GMT): tongli (Wed, 28 Feb 2018 01:14:43 GMT): tongli (Wed, 28 Feb 2018 01:17:04 GMT): baohua (Wed, 28 Feb 2018 01:29:51 GMT): DarshanBc (Wed, 28 Feb 2018 06:42:11 GMT): DarshanBc (Wed, 28 Feb 2018 06:45:49 GMT): DarshanBc (Wed, 28 Feb 2018 06:46:14 GMT): baohua (Wed, 28 Feb 2018 07:15:33 GMT): DarshanBc (Wed, 28 Feb 2018 07:16:44 GMT): DarshanBc (Wed, 28 Feb 2018 07:42:49 GMT): DarshanBc (Wed, 28 Feb 2018 07:42:49 GMT): jeffguo (Wed, 28 Feb 2018 09:22:05 GMT): chandrakanthm (Wed, 28 Feb 2018 09:51:29 GMT): tongli (Wed, 28 Feb 2018 19:18:37 GMT): tongli (Wed, 28 Feb 2018 19:19:13 GMT): tongli (Wed, 28 Feb 2018 19:19:59 GMT): arunmitteam (Wed, 28 Feb 2018 23:09:18 GMT): arunmitteam (Wed, 28 Feb 2018 23:10:26 GMT): arunmitteam (Wed, 28 Feb 2018 23:12:39 GMT): bleggett (Thu, 01 Mar 2018 15:51:45 GMT): SKDHANUKA (Thu, 01 Mar 2018 17:20:43 GMT): arunmitteam (Thu, 01 Mar 2018 19:25:10 GMT): Amir (Thu, 01 Mar 2018 19:37:00 GMT): bleggett (Thu, 01 Mar 2018 20:13:44 GMT): Amir (Thu, 01 Mar 2018 21:36:10 GMT): bleggett (Thu, 01 Mar 2018 22:20:40 GMT): bleggett (Thu, 01 Mar 2018 22:20:40 GMT): lujiang (Fri, 02 Mar 2018 01:25:46 GMT): baohua (Fri, 02 Mar 2018 05:12:16 GMT): jaswanth (Fri, 02 Mar 2018 05:40:01 GMT): baohua (Fri, 02 Mar 2018 12:08:04 GMT): hightall (Fri, 02 Mar 2018 14:20:42 GMT): baohua (Fri, 02 Mar 2018 14:42:44 GMT): alexvicegrab (Fri, 02 Mar 2018 15:41:46 GMT): pauldev (Fri, 02 Mar 2018 16:25:03 GMT): tongli (Fri, 02 Mar 2018 18:32:02 GMT): tongli (Fri, 02 Mar 2018 18:32:29 GMT): arunmitteam (Fri, 02 Mar 2018 20:12:06 GMT): vmuravyev (Sat, 03 Mar 2018 11:38:35 GMT): ashishapy (Sat, 03 Mar 2018 19:37:14 GMT): ashishapy (Sat, 03 Mar 2018 19:41:41 GMT): ashishapy (Sat, 03 Mar 2018 19:42:55 GMT): ashishapy (Sat, 03 Mar 2018 19:43:34 GMT): tongli (Sat, 03 Mar 2018 23:32:58 GMT): hightall (Sun, 04 Mar 2018 06:36:46 GMT): hightall (Sun, 04 Mar 2018 08:18:18 GMT): baohua (Sun, 04 Mar 2018 08:31:43 GMT): hightall (Sun, 04 Mar 2018 13:06:21 GMT): baohua (Sun, 04 Mar 2018 13:13:53 GMT): arunmitteam (Sun, 04 Mar 2018 17:17:26 GMT): arunmitteam (Sun, 04 Mar 2018 17:21:56 GMT): arunmitteam (Sun, 04 Mar 2018 17:22:21 GMT): ChainMind (Sun, 04 Mar 2018 20:55:24 GMT): CaryMa (Mon, 05 Mar 2018 00:42:14 GMT): baohua (Mon, 05 Mar 2018 03:29:57 GMT): harsha (Mon, 05 Mar 2018 15:04:20 GMT): harsha (Mon, 05 Mar 2018 15:04:20 GMT): jtclark (Mon, 05 Mar 2018 17:54:15 GMT): marek5050 (Mon, 05 Mar 2018 22:57:28 GMT): nolimitkun (Tue, 06 Mar 2018 00:32:44 GMT): nolimitkun (Tue, 06 Mar 2018 00:33:21 GMT): nolimitkun (Tue, 06 Mar 2018 00:33:37 GMT): tongli (Tue, 06 Mar 2018 00:33:48 GMT): tongli (Tue, 06 Mar 2018 00:33:59 GMT): tongli (Tue, 06 Mar 2018 00:34:43 GMT): nolimitkun (Tue, 06 Mar 2018 00:34:47 GMT): tongli (Tue, 06 Mar 2018 00:35:11 GMT): nolimitkun (Tue, 06 Mar 2018 00:36:11 GMT): nolimitkun (Tue, 06 Mar 2018 00:36:38 GMT): tongli (Tue, 06 Mar 2018 00:36:45 GMT): tongli (Tue, 06 Mar 2018 00:37:25 GMT): tongli (Tue, 06 Mar 2018 00:37:47 GMT): nolimitkun (Tue, 06 Mar 2018 00:38:16 GMT): tongli (Tue, 06 Mar 2018 00:39:02 GMT): nolimitkun (Tue, 06 Mar 2018 00:41:32 GMT): tongli (Tue, 06 Mar 2018 00:42:20 GMT): tongli (Tue, 06 Mar 2018 00:42:26 GMT): tongli (Tue, 06 Mar 2018 00:42:54 GMT): nolimitkun (Tue, 06 Mar 2018 00:43:40 GMT): nolimitkun (Tue, 06 Mar 2018 00:43:44 GMT): tongli (Tue, 06 Mar 2018 00:45:09 GMT): tongli (Tue, 06 Mar 2018 00:45:18 GMT): tongli (Tue, 06 Mar 2018 00:45:24 GMT): nolimitkun (Tue, 06 Mar 2018 00:47:37 GMT): nolimitkun (Tue, 06 Mar 2018 00:47:48 GMT): tongli (Tue, 06 Mar 2018 00:50:33 GMT): tongli (Tue, 06 Mar 2018 00:51:03 GMT): QQ-ing (Tue, 06 Mar 2018 01:13:14 GMT): baohua (Tue, 06 Mar 2018 01:32:38 GMT): Amir (Tue, 06 Mar 2018 01:41:12 GMT): baohua (Tue, 06 Mar 2018 03:01:11 GMT): harsha (Tue, 06 Mar 2018 03:06:23 GMT): Amir (Tue, 06 Mar 2018 03:14:14 GMT): magicliang (Tue, 06 Mar 2018 10:23:21 GMT): minfun (Tue, 06 Mar 2018 13:40:41 GMT): tongli (Tue, 06 Mar 2018 14:43:37 GMT): tongli (Tue, 06 Mar 2018 15:00:36 GMT): tongli (Tue, 06 Mar 2018 16:02:50 GMT): mikezaccardo (Tue, 06 Mar 2018 20:13:11 GMT): Biyiklioglu (Tue, 06 Mar 2018 20:48:10 GMT): baohua (Wed, 07 Mar 2018 03:12:47 GMT): magicliang (Wed, 07 Mar 2018 03:13:24 GMT): baohua (Wed, 07 Mar 2018 03:13:33 GMT): tongli (Wed, 07 Mar 2018 03:14:13 GMT): jack21 (Wed, 07 Mar 2018 03:44:36 GMT): Varun2887 (Wed, 07 Mar 2018 04:53:16 GMT): wininani (Wed, 07 Mar 2018 05:45:34 GMT): baohua (Wed, 07 Mar 2018 14:26:25 GMT): jack21 (Wed, 07 Mar 2018 14:27:05 GMT): harsha (Wed, 07 Mar 2018 14:28:35 GMT): baohua (Wed, 07 Mar 2018 14:29:44 GMT): baohua (Wed, 07 Mar 2018 14:30:07 GMT): baohua (Wed, 07 Mar 2018 14:30:35 GMT): harsha (Wed, 07 Mar 2018 15:08:11 GMT): vishwasbalakrishna (Wed, 07 Mar 2018 21:29:33 GMT): baohua (Thu, 08 Mar 2018 02:09:34 GMT): Amir (Thu, 08 Mar 2018 03:28:25 GMT): Amir (Thu, 08 Mar 2018 03:31:02 GMT): tongli (Thu, 08 Mar 2018 03:48:02 GMT): dinesh.rivankar (Thu, 08 Mar 2018 04:25:35 GMT): baohua (Thu, 08 Mar 2018 05:53:09 GMT): baohua (Thu, 08 Mar 2018 05:53:16 GMT): baohua (Thu, 08 Mar 2018 05:54:14 GMT): lucienlu (Thu, 08 Mar 2018 08:19:36 GMT): liujinzhong (Thu, 08 Mar 2018 09:33:26 GMT): liujinzhong (Thu, 08 Mar 2018 09:39:01 GMT): harsha (Thu, 08 Mar 2018 09:40:12 GMT): liujinzhong (Thu, 08 Mar 2018 09:42:51 GMT): liujinzhong (Thu, 08 Mar 2018 09:45:11 GMT): harsha (Thu, 08 Mar 2018 09:48:40 GMT): liujinzhong (Thu, 08 Mar 2018 09:49:32 GMT): SriramaSharma (Thu, 08 Mar 2018 09:51:45 GMT): Amir (Thu, 08 Mar 2018 19:23:40 GMT): baohua (Fri, 09 Mar 2018 02:53:01 GMT): RahulK22 (Fri, 09 Mar 2018 04:28:46 GMT): liujinzhong (Fri, 09 Mar 2018 10:46:29 GMT): Luke_Chen (Fri, 09 Mar 2018 10:51:41 GMT): Luke_Chen (Fri, 09 Mar 2018 10:53:00 GMT): liujinzhong (Fri, 09 Mar 2018 11:53:08 GMT): liujinzhong (Fri, 09 Mar 2018 11:54:50 GMT): Luke_Chen (Fri, 09 Mar 2018 11:55:42 GMT): liujinzhong (Fri, 09 Mar 2018 11:56:22 GMT): liujinzhong (Fri, 09 Mar 2018 11:59:26 GMT): Luke_Chen (Fri, 09 Mar 2018 12:05:33 GMT): Luke_Chen (Fri, 09 Mar 2018 12:05:33 GMT): arunmitteam (Fri, 09 Mar 2018 12:08:24 GMT): arunmitteam (Fri, 09 Mar 2018 12:08:54 GMT): baohua (Fri, 09 Mar 2018 12:09:27 GMT): harsha (Fri, 09 Mar 2018 12:09:46 GMT): harsha (Fri, 09 Mar 2018 13:00:00 GMT): harsha (Fri, 09 Mar 2018 13:00:00 GMT): harsha (Fri, 09 Mar 2018 13:00:00 GMT): baohua (Fri, 09 Mar 2018 13:29:34 GMT): baohua (Fri, 09 Mar 2018 13:41:50 GMT): tongli (Fri, 09 Mar 2018 13:43:14 GMT): JanRzepecki (Fri, 09 Mar 2018 14:03:23 GMT): tongli (Fri, 09 Mar 2018 14:38:30 GMT): tongli (Fri, 09 Mar 2018 14:38:33 GMT): tongli (Fri, 09 Mar 2018 14:39:21 GMT): tongli (Fri, 09 Mar 2018 14:39:30 GMT): baohua (Fri, 09 Mar 2018 14:42:19 GMT): baohua (Fri, 09 Mar 2018 14:42:48 GMT): tongli (Fri, 09 Mar 2018 14:43:06 GMT): tongli (Fri, 09 Mar 2018 14:43:46 GMT): tongli (Fri, 09 Mar 2018 14:44:13 GMT): baohua (Fri, 09 Mar 2018 14:45:17 GMT): baohua (Fri, 09 Mar 2018 14:45:29 GMT): tongli (Fri, 09 Mar 2018 14:45:32 GMT): tongli (Fri, 09 Mar 2018 14:46:08 GMT): tongli (Fri, 09 Mar 2018 14:46:20 GMT): baohua (Fri, 09 Mar 2018 14:46:28 GMT): tongli (Fri, 09 Mar 2018 14:46:35 GMT): tongli (Fri, 09 Mar 2018 14:47:09 GMT): tongli (Fri, 09 Mar 2018 14:47:50 GMT): baohua (Fri, 09 Mar 2018 14:55:23 GMT): baohua (Fri, 09 Mar 2018 14:55:34 GMT): tongli (Fri, 09 Mar 2018 14:57:21 GMT): baohua (Fri, 09 Mar 2018 15:04:44 GMT): Amir (Fri, 09 Mar 2018 18:51:15 GMT): Amir (Sat, 10 Mar 2018 00:17:13 GMT): Amir (Sat, 10 Mar 2018 00:17:13 GMT): baohua (Sat, 10 Mar 2018 00:36:05 GMT): mrtrantuan (Sat, 10 Mar 2018 02:34:45 GMT): ChrisFowers (Sat, 10 Mar 2018 03:52:20 GMT): ChrisFowers (Sat, 10 Mar 2018 03:52:59 GMT): ChrisFowers (Sat, 10 Mar 2018 16:29:40 GMT): ChrisFowers (Sat, 10 Mar 2018 16:29:59 GMT): yopep (Sat, 10 Mar 2018 16:53:32 GMT): maddydev (Sat, 10 Mar 2018 23:29:38 GMT): Luke_Chen (Sun, 11 Mar 2018 02:18:26 GMT): harsha (Sun, 11 Mar 2018 14:06:13 GMT): yopep (Sun, 11 Mar 2018 15:56:00 GMT): tongli (Mon, 12 Mar 2018 01:33:47 GMT): baohua (Mon, 12 Mar 2018 02:22:07 GMT): tongli (Mon, 12 Mar 2018 02:22:33 GMT): baohua (Mon, 12 Mar 2018 05:28:12 GMT): baohua (Mon, 12 Mar 2018 05:28:22 GMT): baohua (Mon, 12 Mar 2018 05:28:27 GMT): baohua (Mon, 12 Mar 2018 05:32:24 GMT): Luke_Chen (Mon, 12 Mar 2018 06:16:00 GMT): hightall (Mon, 12 Mar 2018 12:35:06 GMT): hightall (Mon, 12 Mar 2018 12:35:16 GMT): baohua (Mon, 12 Mar 2018 12:44:23 GMT): baohua (Mon, 12 Mar 2018 12:44:23 GMT): baohua (Mon, 12 Mar 2018 12:44:23 GMT): ChrisFowers (Mon, 12 Mar 2018 18:21:10 GMT): ChrisFowers (Mon, 12 Mar 2018 18:21:49 GMT): ChrisFowers (Mon, 12 Mar 2018 18:32:44 GMT): ChrisFowers (Mon, 12 Mar 2018 18:32:44 GMT): AlexBA 6 (Mon, 12 Mar 2018 18:37:56 GMT): AlexBA 6 (Mon, 12 Mar 2018 18:53:42 GMT): ChrisFowers (Mon, 12 Mar 2018 18:55:49 GMT): ChrisFowers (Mon, 12 Mar 2018 19:12:26 GMT): ChrisFowers (Mon, 12 Mar 2018 19:12:36 GMT): AlexBA 6 (Mon, 12 Mar 2018 19:51:15 GMT): tongli (Mon, 12 Mar 2018 20:34:47 GMT): tongli (Mon, 12 Mar 2018 20:35:06 GMT): tongli (Mon, 12 Mar 2018 20:35:13 GMT): tongli (Mon, 12 Mar 2018 20:35:32 GMT): tongli (Mon, 12 Mar 2018 20:35:40 GMT): tongli (Mon, 12 Mar 2018 20:36:02 GMT): tongli (Mon, 12 Mar 2018 20:37:39 GMT): tongli (Mon, 12 Mar 2018 20:38:02 GMT): ChrisFowers (Mon, 12 Mar 2018 23:03:37 GMT): rjones (Tue, 13 Mar 2018 01:13:39 GMT): rjones (Tue, 13 Mar 2018 01:13:56 GMT): GopalPanda (Tue, 13 Mar 2018 01:17:51 GMT): baohua (Tue, 13 Mar 2018 01:35:45 GMT): dinesh.rivankar (Tue, 13 Mar 2018 06:34:01 GMT): pb (Tue, 13 Mar 2018 06:37:34 GMT): DRSK (Tue, 13 Mar 2018 06:59:23 GMT): pb (Tue, 13 Mar 2018 07:00:13 GMT): dinesh.rivankar (Tue, 13 Mar 2018 07:07:46 GMT): Nghiadt (Tue, 13 Mar 2018 09:41:09 GMT): rolland (Tue, 13 Mar 2018 09:55:19 GMT): pb (Tue, 13 Mar 2018 11:49:13 GMT): eramitg (Tue, 13 Mar 2018 13:29:24 GMT): rjones (Tue, 13 Mar 2018 14:21:02 GMT): Amir (Tue, 13 Mar 2018 17:42:40 GMT): ctan (Wed, 14 Mar 2018 01:16:35 GMT): yopep (Wed, 14 Mar 2018 05:13:04 GMT): pb (Wed, 14 Mar 2018 05:28:42 GMT): yopep (Wed, 14 Mar 2018 05:30:18 GMT): pb (Wed, 14 Mar 2018 05:44:19 GMT): yopep (Wed, 14 Mar 2018 05:45:42 GMT): yopep (Wed, 14 Mar 2018 05:48:14 GMT): dinesh.rivankar (Wed, 14 Mar 2018 05:53:21 GMT): pb (Wed, 14 Mar 2018 07:15:40 GMT): gxx (Wed, 14 Mar 2018 07:56:52 GMT): dinesh.rivankar (Wed, 14 Mar 2018 08:54:58 GMT): dinesh.rivankar (Wed, 14 Mar 2018 08:54:58 GMT): pb (Wed, 14 Mar 2018 09:30:49 GMT): pb (Wed, 14 Mar 2018 09:30:49 GMT): yopep (Wed, 14 Mar 2018 09:45:19 GMT): yopep (Wed, 14 Mar 2018 09:45:59 GMT): yopep (Wed, 14 Mar 2018 09:46:36 GMT): pb (Wed, 14 Mar 2018 09:48:13 GMT): pb (Wed, 14 Mar 2018 09:50:57 GMT): yopep (Wed, 14 Mar 2018 09:55:44 GMT): pb (Wed, 14 Mar 2018 10:04:35 GMT): yopep (Wed, 14 Mar 2018 10:07:24 GMT): yopep (Wed, 14 Mar 2018 10:08:51 GMT): pb (Wed, 14 Mar 2018 10:19:40 GMT): pb (Wed, 14 Mar 2018 10:19:40 GMT): dinesh.rivankar (Wed, 14 Mar 2018 11:58:15 GMT): DavidHiston (Wed, 14 Mar 2018 12:01:29 GMT): AlexBA 6 (Wed, 14 Mar 2018 15:11:35 GMT): AlexBA 6 (Wed, 14 Mar 2018 15:14:06 GMT): brian038 (Wed, 14 Mar 2018 15:27:06 GMT): tongli (Wed, 14 Mar 2018 16:51:22 GMT): tongli (Wed, 14 Mar 2018 16:51:34 GMT): tongli (Wed, 14 Mar 2018 16:52:03 GMT): baohua (Thu, 15 Mar 2018 01:25:14 GMT): baohua (Thu, 15 Mar 2018 01:39:47 GMT): baohua (Thu, 15 Mar 2018 01:43:15 GMT): baohua (Thu, 15 Mar 2018 01:43:24 GMT): baohua (Thu, 15 Mar 2018 01:43:43 GMT): baohua (Thu, 15 Mar 2018 01:53:57 GMT): baohua (Thu, 15 Mar 2018 01:53:57 GMT): baohua (Thu, 15 Mar 2018 01:53:57 GMT): ctan (Thu, 15 Mar 2018 03:14:45 GMT): hightall (Thu, 15 Mar 2018 03:17:34 GMT): ctan (Thu, 15 Mar 2018 03:26:03 GMT): VipinB (Thu, 15 Mar 2018 14:12:09 GMT): LeonardEdwin (Thu, 15 Mar 2018 14:14:36 GMT): hightall (Thu, 15 Mar 2018 14:27:17 GMT): iDix (Thu, 15 Mar 2018 14:55:14 GMT): eramitg (Thu, 15 Mar 2018 14:59:19 GMT): yopep (Fri, 16 Mar 2018 02:38:45 GMT): yopep (Fri, 16 Mar 2018 02:38:45 GMT): yopep (Fri, 16 Mar 2018 05:09:01 GMT): dinesh.rivankar (Fri, 16 Mar 2018 10:53:41 GMT): eramitg (Fri, 16 Mar 2018 12:30:06 GMT): eramitg (Fri, 16 Mar 2018 12:52:29 GMT): eramitg (Fri, 16 Mar 2018 12:52:36 GMT): Luke_Chen (Fri, 16 Mar 2018 13:24:14 GMT): Luke_Chen (Fri, 16 Mar 2018 13:24:20 GMT): baohua (Fri, 16 Mar 2018 14:57:50 GMT): baohua (Fri, 16 Mar 2018 14:58:05 GMT): Mihai.A (Fri, 16 Mar 2018 15:00:21 GMT): yopep (Fri, 16 Mar 2018 15:03:55 GMT): lanshanwanghao (Sat, 17 Mar 2018 10:49:22 GMT): matthewehoward (Sat, 17 Mar 2018 12:05:28 GMT): baohua (Sun, 18 Mar 2018 10:18:32 GMT): rrantzau (Sun, 18 Mar 2018 18:49:01 GMT): baohua (Mon, 19 Mar 2018 02:31:21 GMT): yopep (Mon, 19 Mar 2018 02:38:07 GMT): yopep (Mon, 19 Mar 2018 03:19:09 GMT): Luke_Chen (Mon, 19 Mar 2018 06:22:52 GMT): yopep (Mon, 19 Mar 2018 06:43:47 GMT): tongli (Mon, 19 Mar 2018 19:40:03 GMT): baohua (Tue, 20 Mar 2018 01:44:37 GMT): baohua (Tue, 20 Mar 2018 01:51:23 GMT): ck0316 (Tue, 20 Mar 2018 03:43:47 GMT): Unni_1994 (Tue, 20 Mar 2018 09:32:54 GMT): Unni_1994 (Tue, 20 Mar 2018 10:16:32 GMT): Unni_1994 (Tue, 20 Mar 2018 10:16:53 GMT): Unni_1994 (Tue, 20 Mar 2018 10:18:34 GMT): yopep (Tue, 20 Mar 2018 10:31:58 GMT): yopep (Tue, 20 Mar 2018 10:36:06 GMT): yopep (Tue, 20 Mar 2018 10:44:39 GMT): Unni_1994 (Tue, 20 Mar 2018 11:00:59 GMT): divudivyansh (Tue, 20 Mar 2018 13:00:42 GMT): walnemer1976 (Tue, 20 Mar 2018 13:02:37 GMT): divudivyansh (Tue, 20 Mar 2018 13:04:17 GMT): walnemer1976 (Tue, 20 Mar 2018 13:07:26 GMT): yopep (Tue, 20 Mar 2018 13:09:31 GMT): walnemer1976 (Tue, 20 Mar 2018 13:10:20 GMT): walnemer1976 (Tue, 20 Mar 2018 13:10:48 GMT): walnemer1976 (Tue, 20 Mar 2018 13:11:04 GMT): yopep (Tue, 20 Mar 2018 13:13:01 GMT): yopep (Tue, 20 Mar 2018 13:22:56 GMT): nitin6ul (Tue, 20 Mar 2018 13:31:01 GMT): walnemer1976 (Tue, 20 Mar 2018 13:48:55 GMT): walnemer1976 (Tue, 20 Mar 2018 13:49:07 GMT): yopep (Tue, 20 Mar 2018 13:50:52 GMT): walnemer1976 (Tue, 20 Mar 2018 13:51:34 GMT): yopep (Tue, 20 Mar 2018 13:56:02 GMT): walnemer1976 (Tue, 20 Mar 2018 15:05:32 GMT): walnemer1976 (Tue, 20 Mar 2018 15:11:38 GMT): walnemer1976 (Tue, 20 Mar 2018 17:44:55 GMT): Amir (Tue, 20 Mar 2018 19:18:45 GMT): Amir (Tue, 20 Mar 2018 19:21:13 GMT): walnemer1976 (Tue, 20 Mar 2018 19:22:19 GMT): walnemer1976 (Tue, 20 Mar 2018 19:22:55 GMT): Amir (Tue, 20 Mar 2018 20:14:28 GMT): walnemer1976 (Tue, 20 Mar 2018 20:30:25 GMT): walnemer1976 (Tue, 20 Mar 2018 20:58:57 GMT): Amir (Tue, 20 Mar 2018 21:01:49 GMT): Amir (Tue, 20 Mar 2018 21:05:16 GMT): walnemer1976 (Tue, 20 Mar 2018 21:10:48 GMT): Amir (Tue, 20 Mar 2018 21:23:33 GMT): walnemer1976 (Tue, 20 Mar 2018 21:53:12 GMT): walnemer1976 (Tue, 20 Mar 2018 21:53:31 GMT): walnemer1976 (Tue, 20 Mar 2018 21:53:56 GMT): walnemer1976 (Tue, 20 Mar 2018 21:54:25 GMT): ShikarSharma (Tue, 20 Mar 2018 22:35:40 GMT): Amir (Tue, 20 Mar 2018 23:04:59 GMT): divudivyansh (Wed, 21 Mar 2018 04:56:48 GMT): baohua (Wed, 21 Mar 2018 05:15:44 GMT): yopep (Wed, 21 Mar 2018 05:17:37 GMT): walnemer1976 (Wed, 21 Mar 2018 05:19:09 GMT): yopep (Wed, 21 Mar 2018 05:33:34 GMT): baohua (Wed, 21 Mar 2018 05:34:23 GMT): yopep (Wed, 21 Mar 2018 05:37:30 GMT): Luke_Chen (Wed, 21 Mar 2018 06:00:38 GMT): Luke_Chen (Wed, 21 Mar 2018 06:03:43 GMT): yopep (Wed, 21 Mar 2018 06:04:21 GMT): yopep (Wed, 21 Mar 2018 06:04:21 GMT): divudivyansh (Wed, 21 Mar 2018 06:04:25 GMT): Luke_Chen (Wed, 21 Mar 2018 06:04:26 GMT): yopep (Wed, 21 Mar 2018 06:09:32 GMT): Luke_Chen (Wed, 21 Mar 2018 06:10:00 GMT): Luke_Chen (Wed, 21 Mar 2018 06:12:02 GMT): baohua (Wed, 21 Mar 2018 06:16:44 GMT): Luke_Chen (Wed, 21 Mar 2018 06:17:13 GMT): Luke_Chen (Wed, 21 Mar 2018 06:17:50 GMT): divudivyansh (Wed, 21 Mar 2018 06:19:55 GMT): yopep (Wed, 21 Mar 2018 07:22:00 GMT): yopep (Wed, 21 Mar 2018 07:23:42 GMT): yopep (Wed, 21 Mar 2018 07:23:42 GMT): lvzewen (Wed, 21 Mar 2018 11:13:21 GMT): yopep (Wed, 21 Mar 2018 12:33:28 GMT): yopep (Wed, 21 Mar 2018 12:33:28 GMT): lujiang (Wed, 21 Mar 2018 13:32:58 GMT): lujiang (Wed, 21 Mar 2018 13:33:06 GMT): baohua (Wed, 21 Mar 2018 14:18:07 GMT): baohua (Wed, 21 Mar 2018 14:22:47 GMT): tonym1 (Wed, 21 Mar 2018 14:23:00 GMT): tonym1 (Wed, 21 Mar 2018 14:26:39 GMT): jverhoelen (Wed, 21 Mar 2018 14:50:06 GMT): hightall (Wed, 21 Mar 2018 15:49:59 GMT): tongli (Wed, 21 Mar 2018 16:24:30 GMT): tonym1 (Wed, 21 Mar 2018 17:49:30 GMT): tongli (Wed, 21 Mar 2018 17:57:19 GMT): tongli (Wed, 21 Mar 2018 17:57:43 GMT): tongli (Wed, 21 Mar 2018 17:58:13 GMT): tongli (Wed, 21 Mar 2018 17:58:21 GMT): tonym1 (Wed, 21 Mar 2018 18:41:19 GMT): Amir (Wed, 21 Mar 2018 21:11:50 GMT): SuddenRainx (Wed, 21 Mar 2018 21:44:01 GMT): yopep (Thu, 22 Mar 2018 01:46:40 GMT): baohua (Thu, 22 Mar 2018 01:47:41 GMT): baohua (Thu, 22 Mar 2018 01:48:01 GMT): yopep (Thu, 22 Mar 2018 01:49:51 GMT): danfeng (Thu, 22 Mar 2018 03:28:50 GMT): suntoe (Thu, 22 Mar 2018 03:48:46 GMT): danfeng (Thu, 22 Mar 2018 05:46:47 GMT): danfeng (Thu, 22 Mar 2018 05:47:04 GMT): danfeng (Thu, 22 Mar 2018 05:47:37 GMT): danfeng (Thu, 22 Mar 2018 05:48:36 GMT): lan (Thu, 22 Mar 2018 05:50:56 GMT): lan (Thu, 22 Mar 2018 05:53:53 GMT): danfeng (Thu, 22 Mar 2018 05:55:05 GMT): danfeng (Thu, 22 Mar 2018 05:56:39 GMT): danfeng (Thu, 22 Mar 2018 05:59:48 GMT): guruce (Thu, 22 Mar 2018 07:00:19 GMT): harsha (Thu, 22 Mar 2018 09:00:09 GMT): baohua (Thu, 22 Mar 2018 10:58:51 GMT): darunk67 (Thu, 22 Mar 2018 11:30:09 GMT): baohua (Thu, 22 Mar 2018 14:20:05 GMT): hightall (Thu, 22 Mar 2018 15:27:35 GMT): baohua (Fri, 23 Mar 2018 00:47:03 GMT): baohua (Fri, 23 Mar 2018 01:48:43 GMT): StevenSiahetiong (Fri, 23 Mar 2018 01:59:34 GMT): PrashantS (Fri, 23 Mar 2018 03:15:20 GMT): yinzhiwu (Fri, 23 Mar 2018 05:47:46 GMT): sillysachin (Fri, 23 Mar 2018 06:52:42 GMT): MrLongmire (Fri, 23 Mar 2018 07:54:13 GMT): tongli (Fri, 23 Mar 2018 12:25:42 GMT): tongli (Fri, 23 Mar 2018 12:26:07 GMT): ashishapy (Fri, 23 Mar 2018 12:34:44 GMT): baohua (Fri, 23 Mar 2018 14:35:15 GMT): Luke_Chen (Fri, 23 Mar 2018 15:45:08 GMT): ashishapy (Fri, 23 Mar 2018 16:34:53 GMT): ashishapy (Fri, 23 Mar 2018 16:38:02 GMT): Amir (Sat, 24 Mar 2018 01:15:00 GMT): baohua (Sat, 24 Mar 2018 11:08:34 GMT): baohua (Sat, 24 Mar 2018 11:08:56 GMT): ashishapy (Sat, 24 Mar 2018 12:04:07 GMT): czar0 (Sun, 25 Mar 2018 21:27:34 GMT): CupidYan (Mon, 26 Mar 2018 01:38:27 GMT): baohua (Mon, 26 Mar 2018 02:52:40 GMT): baohua (Mon, 26 Mar 2018 02:57:47 GMT): Luke_Chen (Mon, 26 Mar 2018 03:05:37 GMT): danfeng (Mon, 26 Mar 2018 06:24:54 GMT): danfeng (Mon, 26 Mar 2018 08:42:41 GMT): danfeng (Mon, 26 Mar 2018 08:43:06 GMT): ashishapy (Mon, 26 Mar 2018 13:01:41 GMT): ss03 (Mon, 26 Mar 2018 15:32:33 GMT): ss03 (Mon, 26 Mar 2018 15:32:49 GMT): ss03 (Mon, 26 Mar 2018 15:33:06 GMT): MattHamilton (Mon, 26 Mar 2018 15:43:50 GMT): ss03 (Mon, 26 Mar 2018 15:51:37 GMT): randallj (Mon, 26 Mar 2018 21:11:58 GMT): randallj (Mon, 26 Mar 2018 21:14:01 GMT): Amir (Mon, 26 Mar 2018 21:26:56 GMT): randallj (Mon, 26 Mar 2018 21:37:42 GMT): randallj (Mon, 26 Mar 2018 21:39:23 GMT): Amir (Mon, 26 Mar 2018 22:49:16 GMT): sillysachin (Tue, 27 Mar 2018 04:56:43 GMT): Luke_Chen (Tue, 27 Mar 2018 06:07:45 GMT): Luke_Chen (Tue, 27 Mar 2018 06:09:54 GMT): lujiang (Tue, 27 Mar 2018 08:48:11 GMT): baohua (Tue, 27 Mar 2018 08:50:17 GMT): zian.yusuf (Tue, 27 Mar 2018 09:40:59 GMT): SaurabhDayama (Tue, 27 Mar 2018 18:31:04 GMT): randallj (Tue, 27 Mar 2018 20:25:54 GMT): JiuZhuYou (Wed, 28 Mar 2018 06:36:36 GMT): DaraPenhchet (Wed, 28 Mar 2018 07:43:52 GMT): AshishMishra 1 (Wed, 28 Mar 2018 09:38:42 GMT): Ricckk (Wed, 28 Mar 2018 12:14:30 GMT): hightall (Wed, 28 Mar 2018 14:10:22 GMT): baohua (Wed, 28 Mar 2018 14:37:05 GMT): baohua (Wed, 28 Mar 2018 14:37:25 GMT): baohua (Wed, 28 Mar 2018 14:37:38 GMT): KOttoni (Wed, 28 Mar 2018 16:51:06 GMT): Amir (Wed, 28 Mar 2018 18:28:20 GMT): StevenP 4 (Wed, 28 Mar 2018 20:00:28 GMT): StevenP 4 (Wed, 28 Mar 2018 20:01:57 GMT): hightall (Thu, 29 Mar 2018 01:23:14 GMT): baohua (Thu, 29 Mar 2018 01:57:59 GMT): ZPDCEO (Thu, 29 Mar 2018 06:28:18 GMT): robin666 (Thu, 29 Mar 2018 07:31:01 GMT): harsha (Thu, 29 Mar 2018 10:21:44 GMT): baohua (Thu, 29 Mar 2018 10:40:00 GMT): baohua (Thu, 29 Mar 2018 10:40:34 GMT): harsha (Thu, 29 Mar 2018 11:40:41 GMT): AshishMishra 1 (Thu, 29 Mar 2018 12:14:10 GMT): harsha (Thu, 29 Mar 2018 13:10:44 GMT): harsha (Thu, 29 Mar 2018 13:10:44 GMT): rameshthoomu (Thu, 29 Mar 2018 18:38:20 GMT): harsha (Fri, 30 Mar 2018 08:19:22 GMT): thalisson (Sun, 01 Apr 2018 02:39:58 GMT): Othman.Darwish (Sun, 01 Apr 2018 06:34:04 GMT): yazanalhjaj (Sun, 01 Apr 2018 08:36:51 GMT): Nghiadt (Sun, 01 Apr 2018 15:36:47 GMT): ashishapy (Mon, 02 Apr 2018 02:03:05 GMT): greg.haskins (Mon, 02 Apr 2018 02:03:05 GMT): harsha (Mon, 02 Apr 2018 05:49:09 GMT): hightall (Mon, 02 Apr 2018 06:17:55 GMT): dinesh.rivankar (Mon, 02 Apr 2018 08:19:39 GMT): dinesh.rivankar (Mon, 02 Apr 2018 08:57:15 GMT): dinesh.rivankar (Mon, 02 Apr 2018 08:57:51 GMT): baohua (Mon, 02 Apr 2018 09:27:31 GMT): baohua (Mon, 02 Apr 2018 09:31:24 GMT): baohua (Mon, 02 Apr 2018 09:31:26 GMT): baohua (Mon, 02 Apr 2018 09:31:56 GMT): baohua (Mon, 02 Apr 2018 09:32:07 GMT): dinesh.rivankar (Mon, 02 Apr 2018 09:33:32 GMT): dinesh.rivankar (Mon, 02 Apr 2018 09:50:21 GMT): tongli (Mon, 02 Apr 2018 13:23:35 GMT): tongli (Mon, 02 Apr 2018 13:24:25 GMT): tongli (Mon, 02 Apr 2018 14:33:05 GMT): Nghiadt (Mon, 02 Apr 2018 15:34:52 GMT): Nghiadt (Mon, 02 Apr 2018 15:40:39 GMT): collincusce (Mon, 02 Apr 2018 19:30:05 GMT): collincusce (Mon, 02 Apr 2018 19:33:01 GMT): collincusce (Mon, 02 Apr 2018 19:33:24 GMT): collincusce (Mon, 02 Apr 2018 19:35:00 GMT): collincusce (Mon, 02 Apr 2018 19:35:03 GMT): collincusce (Mon, 02 Apr 2018 19:54:51 GMT): coffeeplease (Mon, 02 Apr 2018 20:57:16 GMT): baohua (Tue, 03 Apr 2018 00:17:20 GMT): baohua (Tue, 03 Apr 2018 00:18:21 GMT): baohua (Tue, 03 Apr 2018 00:18:21 GMT): baohua (Tue, 03 Apr 2018 01:15:46 GMT): baohua (Tue, 03 Apr 2018 01:21:18 GMT): vparihar (Tue, 03 Apr 2018 04:22:28 GMT): baohua (Tue, 03 Apr 2018 04:57:52 GMT): baohua (Tue, 03 Apr 2018 04:58:11 GMT): baohua (Tue, 03 Apr 2018 04:58:39 GMT): dinesh.rivankar (Tue, 03 Apr 2018 05:00:30 GMT): dinesh.rivankar (Tue, 03 Apr 2018 05:01:51 GMT): baohua (Tue, 03 Apr 2018 05:02:11 GMT): baohua (Tue, 03 Apr 2018 05:02:40 GMT): baohua (Tue, 03 Apr 2018 05:02:51 GMT): dinesh.rivankar (Tue, 03 Apr 2018 05:03:05 GMT): dinesh.rivankar (Tue, 03 Apr 2018 05:03:18 GMT): baohua (Tue, 03 Apr 2018 05:07:25 GMT): Amir (Tue, 03 Apr 2018 07:01:57 GMT): baohua (Tue, 03 Apr 2018 07:42:58 GMT): Amir (Tue, 03 Apr 2018 08:01:54 GMT): dinesh.rivankar (Tue, 03 Apr 2018 08:45:54 GMT): baohua (Tue, 03 Apr 2018 12:09:15 GMT): tongli (Tue, 03 Apr 2018 13:22:15 GMT): tongli (Tue, 03 Apr 2018 13:22:17 GMT): tongli (Tue, 03 Apr 2018 13:23:06 GMT): hightall (Tue, 03 Apr 2018 13:47:18 GMT): tongli (Tue, 03 Apr 2018 13:47:37 GMT): tongli (Tue, 03 Apr 2018 13:47:46 GMT): hightall (Tue, 03 Apr 2018 13:48:21 GMT): collincusce (Tue, 03 Apr 2018 13:55:44 GMT): collincusce (Tue, 03 Apr 2018 13:56:00 GMT): tongli (Tue, 03 Apr 2018 13:57:13 GMT): collincusce (Tue, 03 Apr 2018 13:57:33 GMT): tongli (Tue, 03 Apr 2018 13:57:44 GMT): collincusce (Tue, 03 Apr 2018 13:57:52 GMT): collincusce (Tue, 03 Apr 2018 13:58:01 GMT): collincusce (Tue, 03 Apr 2018 14:00:16 GMT): collincusce (Tue, 03 Apr 2018 14:00:54 GMT): collincusce (Tue, 03 Apr 2018 14:00:59 GMT): tongli (Tue, 03 Apr 2018 14:04:50 GMT): tongli (Tue, 03 Apr 2018 14:07:48 GMT): collincusce (Tue, 03 Apr 2018 14:08:41 GMT): tongli (Tue, 03 Apr 2018 14:09:46 GMT): tongli (Tue, 03 Apr 2018 14:10:41 GMT): tongli (Tue, 03 Apr 2018 14:11:15 GMT): collincusce (Tue, 03 Apr 2018 14:11:31 GMT): tongli (Tue, 03 Apr 2018 14:11:54 GMT): tongli (Tue, 03 Apr 2018 14:12:11 GMT): collincusce (Tue, 03 Apr 2018 14:12:21 GMT): tongli (Tue, 03 Apr 2018 14:12:40 GMT): tongli (Tue, 03 Apr 2018 14:13:01 GMT): collincusce (Tue, 03 Apr 2018 14:13:01 GMT): collincusce (Tue, 03 Apr 2018 14:13:24 GMT): collincusce (Tue, 03 Apr 2018 14:13:27 GMT): tongli (Tue, 03 Apr 2018 14:13:43 GMT): collincusce (Tue, 03 Apr 2018 14:13:47 GMT): tongli (Tue, 03 Apr 2018 14:14:04 GMT): collincusce (Tue, 03 Apr 2018 14:14:10 GMT): tongli (Tue, 03 Apr 2018 14:14:12 GMT): collincusce (Tue, 03 Apr 2018 14:14:27 GMT): collincusce (Tue, 03 Apr 2018 14:15:03 GMT): collincusce (Tue, 03 Apr 2018 14:15:09 GMT): tongli (Tue, 03 Apr 2018 14:15:12 GMT): collincusce (Tue, 03 Apr 2018 14:15:12 GMT): tongli (Tue, 03 Apr 2018 14:15:21 GMT): tongli (Tue, 03 Apr 2018 14:15:56 GMT): collincusce (Tue, 03 Apr 2018 14:16:11 GMT): collincusce (Tue, 03 Apr 2018 14:16:18 GMT): tongli (Tue, 03 Apr 2018 14:16:23 GMT): tongli (Tue, 03 Apr 2018 14:16:32 GMT): tongli (Tue, 03 Apr 2018 14:18:19 GMT): collincusce (Tue, 03 Apr 2018 14:19:22 GMT): tongli (Tue, 03 Apr 2018 14:19:26 GMT): collincusce (Tue, 03 Apr 2018 14:19:39 GMT): tongli (Tue, 03 Apr 2018 14:19:42 GMT): tongli (Tue, 03 Apr 2018 14:19:43 GMT): tongli (Tue, 03 Apr 2018 14:19:58 GMT): collincusce (Tue, 03 Apr 2018 14:20:16 GMT): collincusce (Tue, 03 Apr 2018 14:20:26 GMT): collincusce (Tue, 03 Apr 2018 14:20:41 GMT): collincusce (Tue, 03 Apr 2018 14:20:55 GMT): collincusce (Tue, 03 Apr 2018 14:21:11 GMT): tongli (Tue, 03 Apr 2018 14:21:15 GMT): collincusce (Tue, 03 Apr 2018 14:21:19 GMT): collincusce (Tue, 03 Apr 2018 14:21:19 GMT): collincusce (Tue, 03 Apr 2018 14:21:24 GMT): collincusce (Tue, 03 Apr 2018 14:21:35 GMT): tongli (Tue, 03 Apr 2018 14:22:15 GMT): collincusce (Tue, 03 Apr 2018 14:22:18 GMT): collincusce (Tue, 03 Apr 2018 14:22:23 GMT): tongli (Tue, 03 Apr 2018 14:22:27 GMT): collincusce (Tue, 03 Apr 2018 14:22:47 GMT): collincusce (Tue, 03 Apr 2018 14:22:56 GMT): tongli (Tue, 03 Apr 2018 14:23:10 GMT): tongli (Tue, 03 Apr 2018 14:23:17 GMT): tongli (Tue, 03 Apr 2018 14:23:20 GMT): collincusce (Tue, 03 Apr 2018 14:23:22 GMT): collincusce (Tue, 03 Apr 2018 14:23:41 GMT): collincusce (Tue, 03 Apr 2018 14:23:57 GMT): collincusce (Tue, 03 Apr 2018 14:24:01 GMT): tongli (Tue, 03 Apr 2018 14:24:26 GMT): collincusce (Tue, 03 Apr 2018 14:24:37 GMT): tongli (Tue, 03 Apr 2018 14:24:42 GMT): tongli (Tue, 03 Apr 2018 14:24:49 GMT): tongli (Tue, 03 Apr 2018 14:24:57 GMT): tongli (Tue, 03 Apr 2018 14:25:31 GMT): collincusce (Tue, 03 Apr 2018 14:25:36 GMT): collincusce (Tue, 03 Apr 2018 14:25:37 GMT): collincusce (Tue, 03 Apr 2018 14:25:38 GMT): collincusce (Tue, 03 Apr 2018 14:25:40 GMT): collincusce (Tue, 03 Apr 2018 14:25:58 GMT): tongli (Tue, 03 Apr 2018 14:25:59 GMT): tongli (Tue, 03 Apr 2018 14:26:08 GMT): tongli (Tue, 03 Apr 2018 14:26:22 GMT): collincusce (Tue, 03 Apr 2018 14:26:36 GMT): collincusce (Tue, 03 Apr 2018 14:26:56 GMT): collincusce (Tue, 03 Apr 2018 14:26:59 GMT): collincusce (Tue, 03 Apr 2018 14:27:02 GMT): tongli (Tue, 03 Apr 2018 14:27:09 GMT): collincusce (Tue, 03 Apr 2018 14:27:20 GMT): collincusce (Tue, 03 Apr 2018 14:28:26 GMT): tongli (Tue, 03 Apr 2018 14:28:28 GMT): collincusce (Tue, 03 Apr 2018 14:28:31 GMT): collincusce (Tue, 03 Apr 2018 14:28:39 GMT): collincusce (Tue, 03 Apr 2018 14:28:40 GMT): tongli (Tue, 03 Apr 2018 14:28:46 GMT): tongli (Tue, 03 Apr 2018 14:28:55 GMT): collincusce (Tue, 03 Apr 2018 14:28:55 GMT): collincusce (Tue, 03 Apr 2018 14:29:01 GMT): collincusce (Tue, 03 Apr 2018 14:29:08 GMT): tongli (Tue, 03 Apr 2018 14:29:15 GMT): tongli (Tue, 03 Apr 2018 14:29:23 GMT): collincusce (Tue, 03 Apr 2018 14:29:28 GMT): tongli (Tue, 03 Apr 2018 14:29:38 GMT): collincusce (Tue, 03 Apr 2018 14:29:40 GMT): tongli (Tue, 03 Apr 2018 14:29:44 GMT): collincusce (Tue, 03 Apr 2018 14:29:47 GMT): tongli (Tue, 03 Apr 2018 14:29:50 GMT): collincusce (Tue, 03 Apr 2018 14:29:50 GMT): tongli (Tue, 03 Apr 2018 14:29:55 GMT): tongli (Tue, 03 Apr 2018 14:30:06 GMT): tongli (Tue, 03 Apr 2018 14:30:11 GMT): tongli (Tue, 03 Apr 2018 14:30:32 GMT): collincusce (Tue, 03 Apr 2018 14:30:53 GMT): tongli (Tue, 03 Apr 2018 14:30:56 GMT): collincusce (Tue, 03 Apr 2018 14:31:02 GMT): tongli (Tue, 03 Apr 2018 14:31:08 GMT): collincusce (Tue, 03 Apr 2018 14:31:11 GMT): collincusce (Tue, 03 Apr 2018 14:31:31 GMT): tongli (Tue, 03 Apr 2018 14:31:40 GMT): tongli (Tue, 03 Apr 2018 14:32:12 GMT): tongli (Tue, 03 Apr 2018 14:32:21 GMT): tongli (Tue, 03 Apr 2018 14:32:32 GMT): tongli (Tue, 03 Apr 2018 14:32:42 GMT): tongli (Tue, 03 Apr 2018 14:32:54 GMT): tongli (Tue, 03 Apr 2018 14:33:22 GMT): tongli (Tue, 03 Apr 2018 14:34:05 GMT): tongli (Tue, 03 Apr 2018 14:34:36 GMT): tongli (Tue, 03 Apr 2018 14:35:04 GMT): tongli (Tue, 03 Apr 2018 14:35:10 GMT): collincusce (Tue, 03 Apr 2018 15:03:52 GMT): collincusce (Tue, 03 Apr 2018 15:04:01 GMT): collincusce (Tue, 03 Apr 2018 15:04:20 GMT): collincusce (Tue, 03 Apr 2018 15:05:06 GMT): collincusce (Tue, 03 Apr 2018 15:09:05 GMT): collincusce (Tue, 03 Apr 2018 15:18:26 GMT): JoeN 6 (Tue, 03 Apr 2018 15:22:48 GMT): collincusce (Tue, 03 Apr 2018 15:31:12 GMT): tongli (Tue, 03 Apr 2018 18:08:14 GMT): tongli (Tue, 03 Apr 2018 18:08:41 GMT): tongli (Tue, 03 Apr 2018 18:09:10 GMT): tongli (Tue, 03 Apr 2018 18:09:18 GMT): tongli (Tue, 03 Apr 2018 18:09:57 GMT): tongli (Tue, 03 Apr 2018 18:10:34 GMT): tongli (Tue, 03 Apr 2018 18:10:48 GMT): tongli (Tue, 03 Apr 2018 18:11:06 GMT): collincusce (Tue, 03 Apr 2018 18:13:03 GMT): collincusce (Tue, 03 Apr 2018 18:13:47 GMT): tongli (Tue, 03 Apr 2018 18:14:07 GMT): collincusce (Tue, 03 Apr 2018 18:14:10 GMT): tongli (Tue, 03 Apr 2018 18:14:24 GMT): tongli (Tue, 03 Apr 2018 18:14:30 GMT): collincusce (Tue, 03 Apr 2018 18:14:36 GMT): collincusce (Tue, 03 Apr 2018 18:14:52 GMT): tongli (Tue, 03 Apr 2018 18:14:54 GMT): collincusce (Tue, 03 Apr 2018 18:14:58 GMT): collincusce (Tue, 03 Apr 2018 18:15:15 GMT): tongli (Tue, 03 Apr 2018 18:15:26 GMT): tongli (Tue, 03 Apr 2018 18:15:38 GMT): tongli (Tue, 03 Apr 2018 18:15:54 GMT): tongli (Tue, 03 Apr 2018 18:16:00 GMT): tongli (Tue, 03 Apr 2018 18:16:07 GMT): collincusce (Tue, 03 Apr 2018 18:16:42 GMT): collincusce (Tue, 03 Apr 2018 18:16:58 GMT): tongli (Tue, 03 Apr 2018 18:17:11 GMT): collincusce (Tue, 03 Apr 2018 18:17:18 GMT): collincusce (Tue, 03 Apr 2018 18:17:20 GMT): tongli (Tue, 03 Apr 2018 18:17:38 GMT): collincusce (Tue, 03 Apr 2018 18:18:03 GMT): tongli (Tue, 03 Apr 2018 18:18:04 GMT): collincusce (Tue, 03 Apr 2018 18:18:18 GMT): tongli (Tue, 03 Apr 2018 18:19:00 GMT): collincusce (Tue, 03 Apr 2018 18:19:01 GMT): tongli (Tue, 03 Apr 2018 18:19:02 GMT): collincusce (Tue, 03 Apr 2018 18:19:28 GMT): collincusce (Tue, 03 Apr 2018 18:19:28 GMT): collincusce (Tue, 03 Apr 2018 18:19:41 GMT): tongli (Tue, 03 Apr 2018 18:20:08 GMT): collincusce (Tue, 03 Apr 2018 18:20:24 GMT): collincusce (Tue, 03 Apr 2018 18:20:37 GMT): collincusce (Tue, 03 Apr 2018 18:21:23 GMT): tongli (Tue, 03 Apr 2018 18:21:28 GMT): tongli (Tue, 03 Apr 2018 18:21:45 GMT): collincusce (Tue, 03 Apr 2018 18:21:51 GMT): tongli (Tue, 03 Apr 2018 18:22:16 GMT): collincusce (Tue, 03 Apr 2018 18:22:22 GMT): collincusce (Tue, 03 Apr 2018 18:22:24 GMT): tongli (Tue, 03 Apr 2018 18:22:30 GMT): collincusce (Tue, 03 Apr 2018 18:22:35 GMT): tongli (Tue, 03 Apr 2018 18:22:44 GMT): collincusce (Tue, 03 Apr 2018 18:22:48 GMT): collincusce (Tue, 03 Apr 2018 18:22:58 GMT): tongli (Tue, 03 Apr 2018 18:23:06 GMT): collincusce (Tue, 03 Apr 2018 18:23:10 GMT): collincusce (Tue, 03 Apr 2018 18:23:37 GMT): tongli (Tue, 03 Apr 2018 18:23:40 GMT): collincusce (Tue, 03 Apr 2018 18:23:41 GMT): collincusce (Tue, 03 Apr 2018 18:24:14 GMT): tongli (Tue, 03 Apr 2018 18:24:17 GMT): collincusce (Tue, 03 Apr 2018 18:24:19 GMT): collincusce (Tue, 03 Apr 2018 18:24:38 GMT): collincusce (Tue, 03 Apr 2018 18:24:52 GMT): collincusce (Tue, 03 Apr 2018 18:25:11 GMT): tongli (Tue, 03 Apr 2018 18:25:29 GMT): collincusce (Tue, 03 Apr 2018 18:25:41 GMT): tongli (Tue, 03 Apr 2018 18:25:55 GMT): tongli (Tue, 03 Apr 2018 18:26:16 GMT): tongli (Tue, 03 Apr 2018 18:26:21 GMT): collincusce (Tue, 03 Apr 2018 18:26:27 GMT): tongli (Tue, 03 Apr 2018 18:26:31 GMT): tongli (Tue, 03 Apr 2018 18:26:37 GMT): collincusce (Tue, 03 Apr 2018 18:26:51 GMT): collincusce (Tue, 03 Apr 2018 18:26:55 GMT): tongli (Tue, 03 Apr 2018 18:27:02 GMT): collincusce (Tue, 03 Apr 2018 18:27:07 GMT): tongli (Tue, 03 Apr 2018 18:27:21 GMT): collincusce (Tue, 03 Apr 2018 18:27:22 GMT): collincusce (Tue, 03 Apr 2018 18:27:29 GMT): collincusce (Tue, 03 Apr 2018 18:27:35 GMT): tongli (Tue, 03 Apr 2018 18:27:46 GMT): collincusce (Tue, 03 Apr 2018 18:27:50 GMT): tongli (Tue, 03 Apr 2018 18:28:10 GMT): collincusce (Tue, 03 Apr 2018 18:28:13 GMT): tongli (Tue, 03 Apr 2018 18:28:15 GMT): collincusce (Tue, 03 Apr 2018 18:28:27 GMT): collincusce (Tue, 03 Apr 2018 18:28:49 GMT): tongli (Tue, 03 Apr 2018 18:29:10 GMT): collincusce (Tue, 03 Apr 2018 18:29:35 GMT): tongli (Tue, 03 Apr 2018 18:29:54 GMT): collincusce (Tue, 03 Apr 2018 18:29:58 GMT): collincusce (Tue, 03 Apr 2018 18:30:00 GMT): tongli (Tue, 03 Apr 2018 18:30:10 GMT): tongli (Tue, 03 Apr 2018 18:30:13 GMT): collincusce (Tue, 03 Apr 2018 18:36:45 GMT): collincusce (Tue, 03 Apr 2018 18:36:47 GMT): collincusce (Tue, 03 Apr 2018 18:36:54 GMT): tongli (Tue, 03 Apr 2018 18:37:19 GMT): collincusce (Tue, 03 Apr 2018 18:37:21 GMT): collincusce (Tue, 03 Apr 2018 18:37:21 GMT): collincusce (Tue, 03 Apr 2018 18:37:23 GMT): tongli (Tue, 03 Apr 2018 18:37:31 GMT): collincusce (Tue, 03 Apr 2018 18:38:47 GMT): collincusce (Tue, 03 Apr 2018 18:38:48 GMT): collincusce (Tue, 03 Apr 2018 18:45:32 GMT): collincusce (Tue, 03 Apr 2018 18:45:32 GMT): collincusce (Tue, 03 Apr 2018 18:51:20 GMT): collincusce (Tue, 03 Apr 2018 18:51:28 GMT): collincusce (Tue, 03 Apr 2018 18:56:56 GMT): collincusce (Tue, 03 Apr 2018 18:57:06 GMT): collincusce (Tue, 03 Apr 2018 18:57:58 GMT): collincusce (Tue, 03 Apr 2018 19:10:39 GMT): collincusce (Tue, 03 Apr 2018 19:17:47 GMT): collincusce (Tue, 03 Apr 2018 19:18:06 GMT): collincusce (Tue, 03 Apr 2018 19:42:28 GMT): collincusce (Tue, 03 Apr 2018 19:42:38 GMT): collincusce (Tue, 03 Apr 2018 19:42:54 GMT): collincusce (Tue, 03 Apr 2018 20:03:34 GMT): tongli (Wed, 04 Apr 2018 00:59:58 GMT): tongli (Wed, 04 Apr 2018 01:00:20 GMT): tongli (Wed, 04 Apr 2018 01:00:44 GMT): xiao8 (Wed, 04 Apr 2018 01:17:11 GMT): baohua (Wed, 04 Apr 2018 01:45:17 GMT): greg.haskins (Wed, 04 Apr 2018 01:45:50 GMT): baohua (Wed, 04 Apr 2018 03:28:16 GMT): dinesh.rivankar (Wed, 04 Apr 2018 04:47:26 GMT): ydk210999 (Wed, 04 Apr 2018 04:50:34 GMT): ydk210999 (Wed, 04 Apr 2018 04:51:33 GMT): ydk210999 (Wed, 04 Apr 2018 04:52:36 GMT): niteshsolanki (Wed, 04 Apr 2018 06:02:11 GMT): thiennguyen2824 (Wed, 04 Apr 2018 08:46:49 GMT): nimeshpr (Wed, 04 Apr 2018 23:59:01 GMT): rjones (Thu, 05 Apr 2018 03:13:46 GMT): rjones (Thu, 05 Apr 2018 03:14:00 GMT): kpbird (Thu, 05 Apr 2018 04:38:33 GMT): kpbird (Thu, 05 Apr 2018 04:39:19 GMT): Unni_1994 (Thu, 05 Apr 2018 09:21:13 GMT): Unni_1994 (Thu, 05 Apr 2018 09:22:47 GMT): Unni_1994 (Thu, 05 Apr 2018 09:23:08 GMT): Unni_1994 (Thu, 05 Apr 2018 09:23:40 GMT): tongli (Thu, 05 Apr 2018 11:40:50 GMT): vkblue (Thu, 05 Apr 2018 12:07:37 GMT): Amir (Thu, 05 Apr 2018 12:48:23 GMT): rjones (Thu, 05 Apr 2018 12:52:34 GMT): tongli (Thu, 05 Apr 2018 12:53:35 GMT): rjones (Thu, 05 Apr 2018 12:55:30 GMT): tongli (Thu, 05 Apr 2018 12:55:59 GMT): Unni_1994 (Thu, 05 Apr 2018 12:57:22 GMT): Amir (Thu, 05 Apr 2018 13:01:41 GMT): rjones (Sat, 07 Apr 2018 01:19:10 GMT): tongli (Sat, 07 Apr 2018 02:10:59 GMT): baohua (Sat, 07 Apr 2018 03:21:09 GMT): kkado (Sat, 07 Apr 2018 04:11:23 GMT): ashishapy (Sat, 07 Apr 2018 14:33:54 GMT): Luke_Chen (Sun, 08 Apr 2018 02:33:53 GMT): baohua (Sun, 08 Apr 2018 07:12:46 GMT): baohua (Sun, 08 Apr 2018 07:12:47 GMT): Luke_Chen (Sun, 08 Apr 2018 07:56:18 GMT): baohua (Sun, 08 Apr 2018 07:58:07 GMT): Luke_Chen (Sun, 08 Apr 2018 08:02:38 GMT): Luke_Chen (Sun, 08 Apr 2018 08:16:20 GMT): Luke_Chen (Sun, 08 Apr 2018 08:17:54 GMT): baohua (Sun, 08 Apr 2018 08:18:52 GMT): Luke_Chen (Sun, 08 Apr 2018 08:19:27 GMT): Luke_Chen (Sun, 08 Apr 2018 08:30:36 GMT): baohua (Sun, 08 Apr 2018 08:31:16 GMT): Luke_Chen (Sun, 08 Apr 2018 08:31:27 GMT): Luke_Chen (Sun, 08 Apr 2018 10:13:52 GMT): Luke_Chen (Sun, 08 Apr 2018 10:13:52 GMT): ydk210999 (Sun, 08 Apr 2018 10:20:37 GMT): ydk210999 (Sun, 08 Apr 2018 10:20:47 GMT): ydk210999 (Sun, 08 Apr 2018 10:23:05 GMT): ydk210999 (Sun, 08 Apr 2018 10:28:00 GMT): ydk210999 (Sun, 08 Apr 2018 10:32:31 GMT): baohua (Mon, 09 Apr 2018 01:21:15 GMT): dinesh.rivankar (Mon, 09 Apr 2018 04:22:04 GMT): rjones (Mon, 09 Apr 2018 04:37:48 GMT): combin (Mon, 09 Apr 2018 09:44:15 GMT): combin (Mon, 09 Apr 2018 09:48:11 GMT): combin (Mon, 09 Apr 2018 09:48:11 GMT): combin (Mon, 09 Apr 2018 09:49:16 GMT): rogeriofza (Mon, 09 Apr 2018 11:30:33 GMT): tongli (Mon, 09 Apr 2018 13:04:27 GMT): tongli (Mon, 09 Apr 2018 13:05:01 GMT): baohua (Mon, 09 Apr 2018 13:06:13 GMT): tongli (Mon, 09 Apr 2018 13:06:59 GMT): tongli (Mon, 09 Apr 2018 13:07:14 GMT): baohua (Mon, 09 Apr 2018 13:17:24 GMT): rameshthoomu (Mon, 09 Apr 2018 16:05:53 GMT): rameshthoomu (Mon, 09 Apr 2018 16:05:53 GMT): tongli (Mon, 09 Apr 2018 17:29:48 GMT): tongli (Mon, 09 Apr 2018 17:30:23 GMT): rameshthoomu (Mon, 09 Apr 2018 17:33:17 GMT): baohua (Tue, 10 Apr 2018 01:22:52 GMT): baohua (Tue, 10 Apr 2018 01:25:20 GMT): antitoine (Tue, 10 Apr 2018 09:50:20 GMT): jverhoelen (Tue, 10 Apr 2018 19:03:05 GMT): hightall (Wed, 11 Apr 2018 03:41:05 GMT): baohua (Wed, 11 Apr 2018 05:01:08 GMT): eramitg (Wed, 11 Apr 2018 15:41:02 GMT): DanHendricks (Thu, 12 Apr 2018 02:51:14 GMT): pankajcheema (Thu, 12 Apr 2018 07:39:46 GMT): pankajcheema (Thu, 12 Apr 2018 07:40:19 GMT): pankajcheema (Thu, 12 Apr 2018 07:40:41 GMT): vikramjit (Thu, 12 Apr 2018 09:27:32 GMT): Luke_Chen (Thu, 12 Apr 2018 10:40:33 GMT): baohua (Thu, 12 Apr 2018 10:44:57 GMT): Luke_Chen (Thu, 12 Apr 2018 10:45:56 GMT): hightall (Thu, 12 Apr 2018 12:51:09 GMT): baohua (Thu, 12 Apr 2018 14:03:02 GMT): jerry-zww (Thu, 12 Apr 2018 15:25:03 GMT): Preetam007 (Thu, 12 Apr 2018 16:18:54 GMT): hightall (Fri, 13 Apr 2018 02:32:05 GMT): Blaise 4 (Fri, 13 Apr 2018 03:47:23 GMT): baohua (Fri, 13 Apr 2018 04:56:57 GMT): Luke_Chen (Fri, 13 Apr 2018 05:05:05 GMT): hightall (Fri, 13 Apr 2018 05:05:31 GMT): Luke_Chen (Fri, 13 Apr 2018 05:05:36 GMT): Luke_Chen (Fri, 13 Apr 2018 07:39:02 GMT): hightall (Fri, 13 Apr 2018 07:39:34 GMT): Luke_Chen (Fri, 13 Apr 2018 07:45:04 GMT): hightall (Fri, 13 Apr 2018 07:45:25 GMT): Luke_Chen (Fri, 13 Apr 2018 07:46:44 GMT): baohua (Fri, 13 Apr 2018 12:32:09 GMT): tongli (Fri, 13 Apr 2018 12:51:06 GMT): baohua (Fri, 13 Apr 2018 12:53:36 GMT): Luke_Chen (Fri, 13 Apr 2018 13:43:06 GMT): Luke_Chen (Fri, 13 Apr 2018 13:45:13 GMT): eramitg (Fri, 13 Apr 2018 17:57:49 GMT): tian (Fri, 13 Apr 2018 22:45:57 GMT): hightall (Sat, 14 Apr 2018 05:07:42 GMT): baohua (Sat, 14 Apr 2018 11:08:44 GMT): baohua (Sat, 14 Apr 2018 11:08:44 GMT): eramitg (Sat, 14 Apr 2018 11:09:12 GMT): nmrk (Sat, 14 Apr 2018 11:29:16 GMT): Luke_Chen (Sun, 15 Apr 2018 04:36:22 GMT): rah_acc (Sun, 15 Apr 2018 06:10:22 GMT): Luke_Chen (Sun, 15 Apr 2018 06:30:55 GMT): baohua (Mon, 16 Apr 2018 02:51:36 GMT): mageover (Mon, 16 Apr 2018 07:52:08 GMT): harsha (Mon, 16 Apr 2018 08:23:00 GMT): rice (Mon, 16 Apr 2018 08:44:29 GMT): baohua (Mon, 16 Apr 2018 09:21:50 GMT): hightall (Mon, 16 Apr 2018 14:41:31 GMT): john.d.sheehan (Mon, 16 Apr 2018 16:16:08 GMT): john.d.sheehan (Mon, 16 Apr 2018 16:31:29 GMT): john.d.sheehan (Mon, 16 Apr 2018 16:31:29 GMT): tongli (Mon, 16 Apr 2018 16:44:29 GMT): tongli (Mon, 16 Apr 2018 16:44:44 GMT): john.d.sheehan (Mon, 16 Apr 2018 16:47:45 GMT): tongli (Mon, 16 Apr 2018 16:48:22 GMT): tongli (Mon, 16 Apr 2018 16:48:46 GMT): john.d.sheehan (Mon, 16 Apr 2018 16:55:22 GMT): tongli (Mon, 16 Apr 2018 16:55:38 GMT): tgehrke (Mon, 16 Apr 2018 19:58:44 GMT): frustrated-chain (Mon, 16 Apr 2018 21:29:51 GMT): frustrated-chain (Mon, 16 Apr 2018 22:12:57 GMT): frustrated-chain (Mon, 16 Apr 2018 22:24:04 GMT): frustrated-chain (Mon, 16 Apr 2018 22:24:19 GMT): frustrated-chain (Mon, 16 Apr 2018 22:24:37 GMT): baohua (Tue, 17 Apr 2018 01:32:26 GMT): hightall (Tue, 17 Apr 2018 01:33:59 GMT): baohua (Tue, 17 Apr 2018 01:34:13 GMT): hightall (Tue, 17 Apr 2018 01:45:08 GMT): Luke_Chen (Tue, 17 Apr 2018 04:38:03 GMT): baohua (Tue, 17 Apr 2018 04:45:00 GMT): Luke_Chen (Tue, 17 Apr 2018 04:54:03 GMT): Luke_Chen (Tue, 17 Apr 2018 04:58:28 GMT): PavanRamesh (Tue, 17 Apr 2018 05:52:31 GMT): PavanRamesh (Tue, 17 Apr 2018 06:38:19 GMT): Luke_Chen (Tue, 17 Apr 2018 06:51:01 GMT): PavanRamesh (Tue, 17 Apr 2018 08:15:32 GMT): PavanRamesh (Tue, 17 Apr 2018 08:15:58 GMT): baohua (Tue, 17 Apr 2018 09:17:11 GMT): Luke_Chen (Tue, 17 Apr 2018 09:55:29 GMT): mvillalobos (Tue, 17 Apr 2018 12:00:11 GMT): mvillalobos (Tue, 17 Apr 2018 12:01:39 GMT): mvillalobos (Tue, 17 Apr 2018 12:02:57 GMT): mvillalobos (Tue, 17 Apr 2018 12:04:15 GMT): mvillalobos (Tue, 17 Apr 2018 12:04:51 GMT): mvillalobos (Tue, 17 Apr 2018 12:05:07 GMT): Senthil1 (Tue, 17 Apr 2018 13:36:32 GMT): baohua (Tue, 17 Apr 2018 14:55:19 GMT): frustrated-chain (Tue, 17 Apr 2018 15:54:47 GMT): frustrated-chain (Tue, 17 Apr 2018 15:54:47 GMT): patrickgardella (Tue, 17 Apr 2018 16:39:57 GMT): patrickgardella (Tue, 17 Apr 2018 16:42:58 GMT): vudathasaiomkar (Tue, 17 Apr 2018 17:40:24 GMT): Blaise 4 (Tue, 17 Apr 2018 17:41:54 GMT): frustrated-chain (Tue, 17 Apr 2018 21:40:41 GMT): frustrated-chain (Tue, 17 Apr 2018 21:43:17 GMT): frustrated-chain (Tue, 17 Apr 2018 21:43:17 GMT): rjones (Tue, 17 Apr 2018 22:41:40 GMT): rjones (Tue, 17 Apr 2018 22:41:46 GMT): walmon (Wed, 18 Apr 2018 01:03:47 GMT): baohua (Wed, 18 Apr 2018 01:07:34 GMT): rjones (Wed, 18 Apr 2018 01:07:49 GMT): rjones (Wed, 18 Apr 2018 01:07:57 GMT): rjones (Wed, 18 Apr 2018 01:08:14 GMT): rjones (Wed, 18 Apr 2018 01:08:39 GMT): rjones (Wed, 18 Apr 2018 01:08:44 GMT): baohua (Wed, 18 Apr 2018 01:50:52 GMT): baohua (Wed, 18 Apr 2018 01:53:30 GMT): baohua (Wed, 18 Apr 2018 01:53:30 GMT): rjones (Wed, 18 Apr 2018 01:54:22 GMT): baohua (Wed, 18 Apr 2018 01:54:26 GMT): baohua (Wed, 18 Apr 2018 01:54:26 GMT): baohua (Wed, 18 Apr 2018 01:54:40 GMT): rjones (Wed, 18 Apr 2018 01:54:47 GMT): rjones (Wed, 18 Apr 2018 01:54:58 GMT): baohua (Wed, 18 Apr 2018 01:55:29 GMT): Luke_Chen (Wed, 18 Apr 2018 03:28:01 GMT): baohua (Wed, 18 Apr 2018 04:25:12 GMT): Luke_Chen (Wed, 18 Apr 2018 04:26:22 GMT): tongli (Wed, 18 Apr 2018 12:27:58 GMT): tongli (Wed, 18 Apr 2018 12:28:39 GMT): Amir (Wed, 18 Apr 2018 14:39:21 GMT): Amir (Wed, 18 Apr 2018 14:40:12 GMT): patrickgardella (Wed, 18 Apr 2018 17:20:22 GMT): patrickgardella (Wed, 18 Apr 2018 18:47:25 GMT): patrickgardella (Wed, 18 Apr 2018 20:17:20 GMT): tgehrke (Wed, 18 Apr 2018 21:23:27 GMT): mpiekarska (Wed, 18 Apr 2018 21:23:28 GMT): hightall (Thu, 19 Apr 2018 01:28:04 GMT): baohua (Thu, 19 Apr 2018 01:32:47 GMT): baohua (Thu, 19 Apr 2018 03:35:31 GMT): dampuero (Thu, 19 Apr 2018 07:05:57 GMT): baohua (Thu, 19 Apr 2018 07:23:30 GMT): baohua (Thu, 19 Apr 2018 08:33:59 GMT): acombeau (Thu, 19 Apr 2018 12:39:24 GMT): acombeau (Thu, 19 Apr 2018 12:42:16 GMT): acombeau (Thu, 19 Apr 2018 12:42:53 GMT): hightall (Thu, 19 Apr 2018 15:20:37 GMT): jorgego (Thu, 19 Apr 2018 19:21:15 GMT): tgehrke (Fri, 20 Apr 2018 00:00:32 GMT): tgehrke (Fri, 20 Apr 2018 00:00:32 GMT): baohua (Fri, 20 Apr 2018 01:11:06 GMT): baohua (Fri, 20 Apr 2018 01:11:16 GMT): tgehrke (Fri, 20 Apr 2018 01:12:11 GMT): tgehrke (Fri, 20 Apr 2018 01:12:11 GMT): baohua (Fri, 20 Apr 2018 01:13:47 GMT): baohua (Fri, 20 Apr 2018 01:36:00 GMT): rjones (Fri, 20 Apr 2018 02:23:30 GMT): remmeier (Fri, 20 Apr 2018 06:35:12 GMT): a.ochs (Fri, 20 Apr 2018 13:06:40 GMT): a.ochs (Fri, 20 Apr 2018 13:07:03 GMT): harsha (Fri, 20 Apr 2018 13:19:11 GMT): harsha (Fri, 20 Apr 2018 13:19:25 GMT): a.ochs (Fri, 20 Apr 2018 13:32:29 GMT): acombeau (Fri, 20 Apr 2018 14:18:50 GMT): acombeau (Fri, 20 Apr 2018 14:18:50 GMT): jorgego (Fri, 20 Apr 2018 20:00:23 GMT): jorgego (Fri, 20 Apr 2018 20:00:23 GMT): jorgego (Fri, 20 Apr 2018 20:00:23 GMT): jorgego (Fri, 20 Apr 2018 20:00:23 GMT): jorgego (Fri, 20 Apr 2018 20:00:23 GMT): jorgego (Fri, 20 Apr 2018 20:00:23 GMT): jorgego (Fri, 20 Apr 2018 20:00:23 GMT): ShavkatAynurin (Fri, 20 Apr 2018 20:46:21 GMT): giggle (Sat, 21 Apr 2018 01:53:19 GMT): maqiutian (Sat, 21 Apr 2018 15:26:56 GMT): sunnrunner (Sat, 21 Apr 2018 23:06:03 GMT): lujiang (Sun, 22 Apr 2018 09:31:25 GMT): sstercan (Sun, 22 Apr 2018 09:35:15 GMT): hightall (Mon, 23 Apr 2018 01:59:59 GMT): baohua (Mon, 23 Apr 2018 04:32:55 GMT): giovanniconte (Mon, 23 Apr 2018 10:11:54 GMT): miahfost (Mon, 23 Apr 2018 14:44:46 GMT): tgehrke (Mon, 23 Apr 2018 22:41:05 GMT): tgehrke (Mon, 23 Apr 2018 22:41:05 GMT): baohua (Tue, 24 Apr 2018 00:39:32 GMT): baohua (Tue, 24 Apr 2018 00:40:00 GMT): baohua (Tue, 24 Apr 2018 00:40:47 GMT): tgehrke (Tue, 24 Apr 2018 01:37:47 GMT): tgehrke (Tue, 24 Apr 2018 01:37:47 GMT): tgehrke (Tue, 24 Apr 2018 01:40:49 GMT): tgehrke (Tue, 24 Apr 2018 02:31:20 GMT): baohua (Tue, 24 Apr 2018 03:15:56 GMT): baohua (Tue, 24 Apr 2018 03:17:12 GMT): baohua (Tue, 24 Apr 2018 03:18:26 GMT): tgehrke (Tue, 24 Apr 2018 04:22:17 GMT): baohua (Tue, 24 Apr 2018 04:46:18 GMT): Luke_Chen (Tue, 24 Apr 2018 07:10:16 GMT): Luke_Chen (Tue, 24 Apr 2018 07:16:41 GMT): Luke_Chen (Tue, 24 Apr 2018 07:17:50 GMT): Luke_Chen (Tue, 24 Apr 2018 07:19:04 GMT): baohua (Tue, 24 Apr 2018 08:27:58 GMT): Luke_Chen (Tue, 24 Apr 2018 08:28:14 GMT): baohua (Tue, 24 Apr 2018 08:28:40 GMT): baohua (Tue, 24 Apr 2018 08:28:50 GMT): Luke_Chen (Tue, 24 Apr 2018 08:29:00 GMT): baohua (Tue, 24 Apr 2018 08:29:25 GMT): baohua (Tue, 24 Apr 2018 08:29:37 GMT): Luke_Chen (Tue, 24 Apr 2018 09:00:50 GMT): Luke_Chen (Tue, 24 Apr 2018 09:34:29 GMT): Luke_Chen (Tue, 24 Apr 2018 09:37:14 GMT): Luke_Chen (Tue, 24 Apr 2018 09:37:54 GMT): alkopnin (Tue, 24 Apr 2018 14:07:19 GMT): Luke_Chen (Wed, 25 Apr 2018 02:05:13 GMT): Jayrz (Wed, 25 Apr 2018 02:20:50 GMT): hightall (Wed, 25 Apr 2018 05:15:24 GMT): baohua (Wed, 25 Apr 2018 06:39:34 GMT): baohua (Wed, 25 Apr 2018 06:39:55 GMT): baohua (Wed, 25 Apr 2018 06:40:18 GMT): baohua (Wed, 25 Apr 2018 06:40:41 GMT): hightall (Wed, 25 Apr 2018 06:40:55 GMT): Luke_Chen (Wed, 25 Apr 2018 07:31:28 GMT): baohua (Wed, 25 Apr 2018 07:31:36 GMT): acombeau (Wed, 25 Apr 2018 08:02:34 GMT): acombeau (Wed, 25 Apr 2018 08:02:34 GMT): sillysachin (Wed, 25 Apr 2018 10:20:22 GMT): sillysachin (Wed, 25 Apr 2018 10:20:38 GMT): zwx00 (Wed, 25 Apr 2018 13:05:43 GMT): zwx00 (Wed, 25 Apr 2018 13:07:28 GMT): zwx00 (Wed, 25 Apr 2018 13:10:10 GMT): baohua (Wed, 25 Apr 2018 13:13:09 GMT): baohua (Wed, 25 Apr 2018 13:13:26 GMT): zwx00 (Wed, 25 Apr 2018 13:13:28 GMT): zwx00 (Wed, 25 Apr 2018 13:13:43 GMT): baohua (Wed, 25 Apr 2018 13:14:23 GMT): baohua (Wed, 25 Apr 2018 13:14:45 GMT): zwx00 (Wed, 25 Apr 2018 13:17:33 GMT): zwx00 (Wed, 25 Apr 2018 13:17:38 GMT): tgehrke (Wed, 25 Apr 2018 19:04:35 GMT): tgehrke (Wed, 25 Apr 2018 19:04:35 GMT): Luke_Chen (Thu, 26 Apr 2018 02:09:23 GMT): Luke_Chen (Thu, 26 Apr 2018 02:11:53 GMT): IVictorFeng (Thu, 26 Apr 2018 02:30:04 GMT): SunnyBeanLove (Thu, 26 Apr 2018 02:32:53 GMT): baohua (Thu, 26 Apr 2018 03:07:19 GMT): tgehrke (Thu, 26 Apr 2018 06:43:14 GMT): baohua (Thu, 26 Apr 2018 06:44:16 GMT): baohua (Thu, 26 Apr 2018 06:44:22 GMT): tgehrke (Thu, 26 Apr 2018 06:44:34 GMT): tgehrke (Thu, 26 Apr 2018 06:44:34 GMT): Saschaka (Thu, 26 Apr 2018 07:24:53 GMT): Saschaka (Thu, 26 Apr 2018 07:27:58 GMT): Saschaka (Thu, 26 Apr 2018 07:28:21 GMT): Saschaka (Thu, 26 Apr 2018 07:48:34 GMT): Saschaka (Thu, 26 Apr 2018 07:49:05 GMT): Saschaka (Thu, 26 Apr 2018 07:56:37 GMT): Saschaka (Thu, 26 Apr 2018 07:57:04 GMT): maestrus (Thu, 26 Apr 2018 08:28:51 GMT): Saschaka (Thu, 26 Apr 2018 08:32:20 GMT): baohua (Thu, 26 Apr 2018 11:27:12 GMT): asaningmaxchain123 (Thu, 26 Apr 2018 13:56:19 GMT): asaningmaxchain123 (Thu, 26 Apr 2018 13:57:03 GMT): asaningmaxchain123 (Thu, 26 Apr 2018 13:57:03 GMT): baohua (Thu, 26 Apr 2018 13:57:26 GMT): baohua (Thu, 26 Apr 2018 13:57:34 GMT): hightall (Thu, 26 Apr 2018 13:58:12 GMT): baohua (Thu, 26 Apr 2018 13:58:16 GMT): baohua (Thu, 26 Apr 2018 13:58:30 GMT): baohua (Thu, 26 Apr 2018 13:58:45 GMT): asaningmaxchain123 (Thu, 26 Apr 2018 13:59:35 GMT): asaningmaxchain123 (Thu, 26 Apr 2018 13:59:35 GMT): baohua (Thu, 26 Apr 2018 14:00:44 GMT): asaningmaxchain123 (Thu, 26 Apr 2018 14:01:20 GMT): asaningmaxchain123 (Thu, 26 Apr 2018 14:03:13 GMT): baohua (Thu, 26 Apr 2018 14:06:27 GMT): baohua (Thu, 26 Apr 2018 14:38:08 GMT): coffeeplease (Thu, 26 Apr 2018 15:31:04 GMT): coffeeplease (Thu, 26 Apr 2018 15:36:13 GMT): coffeeplease (Thu, 26 Apr 2018 15:38:43 GMT): coffeeplease (Thu, 26 Apr 2018 16:20:51 GMT): coffeeplease (Thu, 26 Apr 2018 16:25:58 GMT): baohua (Thu, 26 Apr 2018 21:56:41 GMT): Luke_Chen (Fri, 27 Apr 2018 03:24:50 GMT): baohua (Fri, 27 Apr 2018 06:11:51 GMT): xiaozhongqing (Fri, 27 Apr 2018 08:47:03 GMT): samarcho (Fri, 27 Apr 2018 14:30:35 GMT): asaningmaxchain123 (Sat, 28 Apr 2018 01:07:12 GMT): asaningmaxchain123 (Sat, 28 Apr 2018 01:07:12 GMT): careck (Sat, 28 Apr 2018 06:23:45 GMT): careck (Sat, 28 Apr 2018 06:25:28 GMT): careck (Sat, 28 Apr 2018 06:25:44 GMT): careck (Sat, 28 Apr 2018 06:26:25 GMT): careck (Sat, 28 Apr 2018 06:27:03 GMT): careck (Sat, 28 Apr 2018 07:12:43 GMT): careck (Sat, 28 Apr 2018 23:47:28 GMT): careck (Sat, 28 Apr 2018 23:48:01 GMT): careck (Sat, 28 Apr 2018 23:48:15 GMT): careck (Sun, 29 Apr 2018 00:19:37 GMT): careck (Sun, 29 Apr 2018 02:00:48 GMT): waaagh (Sun, 29 Apr 2018 02:17:53 GMT): waaagh (Sun, 29 Apr 2018 02:18:10 GMT): waaagh (Sun, 29 Apr 2018 02:18:47 GMT): waaagh (Sun, 29 Apr 2018 02:18:47 GMT): waaagh (Sun, 29 Apr 2018 02:20:18 GMT): waaagh (Sun, 29 Apr 2018 02:26:38 GMT): hightall (Sun, 29 Apr 2018 11:36:14 GMT): umtyzc (Sun, 29 Apr 2018 13:53:17 GMT): RameshKunhiraman (Mon, 30 Apr 2018 09:48:39 GMT): ChrisFowers (Tue, 01 May 2018 14:35:46 GMT): ChrisFowers (Tue, 01 May 2018 14:35:56 GMT): ChrisFowers (Tue, 01 May 2018 14:36:04 GMT): ChrisFowers (Tue, 01 May 2018 14:43:27 GMT): ChrisFowers (Tue, 01 May 2018 14:54:06 GMT): ChrisFowers (Tue, 01 May 2018 16:36:38 GMT): ChrisFowers (Tue, 01 May 2018 16:55:00 GMT): ChrisFowers (Tue, 01 May 2018 16:55:04 GMT): ChrisFowers (Tue, 01 May 2018 16:55:53 GMT): ChrisFowers (Tue, 01 May 2018 17:25:05 GMT): tongli (Tue, 01 May 2018 17:53:25 GMT): tongli (Tue, 01 May 2018 17:53:34 GMT): tongli (Tue, 01 May 2018 17:54:11 GMT): ChrisFowers (Tue, 01 May 2018 22:46:48 GMT): ChrisFowers (Tue, 01 May 2018 22:48:26 GMT): ChrisFowers (Tue, 01 May 2018 22:48:26 GMT): ChrisFowers (Tue, 01 May 2018 22:48:48 GMT): ChrisFowers (Tue, 01 May 2018 22:49:12 GMT): ChrisFowers (Tue, 01 May 2018 22:49:40 GMT): ChrisFowers (Tue, 01 May 2018 22:49:44 GMT): ChrisFowers (Tue, 01 May 2018 22:54:27 GMT): ChrisFowers (Tue, 01 May 2018 22:54:50 GMT): ChrisFowers (Tue, 01 May 2018 22:56:09 GMT): ChrisFowers (Tue, 01 May 2018 22:56:25 GMT): ChrisFowers (Tue, 01 May 2018 22:56:56 GMT): ChrisFowers (Tue, 01 May 2018 22:57:27 GMT): ChrisFowers (Tue, 01 May 2018 22:57:45 GMT): ChrisFowers (Tue, 01 May 2018 22:58:40 GMT): tongli (Tue, 01 May 2018 23:31:38 GMT): tongli (Tue, 01 May 2018 23:31:41 GMT): tongli (Tue, 01 May 2018 23:31:56 GMT): tongli (Tue, 01 May 2018 23:32:19 GMT): tongli (Tue, 01 May 2018 23:32:32 GMT): tongli (Tue, 01 May 2018 23:32:50 GMT): tongli (Tue, 01 May 2018 23:33:01 GMT): tongli (Tue, 01 May 2018 23:34:00 GMT): tongli (Tue, 01 May 2018 23:34:36 GMT): tongli (Tue, 01 May 2018 23:35:14 GMT): tongli (Tue, 01 May 2018 23:35:26 GMT): tongli (Tue, 01 May 2018 23:35:43 GMT): tongli (Tue, 01 May 2018 23:35:57 GMT): PirangPhan (Wed, 02 May 2018 02:28:39 GMT): Luke_Chen (Wed, 02 May 2018 02:55:45 GMT): hxphung (Wed, 02 May 2018 09:27:03 GMT): hxphung (Wed, 02 May 2018 09:29:13 GMT): tongli (Wed, 02 May 2018 13:41:37 GMT): tongli (Wed, 02 May 2018 13:41:40 GMT): ChrisFowers (Wed, 02 May 2018 14:08:57 GMT): kly4 (Wed, 02 May 2018 17:23:20 GMT): carlcraig (Wed, 02 May 2018 20:53:56 GMT): hightall (Thu, 03 May 2018 01:45:32 GMT): hightall (Thu, 03 May 2018 01:45:56 GMT): baohua (Thu, 03 May 2018 02:11:26 GMT): tongli (Thu, 03 May 2018 02:15:51 GMT): kiranarshakota (Thu, 03 May 2018 04:35:55 GMT): kiranarshakota (Thu, 03 May 2018 04:37:54 GMT): kiranarshakota (Thu, 03 May 2018 04:37:56 GMT): kiranarshakota (Thu, 03 May 2018 04:38:17 GMT): Gvedam (Thu, 03 May 2018 04:41:52 GMT): bjwswang (Thu, 03 May 2018 06:10:28 GMT): bjwswang (Thu, 03 May 2018 06:17:08 GMT): bjwswang (Thu, 03 May 2018 06:43:38 GMT): kiranarshakota (Thu, 03 May 2018 07:19:21 GMT): Sushilpaneru (Thu, 03 May 2018 07:28:51 GMT): baohua (Thu, 03 May 2018 13:45:49 GMT): baohua (Thu, 03 May 2018 13:45:55 GMT): kiranarshakota (Thu, 03 May 2018 13:53:31 GMT): baohua (Thu, 03 May 2018 13:54:03 GMT): baohua (Thu, 03 May 2018 13:54:28 GMT): baohua (Thu, 03 May 2018 13:56:03 GMT): kiranarshakota (Thu, 03 May 2018 13:56:25 GMT): baohua (Thu, 03 May 2018 13:56:46 GMT): kiranarshakota (Thu, 03 May 2018 13:56:54 GMT): baohua (Thu, 03 May 2018 13:56:59 GMT): kiranarshakota (Thu, 03 May 2018 13:57:13 GMT): baohua (Thu, 03 May 2018 13:57:26 GMT): kiranarshakota (Thu, 03 May 2018 14:00:00 GMT): kiranarshakota (Thu, 03 May 2018 14:00:19 GMT): kiranarshakota (Thu, 03 May 2018 14:02:34 GMT): kiranarshakota (Thu, 03 May 2018 14:03:04 GMT): baohua (Thu, 03 May 2018 14:03:27 GMT): baohua (Thu, 03 May 2018 14:03:44 GMT): kiranarshakota (Thu, 03 May 2018 14:04:10 GMT): kiranarshakota (Thu, 03 May 2018 14:07:03 GMT): baohua (Thu, 03 May 2018 14:07:31 GMT): baohua (Thu, 03 May 2018 14:08:07 GMT): kiranarshakota (Thu, 03 May 2018 14:09:54 GMT): kiranarshakota (Thu, 03 May 2018 14:13:20 GMT): baohua (Thu, 03 May 2018 14:13:57 GMT): baohua (Thu, 03 May 2018 14:14:01 GMT): baohua (Thu, 03 May 2018 14:14:12 GMT): kiranarshakota (Thu, 03 May 2018 14:23:05 GMT): kiranarshakota (Thu, 03 May 2018 14:23:18 GMT): baohua (Thu, 03 May 2018 14:23:51 GMT): rjones (Thu, 03 May 2018 15:05:40 GMT): rjones (Thu, 03 May 2018 15:06:02 GMT): rjones (Thu, 03 May 2018 15:08:09 GMT): rjones (Thu, 03 May 2018 15:08:51 GMT): tongli (Thu, 03 May 2018 15:09:56 GMT): tongli (Thu, 03 May 2018 15:10:20 GMT): rjones (Thu, 03 May 2018 15:10:23 GMT): tongli (Thu, 03 May 2018 15:11:18 GMT): rjones (Thu, 03 May 2018 15:11:56 GMT): rjones (Thu, 03 May 2018 15:12:23 GMT): tongli (Thu, 03 May 2018 15:12:36 GMT): rjones (Thu, 03 May 2018 15:13:06 GMT): rjones (Thu, 03 May 2018 15:13:10 GMT): rjones (Thu, 03 May 2018 15:45:21 GMT): tongli (Thu, 03 May 2018 21:38:47 GMT): baohua (Fri, 04 May 2018 01:55:49 GMT): baohua (Fri, 04 May 2018 01:56:14 GMT): baohua (Fri, 04 May 2018 02:02:39 GMT): tongli (Fri, 04 May 2018 02:26:33 GMT): harsha (Fri, 04 May 2018 04:29:34 GMT): harsha (Fri, 04 May 2018 04:29:34 GMT): harsha (Fri, 04 May 2018 04:29:34 GMT): harsha (Fri, 04 May 2018 04:29:34 GMT): baohua (Fri, 04 May 2018 11:39:04 GMT): baohua (Fri, 04 May 2018 11:40:23 GMT): tongli (Fri, 04 May 2018 12:29:06 GMT): hightall (Fri, 04 May 2018 12:30:14 GMT): tongli (Fri, 04 May 2018 12:30:56 GMT): baohua (Fri, 04 May 2018 12:57:00 GMT): hightall (Fri, 04 May 2018 13:38:54 GMT): huanim (Sat, 05 May 2018 06:19:57 GMT): careck (Sun, 06 May 2018 00:56:48 GMT): Luke_Chen (Sun, 06 May 2018 03:30:55 GMT): pravindia (Sun, 06 May 2018 17:06:34 GMT): Luke_Chen (Mon, 07 May 2018 02:53:19 GMT): Luke_Chen (Mon, 07 May 2018 02:53:19 GMT): Luke_Chen (Mon, 07 May 2018 02:53:19 GMT): Luke_Chen (Mon, 07 May 2018 02:54:04 GMT): Luke_Chen (Mon, 07 May 2018 02:54:04 GMT): hightall (Mon, 07 May 2018 02:54:18 GMT): Luke_Chen (Mon, 07 May 2018 02:54:46 GMT): Luke_Chen (Mon, 07 May 2018 02:55:05 GMT): giniz (Mon, 07 May 2018 02:55:32 GMT): hightall (Mon, 07 May 2018 02:55:59 GMT): Luke_Chen (Mon, 07 May 2018 02:56:09 GMT): giniz (Mon, 07 May 2018 02:57:09 GMT): giniz (Mon, 07 May 2018 02:57:35 GMT): Luke_Chen (Mon, 07 May 2018 03:00:15 GMT): Luke_Chen (Mon, 07 May 2018 03:00:16 GMT): giniz (Mon, 07 May 2018 03:03:17 GMT): giniz (Mon, 07 May 2018 03:19:52 GMT): giniz (Mon, 07 May 2018 03:20:20 GMT): giniz (Mon, 07 May 2018 03:20:27 GMT): giniz (Mon, 07 May 2018 03:21:01 GMT): giniz (Mon, 07 May 2018 03:21:16 GMT): giniz (Mon, 07 May 2018 03:21:24 GMT): giniz (Mon, 07 May 2018 03:21:24 GMT): giniz (Mon, 07 May 2018 03:21:24 GMT): giniz (Mon, 07 May 2018 04:38:02 GMT): giniz (Mon, 07 May 2018 04:38:02 GMT): giniz (Mon, 07 May 2018 04:39:24 GMT): giniz (Mon, 07 May 2018 04:51:13 GMT): giniz (Mon, 07 May 2018 04:51:32 GMT): giniz (Mon, 07 May 2018 04:52:01 GMT): giniz (Mon, 07 May 2018 04:52:09 GMT): giniz (Mon, 07 May 2018 04:52:49 GMT): giniz (Mon, 07 May 2018 04:52:49 GMT): giniz (Mon, 07 May 2018 04:54:49 GMT): Luke_Chen (Mon, 07 May 2018 05:19:40 GMT): Luke_Chen (Mon, 07 May 2018 05:20:15 GMT): giniz (Mon, 07 May 2018 05:23:24 GMT): giniz (Mon, 07 May 2018 05:24:02 GMT): giniz (Mon, 07 May 2018 05:24:29 GMT): Luke_Chen (Mon, 07 May 2018 05:31:44 GMT): Luke_Chen (Mon, 07 May 2018 05:32:03 GMT): Luke_Chen (Mon, 07 May 2018 05:32:50 GMT): giniz (Mon, 07 May 2018 05:33:07 GMT): giniz (Mon, 07 May 2018 05:34:05 GMT): Luke_Chen (Mon, 07 May 2018 05:34:16 GMT): giniz (Mon, 07 May 2018 05:35:12 GMT): giniz (Mon, 07 May 2018 05:39:56 GMT): Luke_Chen (Mon, 07 May 2018 05:40:09 GMT): giniz (Mon, 07 May 2018 05:40:13 GMT): giniz (Mon, 07 May 2018 05:40:19 GMT): giniz (Mon, 07 May 2018 05:40:37 GMT): Luke_Chen (Mon, 07 May 2018 05:40:55 GMT): giniz (Mon, 07 May 2018 05:41:47 GMT): giniz (Mon, 07 May 2018 05:41:51 GMT): giniz (Mon, 07 May 2018 05:43:07 GMT): Luke_Chen (Mon, 07 May 2018 05:43:56 GMT): giniz (Mon, 07 May 2018 05:44:26 GMT): giniz (Mon, 07 May 2018 05:44:26 GMT): giniz (Mon, 07 May 2018 05:46:02 GMT): giniz (Mon, 07 May 2018 05:46:54 GMT): Luke_Chen (Mon, 07 May 2018 05:59:03 GMT): giniz (Mon, 07 May 2018 06:00:16 GMT): Luke_Chen (Mon, 07 May 2018 07:22:45 GMT): Luke_Chen (Mon, 07 May 2018 07:22:45 GMT): giniz (Mon, 07 May 2018 07:40:27 GMT): giniz (Mon, 07 May 2018 07:40:33 GMT): giniz (Mon, 07 May 2018 07:41:10 GMT): giniz (Mon, 07 May 2018 07:41:37 GMT): Luke_Chen (Mon, 07 May 2018 07:42:55 GMT): Luke_Chen (Mon, 07 May 2018 07:43:01 GMT): giniz (Mon, 07 May 2018 07:43:06 GMT): Luke_Chen (Mon, 07 May 2018 07:44:07 GMT): giniz (Mon, 07 May 2018 07:45:33 GMT): Luke_Chen (Mon, 07 May 2018 07:46:47 GMT): giniz (Mon, 07 May 2018 07:49:44 GMT): giniz (Mon, 07 May 2018 08:32:52 GMT): giniz (Mon, 07 May 2018 08:32:52 GMT): giniz (Mon, 07 May 2018 08:33:18 GMT): giniz (Mon, 07 May 2018 08:33:21 GMT): giniz (Mon, 07 May 2018 08:33:21 GMT): giniz (Mon, 07 May 2018 08:49:06 GMT): giniz (Mon, 07 May 2018 08:49:06 GMT): giniz (Mon, 07 May 2018 08:49:50 GMT): giniz (Mon, 07 May 2018 08:49:50 GMT): giniz (Mon, 07 May 2018 08:49:50 GMT): giniz (Mon, 07 May 2018 08:56:32 GMT): ArkiZh (Mon, 07 May 2018 12:58:32 GMT): hightall (Mon, 07 May 2018 13:15:07 GMT): hightall (Mon, 07 May 2018 13:32:01 GMT): baohua (Mon, 07 May 2018 13:41:21 GMT): tongli (Mon, 07 May 2018 13:56:19 GMT): hightall (Mon, 07 May 2018 14:20:54 GMT): tongli (Mon, 07 May 2018 14:26:47 GMT): hightall (Mon, 07 May 2018 14:27:12 GMT): baohua (Mon, 07 May 2018 14:36:29 GMT): tongli (Mon, 07 May 2018 14:37:34 GMT): tongli (Mon, 07 May 2018 14:37:53 GMT): baohua (Mon, 07 May 2018 14:38:47 GMT): hightall (Mon, 07 May 2018 14:39:13 GMT): hightall (Mon, 07 May 2018 14:39:40 GMT): baohua (Mon, 07 May 2018 14:40:34 GMT): tongli (Mon, 07 May 2018 14:40:49 GMT): harsha (Mon, 07 May 2018 15:11:33 GMT): harsha (Mon, 07 May 2018 15:11:33 GMT): harsha (Mon, 07 May 2018 15:12:22 GMT): tongli (Mon, 07 May 2018 18:19:46 GMT): tongli (Mon, 07 May 2018 18:19:57 GMT): harsha (Mon, 07 May 2018 18:20:08 GMT): tongli (Mon, 07 May 2018 18:20:52 GMT): tongli (Mon, 07 May 2018 18:21:30 GMT): tongli (Mon, 07 May 2018 18:21:51 GMT): tongli (Mon, 07 May 2018 18:22:01 GMT): harsha (Mon, 07 May 2018 18:38:15 GMT): harsha (Mon, 07 May 2018 18:40:19 GMT): harsha (Mon, 07 May 2018 18:40:19 GMT): harsha (Mon, 07 May 2018 18:40:19 GMT): harsha (Mon, 07 May 2018 18:40:19 GMT): xds2000 (Mon, 07 May 2018 23:47:21 GMT): giniz (Tue, 08 May 2018 01:50:27 GMT): giniz (Tue, 08 May 2018 01:51:03 GMT): giniz (Tue, 08 May 2018 01:51:51 GMT): giniz (Tue, 08 May 2018 01:52:54 GMT): giniz (Tue, 08 May 2018 01:53:19 GMT): giniz (Tue, 08 May 2018 01:53:23 GMT): baohua (Tue, 08 May 2018 02:18:21 GMT): giniz (Tue, 08 May 2018 02:19:10 GMT): giniz (Tue, 08 May 2018 02:20:11 GMT): giniz (Tue, 08 May 2018 02:21:00 GMT): giniz (Tue, 08 May 2018 02:21:24 GMT): giniz (Tue, 08 May 2018 02:22:37 GMT): giniz (Tue, 08 May 2018 02:23:00 GMT): giniz (Tue, 08 May 2018 02:27:23 GMT): giniz (Tue, 08 May 2018 02:27:23 GMT): giniz (Tue, 08 May 2018 02:28:24 GMT): giniz (Tue, 08 May 2018 02:29:06 GMT): giniz (Tue, 08 May 2018 02:31:09 GMT): giniz (Tue, 08 May 2018 02:31:09 GMT): baohua (Tue, 08 May 2018 02:42:39 GMT): baohua (Tue, 08 May 2018 02:43:59 GMT): baohua (Tue, 08 May 2018 02:43:59 GMT): giniz (Tue, 08 May 2018 02:47:22 GMT): baohua (Tue, 08 May 2018 02:48:00 GMT): giniz (Tue, 08 May 2018 02:49:54 GMT): giniz (Tue, 08 May 2018 02:50:34 GMT): giniz (Tue, 08 May 2018 02:50:50 GMT): baohua (Tue, 08 May 2018 02:51:16 GMT): giniz (Tue, 08 May 2018 02:51:57 GMT): baohua (Tue, 08 May 2018 02:56:56 GMT): baohua (Tue, 08 May 2018 02:57:14 GMT): giniz (Tue, 08 May 2018 02:58:08 GMT): giniz (Tue, 08 May 2018 02:59:59 GMT): baohua (Tue, 08 May 2018 03:22:29 GMT): baohua (Tue, 08 May 2018 03:22:49 GMT): giniz (Tue, 08 May 2018 03:27:02 GMT): baohua (Tue, 08 May 2018 03:28:00 GMT): baohua (Tue, 08 May 2018 03:28:32 GMT): hightall (Tue, 08 May 2018 03:31:02 GMT): baohua (Tue, 08 May 2018 03:31:29 GMT): baohua (Tue, 08 May 2018 03:31:29 GMT): hightall (Tue, 08 May 2018 03:31:37 GMT): giniz (Tue, 08 May 2018 04:12:06 GMT): baohua (Tue, 08 May 2018 04:12:34 GMT): giniz (Tue, 08 May 2018 04:12:51 GMT): giniz (Tue, 08 May 2018 07:38:59 GMT): hightall (Tue, 08 May 2018 07:39:34 GMT): giniz (Tue, 08 May 2018 07:40:35 GMT): giniz (Tue, 08 May 2018 07:40:47 GMT): giniz (Tue, 08 May 2018 07:41:22 GMT): giniz (Tue, 08 May 2018 07:51:09 GMT): giniz (Tue, 08 May 2018 07:51:55 GMT): giniz (Tue, 08 May 2018 07:52:22 GMT): hightall (Tue, 08 May 2018 07:54:20 GMT): giniz (Tue, 08 May 2018 07:55:59 GMT): giniz (Tue, 08 May 2018 07:56:26 GMT): giniz (Tue, 08 May 2018 08:13:51 GMT): Saschaka (Tue, 08 May 2018 09:38:03 GMT): acombeau (Tue, 08 May 2018 09:48:57 GMT): acombeau (Tue, 08 May 2018 09:48:57 GMT): vatsal.shah (Tue, 08 May 2018 13:10:50 GMT): tongli (Tue, 08 May 2018 13:51:39 GMT): tongli (Tue, 08 May 2018 13:53:55 GMT): tongli (Tue, 08 May 2018 13:54:06 GMT): kly4 (Tue, 08 May 2018 14:07:47 GMT): hightall (Tue, 08 May 2018 15:31:08 GMT): tongli (Tue, 08 May 2018 15:31:17 GMT): tongli (Tue, 08 May 2018 15:31:58 GMT): hightall (Tue, 08 May 2018 15:43:18 GMT): baohua (Wed, 09 May 2018 00:08:13 GMT): tdespenza (Wed, 09 May 2018 01:27:26 GMT): tdespenza (Wed, 09 May 2018 02:37:18 GMT): baohua (Wed, 09 May 2018 04:23:24 GMT): giniz (Wed, 09 May 2018 06:44:44 GMT): giniz (Wed, 09 May 2018 06:45:41 GMT): giniz (Wed, 09 May 2018 06:46:46 GMT): giniz (Wed, 09 May 2018 06:47:04 GMT): giniz (Wed, 09 May 2018 06:48:55 GMT): baohua (Wed, 09 May 2018 07:31:31 GMT): giniz (Wed, 09 May 2018 07:35:35 GMT): baohua (Wed, 09 May 2018 07:41:31 GMT): baohua (Wed, 09 May 2018 07:42:02 GMT): baohua (Wed, 09 May 2018 07:42:17 GMT): giniz (Wed, 09 May 2018 07:43:46 GMT): giniz (Wed, 09 May 2018 07:43:49 GMT): acombeau (Wed, 09 May 2018 10:24:33 GMT): acombeau (Wed, 09 May 2018 10:24:39 GMT): acombeau (Wed, 09 May 2018 10:24:39 GMT): acombeau (Wed, 09 May 2018 10:24:39 GMT): acombeau (Wed, 09 May 2018 10:24:39 GMT): acombeau (Wed, 09 May 2018 10:24:39 GMT): Gvedam (Wed, 09 May 2018 11:47:15 GMT): Gvedam (Wed, 09 May 2018 11:47:15 GMT): Gvedam (Wed, 09 May 2018 12:39:48 GMT): tongli (Wed, 09 May 2018 14:04:27 GMT): tongli (Wed, 09 May 2018 14:17:57 GMT): hightall (Wed, 09 May 2018 14:19:27 GMT): hightall (Wed, 09 May 2018 14:20:35 GMT): tongli (Wed, 09 May 2018 15:03:04 GMT): tongli (Wed, 09 May 2018 15:03:22 GMT): tongli (Wed, 09 May 2018 15:03:51 GMT): tongli (Wed, 09 May 2018 15:05:13 GMT): tongli (Wed, 09 May 2018 15:05:18 GMT): tongli (Wed, 09 May 2018 15:05:43 GMT): Nicolas 51 (Wed, 09 May 2018 15:10:49 GMT): Nicolas 51 (Wed, 09 May 2018 15:15:36 GMT): Nicolas 51 (Wed, 09 May 2018 15:20:15 GMT): hightall (Wed, 09 May 2018 18:06:27 GMT): FrancoBellagamba (Wed, 09 May 2018 19:50:42 GMT): baohua (Thu, 10 May 2018 00:53:22 GMT): baohua (Thu, 10 May 2018 00:54:30 GMT): giniz (Thu, 10 May 2018 01:59:52 GMT): baohua (Thu, 10 May 2018 04:25:33 GMT): baohua (Thu, 10 May 2018 05:44:14 GMT): baohua (Thu, 10 May 2018 06:06:21 GMT): giniz (Thu, 10 May 2018 06:08:01 GMT): kiranarshakota (Thu, 10 May 2018 06:10:12 GMT): kiranarshakota (Thu, 10 May 2018 06:10:34 GMT): kiranarshakota (Thu, 10 May 2018 06:10:58 GMT): harsha (Thu, 10 May 2018 08:47:44 GMT): baohua (Thu, 10 May 2018 13:52:11 GMT): hightall (Thu, 10 May 2018 14:59:22 GMT): harsha (Thu, 10 May 2018 15:08:35 GMT): baohua (Thu, 10 May 2018 15:26:57 GMT): baohua (Fri, 11 May 2018 01:33:56 GMT): giniz (Fri, 11 May 2018 02:42:07 GMT): DarshanBc (Fri, 11 May 2018 03:41:43 GMT): sgaddam (Fri, 11 May 2018 05:39:31 GMT): Gvedam (Fri, 11 May 2018 07:01:31 GMT): Gvedam (Fri, 11 May 2018 07:01:31 GMT): Gvedam (Fri, 11 May 2018 07:01:31 GMT): Gvedam (Fri, 11 May 2018 07:01:31 GMT): luckydogchina (Fri, 11 May 2018 09:14:17 GMT): luckydogchina (Fri, 11 May 2018 09:14:28 GMT): baohua (Fri, 11 May 2018 09:21:42 GMT): luckydogchina (Fri, 11 May 2018 09:25:32 GMT): luckydogchina (Fri, 11 May 2018 09:25:48 GMT): tongli (Fri, 11 May 2018 17:44:12 GMT): tongli (Fri, 11 May 2018 17:44:59 GMT): tongli (Fri, 11 May 2018 17:45:24 GMT): tongli (Fri, 11 May 2018 17:52:25 GMT): tongli (Fri, 11 May 2018 17:52:58 GMT): cmgabriel (Fri, 11 May 2018 19:49:06 GMT): tallharish (Fri, 11 May 2018 21:57:31 GMT): tdespenza (Sat, 12 May 2018 02:16:02 GMT): tdespenza (Sat, 12 May 2018 02:16:02 GMT): Luke_Chen (Sat, 12 May 2018 02:28:03 GMT): Luke_Chen (Sat, 12 May 2018 02:28:03 GMT): Luke_Chen (Sat, 12 May 2018 02:28:03 GMT): tdespenza (Sat, 12 May 2018 03:09:51 GMT): tdespenza (Sat, 12 May 2018 03:10:31 GMT): Luke_Chen (Sat, 12 May 2018 04:44:49 GMT): tdespenza (Sat, 12 May 2018 11:44:57 GMT): Gvedam (Mon, 14 May 2018 10:14:07 GMT): Gvedam (Mon, 14 May 2018 10:14:07 GMT): tongli (Mon, 14 May 2018 13:44:49 GMT): tongli (Mon, 14 May 2018 13:45:13 GMT): tongli (Mon, 14 May 2018 13:45:31 GMT): tongli (Mon, 14 May 2018 13:46:23 GMT): hightall (Mon, 14 May 2018 13:57:36 GMT): hightall (Mon, 14 May 2018 13:58:00 GMT): hightall (Mon, 14 May 2018 15:38:04 GMT): phulei (Mon, 14 May 2018 18:19:20 GMT): Luke_Chen (Tue, 15 May 2018 01:11:36 GMT): baohua (Tue, 15 May 2018 03:42:32 GMT): tongli (Tue, 15 May 2018 13:55:16 GMT): tongli (Tue, 15 May 2018 13:55:26 GMT): tongli (Tue, 15 May 2018 13:55:37 GMT): tongli (Tue, 15 May 2018 13:56:16 GMT): tgehrke (Tue, 15 May 2018 15:14:46 GMT): mwagner (Tue, 15 May 2018 17:39:43 GMT): JordanDearsley (Tue, 15 May 2018 19:37:42 GMT): JordanDearsley (Tue, 15 May 2018 19:40:10 GMT): JordanDearsley (Tue, 15 May 2018 19:40:10 GMT): JordanDearsley (Tue, 15 May 2018 19:40:10 GMT): JordanDearsley (Tue, 15 May 2018 19:40:10 GMT): ChrisFowers (Tue, 15 May 2018 19:40:17 GMT): JordanDearsley (Tue, 15 May 2018 19:41:51 GMT): JordanDearsley (Tue, 15 May 2018 19:42:05 GMT): psprings (Tue, 15 May 2018 20:29:49 GMT): hightall (Wed, 16 May 2018 01:41:26 GMT): tongli (Wed, 16 May 2018 03:46:42 GMT): Switch2Logic (Wed, 16 May 2018 07:38:56 GMT): h0cin (Wed, 16 May 2018 20:19:11 GMT): baohua (Thu, 17 May 2018 06:18:58 GMT): Gvedam (Thu, 17 May 2018 08:45:13 GMT): Gvedam (Thu, 17 May 2018 08:45:13 GMT): Gvedam (Thu, 17 May 2018 08:45:13 GMT): Gvedam (Thu, 17 May 2018 08:45:13 GMT): Gvedam (Thu, 17 May 2018 08:45:13 GMT): Gvedam (Thu, 17 May 2018 08:45:13 GMT): Gvedam (Thu, 17 May 2018 08:45:13 GMT): Gvedam (Thu, 17 May 2018 08:45:13 GMT): Gvedam (Thu, 17 May 2018 08:45:24 GMT): JordanDearsley (Thu, 17 May 2018 16:13:05 GMT): JordanDearsley (Thu, 17 May 2018 16:14:00 GMT): JordanDearsley (Thu, 17 May 2018 16:14:00 GMT): JordanDearsley (Thu, 17 May 2018 16:27:47 GMT): JordanDearsley (Thu, 17 May 2018 16:27:59 GMT): JordanDearsley (Thu, 17 May 2018 16:28:09 GMT): tongli (Thu, 17 May 2018 16:49:45 GMT): tongli (Thu, 17 May 2018 16:50:23 GMT): h0cin (Thu, 17 May 2018 17:42:57 GMT): h0cin (Thu, 17 May 2018 17:44:26 GMT): JordanDearsley (Thu, 17 May 2018 17:50:32 GMT): h0cin (Thu, 17 May 2018 17:57:29 GMT): h0cin (Thu, 17 May 2018 18:52:16 GMT): tongli (Fri, 18 May 2018 01:36:54 GMT): baohua (Fri, 18 May 2018 09:09:08 GMT): tongli (Fri, 18 May 2018 12:01:02 GMT): Gvedam (Fri, 18 May 2018 12:58:14 GMT): Gvedam (Fri, 18 May 2018 12:58:14 GMT): tongli (Fri, 18 May 2018 13:30:33 GMT): Gvedam (Fri, 18 May 2018 13:55:12 GMT): tongli (Fri, 18 May 2018 14:33:38 GMT): h0cin (Fri, 18 May 2018 14:47:15 GMT): h0cin (Fri, 18 May 2018 14:47:31 GMT): JordanDearsley (Fri, 18 May 2018 15:37:38 GMT): tongli (Fri, 18 May 2018 16:13:00 GMT): tongli (Fri, 18 May 2018 16:13:29 GMT): tongli (Fri, 18 May 2018 16:14:00 GMT): tongli (Fri, 18 May 2018 16:14:21 GMT): tongli (Fri, 18 May 2018 16:14:53 GMT): tongli (Fri, 18 May 2018 16:16:52 GMT): tongli (Fri, 18 May 2018 16:17:17 GMT): tongli (Fri, 18 May 2018 16:17:31 GMT): tongli (Fri, 18 May 2018 16:17:50 GMT): Gvedam (Fri, 18 May 2018 16:54:20 GMT): JordanDearsley (Fri, 18 May 2018 17:37:11 GMT): JordanDearsley (Fri, 18 May 2018 19:04:47 GMT): JordanDearsley (Fri, 18 May 2018 19:18:15 GMT): JordanDearsley (Fri, 18 May 2018 19:19:17 GMT): JordanDearsley (Fri, 18 May 2018 21:42:41 GMT): robwahl (Sat, 19 May 2018 16:08:24 GMT): KotsAshish (Sun, 20 May 2018 16:08:24 GMT): phulei (Sun, 20 May 2018 19:11:46 GMT): phulei (Sun, 20 May 2018 19:15:21 GMT): phulei (Sun, 20 May 2018 19:15:40 GMT): phanikumar (Sun, 20 May 2018 20:40:50 GMT): phulei (Sun, 20 May 2018 21:21:03 GMT): phulei (Sun, 20 May 2018 21:21:18 GMT): RajuSomala (Mon, 21 May 2018 10:49:22 GMT): robinrob (Mon, 21 May 2018 11:59:35 GMT): tongli (Mon, 21 May 2018 13:08:28 GMT): kiranarshakota (Mon, 21 May 2018 16:36:29 GMT): kiranarshakota (Mon, 21 May 2018 16:47:29 GMT): tongli (Mon, 21 May 2018 17:15:20 GMT): tongli (Mon, 21 May 2018 17:15:31 GMT): tongli (Mon, 21 May 2018 17:15:59 GMT): tongli (Mon, 21 May 2018 17:16:45 GMT): tongli (Mon, 21 May 2018 17:17:10 GMT): Vanitha (Mon, 21 May 2018 17:56:59 GMT): Vanitha (Mon, 21 May 2018 18:02:40 GMT): Vanitha (Mon, 21 May 2018 18:02:59 GMT): phulei (Mon, 21 May 2018 18:53:33 GMT): Vanitha (Mon, 21 May 2018 20:47:44 GMT): baohua (Tue, 22 May 2018 01:32:19 GMT): kosullivan_sita (Tue, 22 May 2018 11:03:49 GMT): IngoRammer (Tue, 22 May 2018 14:57:33 GMT): JordanDearsley (Tue, 22 May 2018 14:59:39 GMT): JordanDearsley (Tue, 22 May 2018 14:59:43 GMT): JordanDearsley (Tue, 22 May 2018 15:00:32 GMT): tongli (Tue, 22 May 2018 15:02:24 GMT): tongli (Tue, 22 May 2018 15:02:42 GMT): tongli (Tue, 22 May 2018 15:02:57 GMT): tongli (Tue, 22 May 2018 15:03:12 GMT): tongli (Tue, 22 May 2018 15:03:30 GMT): tongli (Tue, 22 May 2018 15:03:41 GMT): JordanDearsley (Tue, 22 May 2018 15:03:46 GMT): tongli (Tue, 22 May 2018 15:03:52 GMT): tongli (Tue, 22 May 2018 15:03:57 GMT): JordanDearsley (Tue, 22 May 2018 15:04:14 GMT): tongli (Tue, 22 May 2018 15:04:20 GMT): tongli (Tue, 22 May 2018 15:04:32 GMT): tongli (Tue, 22 May 2018 15:04:50 GMT): JordanDearsley (Tue, 22 May 2018 15:04:58 GMT): tongli (Tue, 22 May 2018 15:05:04 GMT): tongli (Tue, 22 May 2018 15:05:17 GMT): JordanDearsley (Tue, 22 May 2018 15:05:33 GMT): tongli (Tue, 22 May 2018 15:05:40 GMT): JordanDearsley (Tue, 22 May 2018 15:05:55 GMT): tongli (Tue, 22 May 2018 15:06:03 GMT): tongli (Tue, 22 May 2018 15:06:20 GMT): tongli (Tue, 22 May 2018 15:07:01 GMT): tongli (Tue, 22 May 2018 15:07:19 GMT): JordanDearsley (Tue, 22 May 2018 15:24:34 GMT): tongli (Tue, 22 May 2018 15:25:01 GMT): tongli (Tue, 22 May 2018 15:25:28 GMT): tongli (Tue, 22 May 2018 15:26:11 GMT): tongli (Tue, 22 May 2018 15:26:48 GMT): tongli (Tue, 22 May 2018 15:27:12 GMT): tongli (Tue, 22 May 2018 15:27:28 GMT): tongli (Tue, 22 May 2018 15:27:32 GMT): JordanDearsley (Tue, 22 May 2018 15:28:58 GMT): tongli (Tue, 22 May 2018 15:29:58 GMT): tongli (Tue, 22 May 2018 15:30:09 GMT): tongli (Tue, 22 May 2018 15:30:14 GMT): tongli (Tue, 22 May 2018 15:30:19 GMT): tongli (Tue, 22 May 2018 15:30:31 GMT): tongli (Tue, 22 May 2018 15:31:06 GMT): tongli (Tue, 22 May 2018 15:31:09 GMT): JordanDearsley (Tue, 22 May 2018 15:31:12 GMT): tongli (Tue, 22 May 2018 15:31:27 GMT): tongli (Tue, 22 May 2018 15:31:30 GMT): miahfost (Tue, 22 May 2018 15:31:38 GMT): tongli (Tue, 22 May 2018 15:31:47 GMT): tongli (Tue, 22 May 2018 15:32:15 GMT): JordanDearsley (Tue, 22 May 2018 15:40:22 GMT): JordanDearsley (Tue, 22 May 2018 15:41:17 GMT): JordanDearsley (Tue, 22 May 2018 15:43:03 GMT): JordanDearsley (Tue, 22 May 2018 15:43:03 GMT): tongli (Tue, 22 May 2018 15:47:39 GMT): tongli (Tue, 22 May 2018 15:47:40 GMT): tongli (Tue, 22 May 2018 15:47:54 GMT): tongli (Tue, 22 May 2018 15:48:06 GMT): JordanDearsley (Tue, 22 May 2018 15:48:48 GMT): tongli (Tue, 22 May 2018 15:49:09 GMT): JordanDearsley (Tue, 22 May 2018 15:49:19 GMT): tongli (Tue, 22 May 2018 15:49:54 GMT): tongli (Tue, 22 May 2018 15:50:00 GMT): tongli (Tue, 22 May 2018 15:50:41 GMT): tongli (Tue, 22 May 2018 15:50:49 GMT): JordanDearsley (Tue, 22 May 2018 15:51:31 GMT): JordanDearsley (Tue, 22 May 2018 15:51:34 GMT): tongli (Tue, 22 May 2018 15:51:36 GMT): tongli (Tue, 22 May 2018 15:52:07 GMT): JordanDearsley (Tue, 22 May 2018 15:52:19 GMT): tongli (Tue, 22 May 2018 15:52:36 GMT): tongli (Tue, 22 May 2018 15:52:48 GMT): tongli (Tue, 22 May 2018 15:53:04 GMT): tongli (Tue, 22 May 2018 15:53:35 GMT): tongli (Tue, 22 May 2018 15:53:56 GMT): tongli (Tue, 22 May 2018 15:54:25 GMT): joe-alewine (Tue, 22 May 2018 17:04:07 GMT): JordanDearsley (Tue, 22 May 2018 17:38:32 GMT): JordanDearsley (Tue, 22 May 2018 17:39:33 GMT): JordanDearsley (Tue, 22 May 2018 17:40:38 GMT): JordanDearsley (Tue, 22 May 2018 17:41:52 GMT): Vanitha (Tue, 22 May 2018 18:29:07 GMT): Vanitha (Tue, 22 May 2018 18:31:14 GMT): vcoss (Tue, 22 May 2018 18:59:05 GMT): sukrit.handa@gmail.com (Wed, 23 May 2018 03:23:09 GMT): tongli (Wed, 23 May 2018 12:23:17 GMT): tongli (Wed, 23 May 2018 12:23:25 GMT): tongli (Wed, 23 May 2018 12:24:20 GMT): vcoss (Wed, 23 May 2018 12:24:47 GMT): tongli (Wed, 23 May 2018 12:24:48 GMT): vcoss (Wed, 23 May 2018 12:24:56 GMT): tongli (Wed, 23 May 2018 12:25:05 GMT): tongli (Wed, 23 May 2018 12:26:03 GMT): tongli (Wed, 23 May 2018 12:26:57 GMT): tongli (Wed, 23 May 2018 12:27:14 GMT): vcoss (Wed, 23 May 2018 12:27:22 GMT): tongli (Wed, 23 May 2018 12:27:37 GMT): vcoss (Wed, 23 May 2018 12:27:41 GMT): tongli (Wed, 23 May 2018 12:27:55 GMT): tongli (Wed, 23 May 2018 12:28:05 GMT): tongli (Wed, 23 May 2018 12:28:32 GMT): tongli (Wed, 23 May 2018 12:28:39 GMT): vcoss (Wed, 23 May 2018 12:29:48 GMT): tongli (Wed, 23 May 2018 12:30:51 GMT): tongli (Wed, 23 May 2018 12:30:58 GMT): tongli (Wed, 23 May 2018 12:31:19 GMT): vcoss (Wed, 23 May 2018 12:32:10 GMT): vcoss (Wed, 23 May 2018 12:32:10 GMT): tongli (Wed, 23 May 2018 12:32:33 GMT): vcoss (Wed, 23 May 2018 12:32:54 GMT): vcoss (Wed, 23 May 2018 12:33:04 GMT): tongli (Wed, 23 May 2018 12:33:14 GMT): tongli (Wed, 23 May 2018 12:33:22 GMT): vcoss (Wed, 23 May 2018 12:33:28 GMT): vcoss (Wed, 23 May 2018 12:33:30 GMT): tongli (Wed, 23 May 2018 12:34:41 GMT): vcoss (Wed, 23 May 2018 13:05:42 GMT): vcoss (Wed, 23 May 2018 13:07:32 GMT): vcoss (Wed, 23 May 2018 13:12:01 GMT): ping40 (Wed, 23 May 2018 13:13:12 GMT): vcoss (Wed, 23 May 2018 13:22:07 GMT): vcoss (Wed, 23 May 2018 13:22:07 GMT): tongli (Wed, 23 May 2018 13:28:03 GMT): vcoss (Wed, 23 May 2018 13:28:32 GMT): tongli (Wed, 23 May 2018 13:28:38 GMT): tongli (Wed, 23 May 2018 13:29:07 GMT): vcoss (Wed, 23 May 2018 13:29:38 GMT): tongli (Wed, 23 May 2018 13:30:11 GMT): vcoss (Wed, 23 May 2018 13:30:41 GMT): vcoss (Wed, 23 May 2018 13:31:47 GMT): vcoss (Wed, 23 May 2018 13:31:59 GMT): vcoss (Wed, 23 May 2018 13:40:26 GMT): JordanDearsley (Wed, 23 May 2018 13:51:02 GMT): JordanDearsley (Wed, 23 May 2018 13:51:51 GMT): tongli (Wed, 23 May 2018 13:52:00 GMT): tongli (Wed, 23 May 2018 13:52:06 GMT): JordanDearsley (Wed, 23 May 2018 13:52:19 GMT): h0cin (Wed, 23 May 2018 16:12:31 GMT): h0cin (Wed, 23 May 2018 16:16:16 GMT): h0cin (Wed, 23 May 2018 16:16:31 GMT): vcoss (Wed, 23 May 2018 18:20:46 GMT): vcoss (Wed, 23 May 2018 18:37:52 GMT): tongli (Wed, 23 May 2018 20:20:52 GMT): tongli (Wed, 23 May 2018 20:21:07 GMT): tongli (Wed, 23 May 2018 20:21:14 GMT): tongli (Wed, 23 May 2018 20:22:03 GMT): tongli (Wed, 23 May 2018 20:22:39 GMT): tongli (Wed, 23 May 2018 20:22:44 GMT): vcoss (Wed, 23 May 2018 20:24:28 GMT): vcoss (Wed, 23 May 2018 20:24:31 GMT): tongli (Wed, 23 May 2018 20:24:44 GMT): tongli (Wed, 23 May 2018 20:25:04 GMT): tongli (Wed, 23 May 2018 20:25:41 GMT): tongli (Wed, 23 May 2018 20:28:06 GMT): tongli (Wed, 23 May 2018 20:28:14 GMT): tongli (Wed, 23 May 2018 20:28:36 GMT): vcoss (Wed, 23 May 2018 21:51:58 GMT): vcoss (Wed, 23 May 2018 22:08:57 GMT): tongli (Thu, 24 May 2018 01:39:16 GMT): tongli (Thu, 24 May 2018 01:39:39 GMT): baohua (Thu, 24 May 2018 01:45:37 GMT): baohua (Thu, 24 May 2018 01:46:43 GMT): BorgAmos (Thu, 24 May 2018 05:50:00 GMT): vikramjit (Thu, 24 May 2018 10:49:48 GMT): rjbrock (Thu, 24 May 2018 14:30:31 GMT): tongli (Thu, 24 May 2018 14:38:54 GMT): tongli (Thu, 24 May 2018 14:39:21 GMT): baohua (Thu, 24 May 2018 14:40:06 GMT): tongli (Thu, 24 May 2018 14:40:33 GMT): tongli (Thu, 24 May 2018 14:40:53 GMT): baohua (Thu, 24 May 2018 14:46:56 GMT): erikvaningen (Thu, 24 May 2018 14:56:55 GMT): erikvaningen (Thu, 24 May 2018 14:57:23 GMT): JordanDearsley (Thu, 24 May 2018 15:00:53 GMT): JordanDearsley (Thu, 24 May 2018 15:00:53 GMT): JordanDearsley (Thu, 24 May 2018 15:00:53 GMT): tongli (Thu, 24 May 2018 15:01:40 GMT): JordanDearsley (Thu, 24 May 2018 15:01:49 GMT): tongli (Thu, 24 May 2018 15:02:13 GMT): tongli (Thu, 24 May 2018 15:02:15 GMT): JordanDearsley (Thu, 24 May 2018 15:02:26 GMT): JordanDearsley (Thu, 24 May 2018 15:02:29 GMT): tongli (Thu, 24 May 2018 15:03:00 GMT): JordanDearsley (Thu, 24 May 2018 15:03:34 GMT): tongli (Thu, 24 May 2018 15:03:48 GMT): JordanDearsley (Thu, 24 May 2018 15:09:33 GMT): JordanDearsley (Thu, 24 May 2018 15:09:41 GMT): KellyCooper (Thu, 24 May 2018 15:15:54 GMT): KellyCooper (Thu, 24 May 2018 15:29:13 GMT): JordanDearsley (Thu, 24 May 2018 16:22:40 GMT): JordanDearsley (Thu, 24 May 2018 16:23:29 GMT): JordanDearsley (Thu, 24 May 2018 17:46:19 GMT): JordanDearsley (Thu, 24 May 2018 17:46:19 GMT): JordanDearsley (Thu, 24 May 2018 17:46:19 GMT): JordanDearsley (Thu, 24 May 2018 17:46:48 GMT): JordanDearsley (Thu, 24 May 2018 17:48:59 GMT): JordanDearsley (Thu, 24 May 2018 17:53:55 GMT): JordanDearsley (Thu, 24 May 2018 17:53:55 GMT): JordanDearsley (Thu, 24 May 2018 17:53:55 GMT): JordanDearsley (Thu, 24 May 2018 17:53:55 GMT): JordanDearsley (Thu, 24 May 2018 17:53:55 GMT): tongli (Thu, 24 May 2018 17:55:01 GMT): JordanDearsley (Thu, 24 May 2018 17:55:43 GMT): JordanDearsley (Thu, 24 May 2018 19:30:33 GMT): tongli (Thu, 24 May 2018 19:31:10 GMT): tongli (Thu, 24 May 2018 19:31:15 GMT): tongli (Thu, 24 May 2018 19:31:44 GMT): tongli (Thu, 24 May 2018 19:31:57 GMT): tongli (Thu, 24 May 2018 19:32:06 GMT): JordanDearsley (Thu, 24 May 2018 19:43:46 GMT): JordanDearsley (Thu, 24 May 2018 21:39:04 GMT): JordanDearsley (Thu, 24 May 2018 21:40:17 GMT): JordanDearsley (Thu, 24 May 2018 21:41:39 GMT): JordanDearsley (Thu, 24 May 2018 21:41:55 GMT): JordanDearsley (Thu, 24 May 2018 21:41:55 GMT): JordanDearsley (Thu, 24 May 2018 21:41:58 GMT): JordanDearsley (Thu, 24 May 2018 21:41:58 GMT): apoorv31 (Fri, 25 May 2018 07:18:05 GMT): miahfost (Fri, 25 May 2018 12:40:05 GMT): baohua (Fri, 25 May 2018 13:06:35 GMT): baohua (Fri, 25 May 2018 13:06:42 GMT): baohua (Fri, 25 May 2018 13:07:14 GMT): miahfost (Fri, 25 May 2018 13:07:16 GMT): baohua (Fri, 25 May 2018 13:07:23 GMT): miahfost (Fri, 25 May 2018 13:07:32 GMT): miahfost (Fri, 25 May 2018 13:07:56 GMT): baohua (Fri, 25 May 2018 13:08:12 GMT): baohua (Fri, 25 May 2018 13:08:38 GMT): miahfost (Fri, 25 May 2018 13:08:46 GMT): miahfost (Fri, 25 May 2018 13:09:22 GMT): tongli (Fri, 25 May 2018 13:09:45 GMT): miahfost (Fri, 25 May 2018 13:10:13 GMT): tongli (Fri, 25 May 2018 13:10:25 GMT): miahfost (Fri, 25 May 2018 13:10:31 GMT): tongli (Fri, 25 May 2018 13:11:05 GMT): miahfost (Fri, 25 May 2018 13:11:10 GMT): miahfost (Fri, 25 May 2018 13:11:31 GMT): tongli (Fri, 25 May 2018 13:11:35 GMT): miahfost (Fri, 25 May 2018 13:11:47 GMT): tongli (Fri, 25 May 2018 13:12:12 GMT): tongli (Fri, 25 May 2018 13:13:02 GMT): miahfost (Fri, 25 May 2018 13:13:26 GMT): tongli (Fri, 25 May 2018 13:13:36 GMT): tongli (Fri, 25 May 2018 13:13:50 GMT): miahfost (Fri, 25 May 2018 13:14:18 GMT): tongli (Fri, 25 May 2018 13:14:18 GMT): tongli (Fri, 25 May 2018 13:14:23 GMT): tongli (Fri, 25 May 2018 13:15:06 GMT): baohua (Fri, 25 May 2018 13:15:46 GMT): tongli (Fri, 25 May 2018 13:16:27 GMT): baohua (Fri, 25 May 2018 13:16:47 GMT): tongli (Fri, 25 May 2018 13:18:32 GMT): tongli (Fri, 25 May 2018 13:19:08 GMT): tongli (Fri, 25 May 2018 13:19:42 GMT): tongli (Fri, 25 May 2018 13:19:45 GMT): tongli (Fri, 25 May 2018 13:19:51 GMT): tongli (Fri, 25 May 2018 13:20:01 GMT): tongli (Fri, 25 May 2018 13:20:11 GMT): tongli (Fri, 25 May 2018 13:20:16 GMT): tongli (Fri, 25 May 2018 13:20:25 GMT): miahfost (Fri, 25 May 2018 13:22:13 GMT): JordanDearsley (Fri, 25 May 2018 13:50:30 GMT): JordanDearsley (Fri, 25 May 2018 13:51:10 GMT): tongli (Fri, 25 May 2018 13:51:22 GMT): JordanDearsley (Fri, 25 May 2018 13:52:13 GMT): JordanDearsley (Fri, 25 May 2018 13:53:53 GMT): tongli (Fri, 25 May 2018 13:53:54 GMT): JordanDearsley (Fri, 25 May 2018 13:54:11 GMT): tongli (Fri, 25 May 2018 13:54:19 GMT): JordanDearsley (Fri, 25 May 2018 13:54:25 GMT): tongli (Fri, 25 May 2018 13:54:54 GMT): tongli (Fri, 25 May 2018 13:55:15 GMT): tongli (Fri, 25 May 2018 13:55:34 GMT): tongli (Fri, 25 May 2018 13:56:00 GMT): tongli (Fri, 25 May 2018 13:56:24 GMT): tongli (Fri, 25 May 2018 13:56:33 GMT): JordanDearsley (Fri, 25 May 2018 13:56:41 GMT): JordanDearsley (Fri, 25 May 2018 13:57:26 GMT): tongli (Fri, 25 May 2018 13:57:27 GMT): tongli (Fri, 25 May 2018 13:57:51 GMT): tongli (Fri, 25 May 2018 13:58:03 GMT): tongli (Fri, 25 May 2018 13:58:44 GMT): tongli (Fri, 25 May 2018 13:58:57 GMT): JordanDearsley (Fri, 25 May 2018 13:59:16 GMT): JordanDearsley (Fri, 25 May 2018 13:59:17 GMT): tongli (Fri, 25 May 2018 13:59:24 GMT): JordanDearsley (Fri, 25 May 2018 14:00:13 GMT): tongli (Fri, 25 May 2018 14:00:56 GMT): tongli (Fri, 25 May 2018 14:01:05 GMT): JordanDearsley (Fri, 25 May 2018 14:01:14 GMT): tongli (Fri, 25 May 2018 14:01:29 GMT): tongli (Fri, 25 May 2018 14:01:33 GMT): tongli (Fri, 25 May 2018 14:01:34 GMT): tongli (Fri, 25 May 2018 14:01:54 GMT): tongli (Fri, 25 May 2018 14:02:05 GMT): tongli (Fri, 25 May 2018 14:02:22 GMT): tongli (Fri, 25 May 2018 14:02:34 GMT): tongli (Fri, 25 May 2018 14:03:09 GMT): tongli (Fri, 25 May 2018 14:03:42 GMT): JordanDearsley (Fri, 25 May 2018 14:03:47 GMT): JordanDearsley (Fri, 25 May 2018 14:04:01 GMT): tongli (Fri, 25 May 2018 14:04:14 GMT): JordanDearsley (Fri, 25 May 2018 14:04:30 GMT): JordanDearsley (Fri, 25 May 2018 14:05:10 GMT): JordanDearsley (Fri, 25 May 2018 14:05:16 GMT): JordanDearsley (Fri, 25 May 2018 14:09:19 GMT): tongli (Fri, 25 May 2018 14:37:26 GMT): tongli (Fri, 25 May 2018 14:38:22 GMT): tongli (Fri, 25 May 2018 14:38:46 GMT): JordanDearsley (Fri, 25 May 2018 14:38:58 GMT): JordanDearsley (Fri, 25 May 2018 14:39:15 GMT): JordanDearsley (Fri, 25 May 2018 14:39:30 GMT): tongli (Fri, 25 May 2018 14:39:36 GMT): JordanDearsley (Fri, 25 May 2018 14:39:48 GMT): tongli (Fri, 25 May 2018 14:40:14 GMT): tongli (Fri, 25 May 2018 14:40:31 GMT): DerHolzloeffel (Fri, 25 May 2018 16:40:03 GMT): DerHolzloeffel (Fri, 25 May 2018 16:42:08 GMT): DerHolzloeffel (Fri, 25 May 2018 16:42:13 GMT): DerHolzloeffel (Fri, 25 May 2018 16:42:31 GMT): swapnasagarpradhan (Fri, 25 May 2018 19:32:33 GMT): phulei (Fri, 25 May 2018 19:36:24 GMT): phulei (Fri, 25 May 2018 19:37:22 GMT): JordanDearsley (Fri, 25 May 2018 20:14:50 GMT): JordanDearsley (Fri, 25 May 2018 20:18:16 GMT): JordanDearsley (Fri, 25 May 2018 20:18:16 GMT): JordanDearsley (Fri, 25 May 2018 20:18:16 GMT): JordanDearsley (Fri, 25 May 2018 20:29:22 GMT): baohua (Mon, 28 May 2018 03:18:22 GMT): baohua (Mon, 28 May 2018 03:18:44 GMT): baohua (Mon, 28 May 2018 03:20:06 GMT): DerHolzloeffel (Mon, 28 May 2018 12:14:43 GMT): JordanDearsley (Mon, 28 May 2018 13:59:29 GMT): JordanDearsley (Mon, 28 May 2018 13:59:29 GMT): guangyingyuan (Mon, 28 May 2018 14:56:17 GMT): guangyingyuan (Mon, 28 May 2018 15:09:27 GMT): guangyingyuan (Mon, 28 May 2018 15:09:27 GMT): guangyingyuan (Mon, 28 May 2018 15:17:38 GMT): JordanDearsley (Mon, 28 May 2018 15:51:13 GMT): phulei (Mon, 28 May 2018 17:01:39 GMT): phulei (Mon, 28 May 2018 17:02:05 GMT): phulei (Mon, 28 May 2018 17:11:11 GMT): phulei (Mon, 28 May 2018 17:15:31 GMT): JordanDearsley (Mon, 28 May 2018 18:49:12 GMT): baohua (Tue, 29 May 2018 01:50:59 GMT): baohua (Tue, 29 May 2018 01:53:31 GMT): baohua (Tue, 29 May 2018 01:53:52 GMT): baohua (Tue, 29 May 2018 01:56:47 GMT): guangyingyuan (Tue, 29 May 2018 02:00:22 GMT): BALACHUPPALA (Tue, 29 May 2018 06:03:01 GMT): chandrakanthm (Tue, 29 May 2018 08:33:55 GMT): Vanitha (Tue, 29 May 2018 13:18:25 GMT): Vanitha (Tue, 29 May 2018 13:22:07 GMT): Vanitha (Tue, 29 May 2018 13:23:25 GMT): Vanitha (Tue, 29 May 2018 13:24:14 GMT): Tfabric (Tue, 29 May 2018 14:17:02 GMT): Tfabric (Tue, 29 May 2018 14:18:09 GMT): Tfabric (Tue, 29 May 2018 14:19:18 GMT): Tfabric (Tue, 29 May 2018 14:19:26 GMT): tongli (Tue, 29 May 2018 14:23:01 GMT): tongli (Tue, 29 May 2018 14:25:06 GMT): tongli (Tue, 29 May 2018 14:25:26 GMT): tongli (Tue, 29 May 2018 14:25:45 GMT): tongli (Tue, 29 May 2018 14:26:01 GMT): Vanitha (Tue, 29 May 2018 14:26:11 GMT): tongli (Tue, 29 May 2018 14:26:48 GMT): Vanitha (Tue, 29 May 2018 14:27:29 GMT): tongli (Tue, 29 May 2018 14:28:11 GMT): Vanitha (Tue, 29 May 2018 14:28:20 GMT): Vanitha (Tue, 29 May 2018 14:28:20 GMT): tongli (Tue, 29 May 2018 14:28:42 GMT): tongli (Tue, 29 May 2018 14:29:12 GMT): tongli (Tue, 29 May 2018 14:29:15 GMT): Vanitha (Tue, 29 May 2018 14:31:16 GMT): tongli (Tue, 29 May 2018 14:31:52 GMT): Vanitha (Tue, 29 May 2018 14:31:56 GMT): tongli (Tue, 29 May 2018 14:32:22 GMT): tongli (Tue, 29 May 2018 14:32:36 GMT): Vanitha (Tue, 29 May 2018 14:33:31 GMT): Vanitha (Tue, 29 May 2018 14:33:40 GMT): tongli (Tue, 29 May 2018 14:34:14 GMT): tongli (Tue, 29 May 2018 14:34:38 GMT): Vanitha (Tue, 29 May 2018 14:35:49 GMT): tongli (Tue, 29 May 2018 14:36:30 GMT): Vanitha (Tue, 29 May 2018 14:40:18 GMT): tongli (Tue, 29 May 2018 14:42:43 GMT): Vanitha (Tue, 29 May 2018 14:46:47 GMT): tongli (Tue, 29 May 2018 14:49:51 GMT): Vanitha (Tue, 29 May 2018 14:51:14 GMT): JordanDearsley (Tue, 29 May 2018 15:04:22 GMT): tongli (Tue, 29 May 2018 15:13:29 GMT): tongli (Tue, 29 May 2018 15:15:17 GMT): tongli (Tue, 29 May 2018 15:15:38 GMT): tongli (Tue, 29 May 2018 15:16:07 GMT): tongli (Tue, 29 May 2018 15:16:29 GMT): tongli (Tue, 29 May 2018 15:16:37 GMT): JordanDearsley (Tue, 29 May 2018 15:21:20 GMT): JordanDearsley (Tue, 29 May 2018 15:21:39 GMT): tongli (Tue, 29 May 2018 15:21:42 GMT): JordanDearsley (Tue, 29 May 2018 15:22:17 GMT): tongli (Tue, 29 May 2018 15:22:35 GMT): tongli (Tue, 29 May 2018 15:23:04 GMT): JordanDearsley (Tue, 29 May 2018 15:23:08 GMT): tongli (Tue, 29 May 2018 15:23:14 GMT): JordanDearsley (Tue, 29 May 2018 15:23:32 GMT): tongli (Tue, 29 May 2018 15:23:44 GMT): tongli (Tue, 29 May 2018 15:23:48 GMT): JordanDearsley (Tue, 29 May 2018 15:23:58 GMT): nagap (Tue, 29 May 2018 20:52:23 GMT): poleriobabao (Wed, 30 May 2018 02:34:43 GMT): poleriobabao (Wed, 30 May 2018 02:35:48 GMT): rkrish82 (Wed, 30 May 2018 04:06:36 GMT): rkrish82 (Wed, 30 May 2018 04:26:23 GMT): sunyoubo (Wed, 30 May 2018 06:18:48 GMT): sunyoubo (Wed, 30 May 2018 06:35:29 GMT): sunyoubo (Wed, 30 May 2018 06:35:58 GMT): sunyoubo (Wed, 30 May 2018 07:30:10 GMT): Luke_Chen (Wed, 30 May 2018 08:21:35 GMT): Luke_Chen (Wed, 30 May 2018 08:22:06 GMT): sunyoubo (Wed, 30 May 2018 08:23:05 GMT): sunyoubo (Wed, 30 May 2018 08:27:40 GMT): sunyoubo (Wed, 30 May 2018 08:31:45 GMT): Luke_Chen (Wed, 30 May 2018 08:33:24 GMT): sunyoubo (Wed, 30 May 2018 08:46:53 GMT): sunyoubo (Wed, 30 May 2018 08:47:37 GMT): Luke_Chen (Wed, 30 May 2018 08:49:43 GMT): sunyoubo (Wed, 30 May 2018 08:50:52 GMT): Luke_Chen (Wed, 30 May 2018 08:54:59 GMT): poleriobabao (Wed, 30 May 2018 10:49:22 GMT): Amir (Wed, 30 May 2018 11:58:17 GMT): sunyoubo (Wed, 30 May 2018 12:10:52 GMT): sunyoubo (Wed, 30 May 2018 12:22:16 GMT): sunyoubo (Wed, 30 May 2018 12:27:06 GMT): Amir (Wed, 30 May 2018 12:55:50 GMT): JordanDearsley (Wed, 30 May 2018 20:22:33 GMT): baohua (Thu, 31 May 2018 01:37:14 GMT): rkrish82 (Thu, 31 May 2018 08:43:55 GMT): peter.danko (Thu, 31 May 2018 09:17:16 GMT): Alex0 (Thu, 31 May 2018 10:08:24 GMT): esanchma (Thu, 31 May 2018 10:43:39 GMT): MonnyClara (Thu, 31 May 2018 12:32:43 GMT): tongli (Thu, 31 May 2018 12:44:50 GMT): tongli (Thu, 31 May 2018 12:45:27 GMT): JordanDearsley (Thu, 31 May 2018 19:00:08 GMT): JordanDearsley (Thu, 31 May 2018 19:00:46 GMT): JordanDearsley (Thu, 31 May 2018 19:02:34 GMT): phulei (Thu, 31 May 2018 19:28:51 GMT): phulei (Thu, 31 May 2018 19:28:53 GMT): phulei (Thu, 31 May 2018 19:29:17 GMT): tongli (Thu, 31 May 2018 20:01:14 GMT): Amir (Fri, 01 Jun 2018 00:08:01 GMT): chandrakanthMamillapalli (Fri, 01 Jun 2018 03:12:01 GMT): chandrakanthMamillapalli (Fri, 01 Jun 2018 03:15:46 GMT): chandrakanthMamillapalli (Fri, 01 Jun 2018 03:17:15 GMT): rkrish82 (Fri, 01 Jun 2018 07:58:22 GMT): rkrish82 (Fri, 01 Jun 2018 07:58:27 GMT): chandrakanthMamillapalli (Fri, 01 Jun 2018 08:26:48 GMT): tongli (Fri, 01 Jun 2018 13:54:57 GMT): tongli (Fri, 01 Jun 2018 13:55:11 GMT): Vanitha (Fri, 01 Jun 2018 15:09:18 GMT): Vanitha (Fri, 01 Jun 2018 15:09:38 GMT): phulei (Fri, 01 Jun 2018 16:10:16 GMT): phulei (Fri, 01 Jun 2018 16:10:49 GMT): phulei (Fri, 01 Jun 2018 16:11:00 GMT): phulei (Fri, 01 Jun 2018 16:14:28 GMT): Vanitha (Fri, 01 Jun 2018 17:16:48 GMT): Vanitha (Fri, 01 Jun 2018 17:17:06 GMT): phulei (Fri, 01 Jun 2018 17:17:43 GMT): phulei (Fri, 01 Jun 2018 17:18:00 GMT): phulei (Fri, 01 Jun 2018 17:42:54 GMT): Vanitha (Fri, 01 Jun 2018 17:44:57 GMT): Vanitha (Fri, 01 Jun 2018 17:45:28 GMT): phulei (Fri, 01 Jun 2018 17:46:20 GMT): phulei (Fri, 01 Jun 2018 17:47:47 GMT): Vanitha (Fri, 01 Jun 2018 17:49:59 GMT): Vanitha (Fri, 01 Jun 2018 17:52:32 GMT): Vanitha (Fri, 01 Jun 2018 17:53:23 GMT): Vanitha (Fri, 01 Jun 2018 17:53:23 GMT): phulei (Fri, 01 Jun 2018 17:53:51 GMT): phulei (Fri, 01 Jun 2018 17:54:08 GMT): Vanitha (Fri, 01 Jun 2018 17:54:25 GMT): Vanitha (Fri, 01 Jun 2018 17:54:57 GMT): phulei (Fri, 01 Jun 2018 18:01:18 GMT): phulei (Fri, 01 Jun 2018 18:01:50 GMT): Vanitha (Fri, 01 Jun 2018 18:02:47 GMT): Vanitha (Fri, 01 Jun 2018 18:03:44 GMT): phulei (Fri, 01 Jun 2018 18:04:27 GMT): phulei (Fri, 01 Jun 2018 18:04:27 GMT): Vanitha (Fri, 01 Jun 2018 18:04:33 GMT): phulei (Fri, 01 Jun 2018 18:04:46 GMT): Vanitha (Fri, 01 Jun 2018 18:05:12 GMT): Vanitha (Fri, 01 Jun 2018 18:05:30 GMT): phulei (Fri, 01 Jun 2018 18:05:39 GMT): Vanitha (Fri, 01 Jun 2018 18:05:49 GMT): phulei (Fri, 01 Jun 2018 18:06:26 GMT): Vanitha (Fri, 01 Jun 2018 18:06:49 GMT): Vanitha (Fri, 01 Jun 2018 18:07:53 GMT): Vanitha (Fri, 01 Jun 2018 18:07:56 GMT): phulei (Fri, 01 Jun 2018 18:08:35 GMT): phulei (Fri, 01 Jun 2018 18:09:33 GMT): phulei (Fri, 01 Jun 2018 18:10:16 GMT): Amir (Fri, 01 Jun 2018 18:13:42 GMT): phulei (Fri, 01 Jun 2018 18:14:40 GMT): phulei (Fri, 01 Jun 2018 18:14:48 GMT): phulei (Fri, 01 Jun 2018 18:16:09 GMT): Amir (Fri, 01 Jun 2018 18:16:10 GMT): phulei (Fri, 01 Jun 2018 18:16:22 GMT): Amir (Fri, 01 Jun 2018 18:16:44 GMT): phulei (Fri, 01 Jun 2018 18:16:48 GMT): Amir (Fri, 01 Jun 2018 18:16:55 GMT): Amir (Fri, 01 Jun 2018 18:17:53 GMT): phulei (Fri, 01 Jun 2018 18:18:55 GMT): phulei (Fri, 01 Jun 2018 18:19:17 GMT): Amir (Fri, 01 Jun 2018 18:19:29 GMT): phulei (Fri, 01 Jun 2018 18:21:13 GMT): phulei (Fri, 01 Jun 2018 18:21:35 GMT): Amir (Fri, 01 Jun 2018 18:22:17 GMT): phulei (Fri, 01 Jun 2018 18:22:25 GMT): phulei (Fri, 01 Jun 2018 18:22:39 GMT): Amir (Fri, 01 Jun 2018 18:23:31 GMT): phulei (Fri, 01 Jun 2018 18:24:21 GMT): phulei (Fri, 01 Jun 2018 18:24:29 GMT): phulei (Fri, 01 Jun 2018 18:27:33 GMT): Amir (Fri, 01 Jun 2018 18:27:45 GMT): phulei (Fri, 01 Jun 2018 18:28:10 GMT): Amir (Fri, 01 Jun 2018 18:28:26 GMT): phulei (Fri, 01 Jun 2018 18:28:40 GMT): phulei (Fri, 01 Jun 2018 18:28:53 GMT): phulei (Fri, 01 Jun 2018 18:28:53 GMT): phulei (Fri, 01 Jun 2018 18:29:24 GMT): Amir (Fri, 01 Jun 2018 18:29:29 GMT): phulei (Fri, 01 Jun 2018 18:30:11 GMT): Amir (Fri, 01 Jun 2018 18:30:14 GMT): phulei (Fri, 01 Jun 2018 18:30:34 GMT): Amir (Fri, 01 Jun 2018 18:30:47 GMT): phulei (Fri, 01 Jun 2018 18:30:54 GMT): Amir (Fri, 01 Jun 2018 18:31:50 GMT): phulei (Fri, 01 Jun 2018 18:31:54 GMT): phulei (Fri, 01 Jun 2018 18:32:17 GMT): phulei (Fri, 01 Jun 2018 18:32:32 GMT): phulei (Fri, 01 Jun 2018 18:32:41 GMT): Amir (Fri, 01 Jun 2018 18:33:13 GMT): Amir (Fri, 01 Jun 2018 18:33:42 GMT): phulei (Fri, 01 Jun 2018 18:33:48 GMT): Amir (Fri, 01 Jun 2018 18:34:51 GMT): phulei (Fri, 01 Jun 2018 18:34:57 GMT): phulei (Fri, 01 Jun 2018 18:35:18 GMT): Amir (Fri, 01 Jun 2018 18:35:22 GMT): Amir (Fri, 01 Jun 2018 18:35:33 GMT): phulei (Fri, 01 Jun 2018 18:36:25 GMT): phulei (Fri, 01 Jun 2018 18:36:37 GMT): Vanitha (Fri, 01 Jun 2018 19:00:02 GMT): Vanitha (Fri, 01 Jun 2018 19:00:20 GMT): phulei (Fri, 01 Jun 2018 19:00:37 GMT): phulei (Fri, 01 Jun 2018 19:00:50 GMT): Vanitha (Fri, 01 Jun 2018 19:01:38 GMT): Vanitha (Fri, 01 Jun 2018 19:01:56 GMT): Vanitha (Fri, 01 Jun 2018 19:02:46 GMT): phulei (Fri, 01 Jun 2018 19:03:33 GMT): Vanitha (Fri, 01 Jun 2018 19:03:38 GMT): phulei (Fri, 01 Jun 2018 19:03:39 GMT): phulei (Fri, 01 Jun 2018 19:03:45 GMT): phulei (Fri, 01 Jun 2018 19:04:35 GMT): phulei (Fri, 01 Jun 2018 19:04:44 GMT): Vanitha (Fri, 01 Jun 2018 19:07:22 GMT): Vanitha (Fri, 01 Jun 2018 19:07:22 GMT): Vanitha (Fri, 01 Jun 2018 19:07:38 GMT): phulei (Fri, 01 Jun 2018 20:50:52 GMT): phulei (Fri, 01 Jun 2018 20:50:52 GMT): DuPeng (Sat, 02 Jun 2018 03:34:08 GMT): chandrakanthMamillapalli (Sun, 03 Jun 2018 00:21:14 GMT): chandrakanthMamillapalli (Sun, 03 Jun 2018 00:21:14 GMT): chandrakanthMamillapalli (Sun, 03 Jun 2018 00:24:10 GMT): chandrakanthMamillapalli (Sun, 03 Jun 2018 03:19:16 GMT): chandrakanthMamillapalli (Sun, 03 Jun 2018 03:19:16 GMT): chandrakanthMamillapalli (Sun, 03 Jun 2018 04:56:49 GMT): chandrakanthMamillapalli (Sun, 03 Jun 2018 04:57:25 GMT): chandrakanthMamillapalli (Sun, 03 Jun 2018 06:53:43 GMT): chandrakanthMamillapalli (Sun, 03 Jun 2018 07:01:43 GMT): chandrakanthMamillapalli (Sun, 03 Jun 2018 07:02:25 GMT): hightall (Mon, 04 Jun 2018 07:41:55 GMT): Amir (Mon, 04 Jun 2018 09:17:23 GMT): Amir (Mon, 04 Jun 2018 09:17:54 GMT): Amir (Mon, 04 Jun 2018 09:19:15 GMT): suryasuresh06 (Mon, 04 Jun 2018 09:22:58 GMT): suryasuresh06 (Mon, 04 Jun 2018 09:26:38 GMT): suryasuresh06 (Mon, 04 Jun 2018 09:26:51 GMT): phulei (Mon, 04 Jun 2018 09:49:02 GMT): suryasuresh06 (Mon, 04 Jun 2018 10:02:58 GMT): suryasuresh06 (Mon, 04 Jun 2018 10:06:18 GMT): suryasuresh06 (Mon, 04 Jun 2018 10:06:23 GMT): suryasuresh06 (Mon, 04 Jun 2018 10:06:36 GMT): suryasuresh06 (Mon, 04 Jun 2018 10:06:36 GMT): phulei (Mon, 04 Jun 2018 12:52:44 GMT): phulei (Mon, 04 Jun 2018 12:52:44 GMT): beshogun (Mon, 04 Jun 2018 17:31:40 GMT): beshogun (Mon, 04 Jun 2018 17:34:32 GMT): phulei (Mon, 04 Jun 2018 19:38:15 GMT): phulei (Mon, 04 Jun 2018 19:38:15 GMT): phulei (Mon, 04 Jun 2018 19:38:38 GMT): phulei (Mon, 04 Jun 2018 19:40:39 GMT): phulei (Mon, 04 Jun 2018 19:40:39 GMT): rkrish82 (Tue, 05 Jun 2018 03:33:01 GMT): rkrish82 (Tue, 05 Jun 2018 03:33:46 GMT): Luke_Chen (Tue, 05 Jun 2018 08:02:11 GMT): rkrish82 (Tue, 05 Jun 2018 08:56:55 GMT): rkrish82 (Tue, 05 Jun 2018 08:57:29 GMT): rkrish82 (Tue, 05 Jun 2018 08:57:52 GMT): kiranarshakota (Tue, 05 Jun 2018 13:09:47 GMT): kiranarshakota (Tue, 05 Jun 2018 13:09:58 GMT): kiranarshakota (Tue, 05 Jun 2018 13:09:59 GMT): rkrish82 (Tue, 05 Jun 2018 15:25:06 GMT): abraham (Wed, 06 Jun 2018 01:33:28 GMT): tongli (Wed, 06 Jun 2018 02:39:07 GMT): tongli (Wed, 06 Jun 2018 02:39:28 GMT): chandrakanthMamillapalli (Wed, 06 Jun 2018 02:57:58 GMT): chandrakanthMamillapalli (Wed, 06 Jun 2018 02:58:02 GMT): rkrish82 (Wed, 06 Jun 2018 04:08:50 GMT): rkrish82 (Wed, 06 Jun 2018 04:08:52 GMT): chandrakanthMamillapalli (Wed, 06 Jun 2018 09:05:03 GMT): tongli (Wed, 06 Jun 2018 14:04:47 GMT): scally (Wed, 06 Jun 2018 14:21:06 GMT): scally (Wed, 06 Jun 2018 14:26:39 GMT): scally (Wed, 06 Jun 2018 14:26:56 GMT): Luke_Chen (Wed, 06 Jun 2018 15:29:07 GMT): Luke_Chen (Wed, 06 Jun 2018 15:29:07 GMT): krabradosty (Wed, 06 Jun 2018 15:53:09 GMT): krabradosty (Wed, 06 Jun 2018 15:55:06 GMT): tongli (Wed, 06 Jun 2018 16:51:39 GMT): tongli (Wed, 06 Jun 2018 16:52:17 GMT): tongli (Wed, 06 Jun 2018 16:52:38 GMT): krabradosty (Wed, 06 Jun 2018 16:53:15 GMT): tongli (Wed, 06 Jun 2018 16:53:56 GMT): krabradosty (Wed, 06 Jun 2018 16:54:28 GMT): tongli (Wed, 06 Jun 2018 16:54:43 GMT): tongli (Wed, 06 Jun 2018 16:54:58 GMT): tongli (Wed, 06 Jun 2018 16:55:13 GMT): tongli (Wed, 06 Jun 2018 16:55:26 GMT): krabradosty (Wed, 06 Jun 2018 16:55:56 GMT): tongli (Wed, 06 Jun 2018 16:56:01 GMT): tongli (Wed, 06 Jun 2018 16:56:36 GMT): tongli (Wed, 06 Jun 2018 16:56:51 GMT): tongli (Wed, 06 Jun 2018 16:57:22 GMT): krabradosty (Wed, 06 Jun 2018 18:28:50 GMT): krabradosty (Wed, 06 Jun 2018 18:28:50 GMT): tongli (Wed, 06 Jun 2018 19:36:24 GMT): tongli (Wed, 06 Jun 2018 19:36:34 GMT): tongli (Wed, 06 Jun 2018 19:36:40 GMT): phulei (Wed, 06 Jun 2018 20:16:42 GMT): phulei (Wed, 06 Jun 2018 20:17:23 GMT): chandrakanthMamillapalli (Wed, 06 Jun 2018 21:27:46 GMT): chandrakanthMamillapalli (Wed, 06 Jun 2018 21:28:26 GMT): vdods (Thu, 07 Jun 2018 00:31:51 GMT): Luke_Chen (Thu, 07 Jun 2018 05:18:16 GMT): tfls08 (Thu, 07 Jun 2018 07:02:56 GMT): krabradosty (Thu, 07 Jun 2018 07:05:43 GMT): baohua (Thu, 07 Jun 2018 08:12:09 GMT): dklesev (Thu, 07 Jun 2018 09:26:22 GMT): YashGanthe (Thu, 07 Jun 2018 10:06:56 GMT): Vanitha (Thu, 07 Jun 2018 13:25:55 GMT): baohua (Thu, 07 Jun 2018 14:10:46 GMT): baohua (Thu, 07 Jun 2018 14:11:17 GMT): baohua (Thu, 07 Jun 2018 14:12:12 GMT): baohua (Thu, 07 Jun 2018 14:12:12 GMT): krabradosty (Thu, 07 Jun 2018 14:30:07 GMT): krabradosty (Thu, 07 Jun 2018 14:30:07 GMT): krabradosty (Thu, 07 Jun 2018 14:30:07 GMT): tongli (Thu, 07 Jun 2018 14:40:59 GMT): tongli (Thu, 07 Jun 2018 14:41:39 GMT): infowolfe (Thu, 07 Jun 2018 17:48:17 GMT): Luke_Chen (Fri, 08 Jun 2018 02:17:34 GMT): LichenShen (Fri, 08 Jun 2018 03:46:42 GMT): LichenShen (Fri, 08 Jun 2018 03:49:55 GMT): baohua (Fri, 08 Jun 2018 06:31:34 GMT): aKesav (Fri, 08 Jun 2018 07:34:28 GMT): YashGanthe (Fri, 08 Jun 2018 09:12:26 GMT): Luke_Chen (Fri, 08 Jun 2018 09:32:49 GMT): YashGanthe (Fri, 08 Jun 2018 10:43:26 GMT): ShuoWang (Fri, 08 Jun 2018 12:05:44 GMT): baohua (Fri, 08 Jun 2018 12:31:59 GMT): LichenShen (Fri, 08 Jun 2018 12:34:41 GMT): baohua (Fri, 08 Jun 2018 13:01:16 GMT): baohua (Fri, 08 Jun 2018 13:01:37 GMT): baohua (Fri, 08 Jun 2018 13:02:21 GMT): NicholasBaro (Fri, 08 Jun 2018 18:24:16 GMT): NicholasBaro (Fri, 08 Jun 2018 18:26:22 GMT): enjoythecode (Sat, 09 Jun 2018 07:08:19 GMT): DmitryK4 (Sun, 10 Jun 2018 16:08:14 GMT): DmitryK4 (Sun, 10 Jun 2018 22:17:46 GMT): rkrish82 (Mon, 11 Jun 2018 05:21:36 GMT): nabil941994 (Mon, 11 Jun 2018 08:27:12 GMT): nabil941994 (Mon, 11 Jun 2018 08:28:02 GMT): muna (Mon, 11 Jun 2018 13:21:18 GMT): suryalanka (Mon, 11 Jun 2018 15:26:36 GMT): suryalanka (Mon, 11 Jun 2018 15:29:01 GMT): NicholasBaro (Mon, 11 Jun 2018 17:34:29 GMT): tongli (Tue, 12 Jun 2018 11:13:34 GMT): tongli (Tue, 12 Jun 2018 11:14:03 GMT): AshishMishra 1 (Tue, 12 Jun 2018 11:29:36 GMT): tongli (Tue, 12 Jun 2018 12:31:50 GMT): nasserfci (Tue, 12 Jun 2018 12:57:28 GMT): tongli (Tue, 12 Jun 2018 15:08:10 GMT): nasserfci (Tue, 12 Jun 2018 15:19:08 GMT): chandrakanthMamillapalli (Tue, 12 Jun 2018 18:19:54 GMT): chandrakanthMamillapalli (Tue, 12 Jun 2018 18:24:24 GMT): chandrakanthMamillapalli (Tue, 12 Jun 2018 18:24:57 GMT): chandrakanthMamillapalli (Tue, 12 Jun 2018 18:26:07 GMT): chandrakanthMamillapalli (Tue, 12 Jun 2018 18:26:07 GMT): DmitryK4 (Tue, 12 Jun 2018 22:37:05 GMT): baohua (Wed, 13 Jun 2018 01:59:15 GMT): xwu (Wed, 13 Jun 2018 08:36:50 GMT): SmartContract2018 (Wed, 13 Jun 2018 15:26:00 GMT): SmartContract2018 (Wed, 13 Jun 2018 15:26:31 GMT): SmartContract2018 (Wed, 13 Jun 2018 15:26:54 GMT): SmartContract2018 (Wed, 13 Jun 2018 15:27:34 GMT): SmartContract2018 (Wed, 13 Jun 2018 15:27:52 GMT): SmartContract2018 (Wed, 13 Jun 2018 15:28:07 GMT): infowolfe (Wed, 13 Jun 2018 20:56:37 GMT): infowolfe (Wed, 13 Jun 2018 20:56:37 GMT): mna2016 (Wed, 13 Jun 2018 21:16:21 GMT): mna2016 (Wed, 13 Jun 2018 21:18:29 GMT): infowolfe (Wed, 13 Jun 2018 21:40:26 GMT): nabil941994 (Thu, 14 Jun 2018 09:46:22 GMT): Tfabric (Thu, 14 Jun 2018 10:09:16 GMT): Tfabric (Thu, 14 Jun 2018 10:09:51 GMT): Tfabric (Thu, 14 Jun 2018 10:10:23 GMT): Tfabric (Thu, 14 Jun 2018 12:17:37 GMT): Tfabric (Thu, 14 Jun 2018 13:01:07 GMT): SmartContract2018 (Thu, 14 Jun 2018 13:16:01 GMT): hightall (Thu, 14 Jun 2018 15:08:25 GMT): abody (Thu, 14 Jun 2018 16:18:50 GMT): abody (Thu, 14 Jun 2018 16:23:38 GMT): baohua (Fri, 15 Jun 2018 13:13:00 GMT): kenty (Fri, 15 Jun 2018 15:06:57 GMT): soryboums (Sat, 16 Jun 2018 09:33:20 GMT): soryboums (Sat, 16 Jun 2018 09:50:35 GMT): soryboums (Sat, 16 Jun 2018 11:26:17 GMT): soryboums (Sat, 16 Jun 2018 11:26:17 GMT): jdpond (Sat, 16 Jun 2018 19:05:54 GMT): pankajcheema (Mon, 18 Jun 2018 05:07:39 GMT): pankajcheema (Mon, 18 Jun 2018 05:07:53 GMT): pankajcheema (Mon, 18 Jun 2018 05:07:53 GMT): pankajcheema (Mon, 18 Jun 2018 05:24:11 GMT): pankajcheema (Mon, 18 Jun 2018 05:30:52 GMT): pankajcheema (Mon, 18 Jun 2018 05:30:59 GMT): pankajcheema (Mon, 18 Jun 2018 08:44:56 GMT): pankajcheema (Mon, 18 Jun 2018 08:44:56 GMT): pankajcheema (Mon, 18 Jun 2018 08:44:56 GMT): SeamonJia (Mon, 18 Jun 2018 09:57:59 GMT): pankajcheema (Mon, 18 Jun 2018 12:14:58 GMT): JordanDearsley (Mon, 18 Jun 2018 15:44:42 GMT): ahmedsajid (Mon, 18 Jun 2018 19:06:55 GMT): sosherio (Mon, 18 Jun 2018 21:07:01 GMT): sosherio (Mon, 18 Jun 2018 21:13:30 GMT): vdods (Mon, 18 Jun 2018 22:32:41 GMT): vdods (Mon, 18 Jun 2018 22:33:54 GMT): Luke_Chen (Tue, 19 Jun 2018 02:08:13 GMT): DmitryK4 (Tue, 19 Jun 2018 02:57:45 GMT): DmitryK4 (Tue, 19 Jun 2018 02:57:45 GMT): DmitryK4 (Tue, 19 Jun 2018 02:57:45 GMT): DmitryK4 (Tue, 19 Jun 2018 02:57:45 GMT): pankajcheema (Tue, 19 Jun 2018 04:22:46 GMT): pankajcheema (Tue, 19 Jun 2018 04:22:46 GMT): pankajcheema (Tue, 19 Jun 2018 05:01:22 GMT): csengor (Tue, 19 Jun 2018 05:03:24 GMT): pankajcheema (Tue, 19 Jun 2018 05:51:59 GMT): pankajcheema (Tue, 19 Jun 2018 05:58:29 GMT): pankajcheema (Tue, 19 Jun 2018 05:58:36 GMT): pankajcheema (Tue, 19 Jun 2018 05:59:29 GMT): hightall (Tue, 19 Jun 2018 07:35:46 GMT): pankajcheema (Tue, 19 Jun 2018 08:00:43 GMT): pankajcheema (Tue, 19 Jun 2018 08:00:50 GMT): hightall (Tue, 19 Jun 2018 08:01:09 GMT): pankajcheema (Tue, 19 Jun 2018 08:01:20 GMT): pankajcheema (Tue, 19 Jun 2018 08:01:25 GMT): hightall (Tue, 19 Jun 2018 08:02:18 GMT): pankajcheema (Tue, 19 Jun 2018 08:02:34 GMT): pankajcheema (Tue, 19 Jun 2018 08:02:34 GMT): hightall (Tue, 19 Jun 2018 08:03:24 GMT): pankajcheema (Tue, 19 Jun 2018 08:03:35 GMT): pankajcheema (Tue, 19 Jun 2018 08:03:44 GMT): hightall (Tue, 19 Jun 2018 08:04:21 GMT): pankajcheema (Tue, 19 Jun 2018 08:04:31 GMT): pankajcheema (Tue, 19 Jun 2018 08:05:01 GMT): pankajcheema (Tue, 19 Jun 2018 08:05:18 GMT): pankajcheema (Tue, 19 Jun 2018 08:05:21 GMT): hightall (Tue, 19 Jun 2018 08:08:41 GMT): pankajcheema (Tue, 19 Jun 2018 08:08:45 GMT): pankajcheema (Tue, 19 Jun 2018 08:09:00 GMT): hightall (Tue, 19 Jun 2018 08:09:10 GMT): pankajcheema (Tue, 19 Jun 2018 08:09:18 GMT): pankajcheema (Tue, 19 Jun 2018 08:09:23 GMT): pankajcheema (Tue, 19 Jun 2018 08:47:08 GMT): pankajcheema (Tue, 19 Jun 2018 08:47:12 GMT): pankajcheema (Tue, 19 Jun 2018 08:47:30 GMT): Luke_Chen (Tue, 19 Jun 2018 09:12:29 GMT): sosherio (Tue, 19 Jun 2018 15:32:28 GMT): sosherio (Tue, 19 Jun 2018 15:32:39 GMT): tongli (Tue, 19 Jun 2018 16:58:32 GMT): tongli (Tue, 19 Jun 2018 16:58:37 GMT): phulei (Tue, 19 Jun 2018 17:24:07 GMT): tongli (Tue, 19 Jun 2018 17:24:37 GMT): phulei (Tue, 19 Jun 2018 17:24:37 GMT): tongli (Tue, 19 Jun 2018 17:24:59 GMT): phulei (Tue, 19 Jun 2018 17:25:11 GMT): phulei (Tue, 19 Jun 2018 17:26:07 GMT): alexvicegrab (Tue, 19 Jun 2018 20:37:00 GMT): vdods (Tue, 19 Jun 2018 23:37:52 GMT): hightall (Wed, 20 Jun 2018 01:17:23 GMT): rmamidipudi (Wed, 20 Jun 2018 01:33:36 GMT): rmamidipudi (Wed, 20 Jun 2018 01:36:29 GMT): pankajcheema (Wed, 20 Jun 2018 04:33:37 GMT): pankajcheema (Wed, 20 Jun 2018 04:41:13 GMT): pankajcheema (Wed, 20 Jun 2018 04:41:35 GMT): hightall (Wed, 20 Jun 2018 04:42:12 GMT): pankajcheema (Wed, 20 Jun 2018 04:42:33 GMT): pankajcheema (Wed, 20 Jun 2018 04:42:38 GMT): baohua (Wed, 20 Jun 2018 05:40:31 GMT): infowolfe (Wed, 20 Jun 2018 08:08:57 GMT): Luke_Chen (Wed, 20 Jun 2018 08:29:47 GMT): infowolfe (Wed, 20 Jun 2018 08:30:01 GMT): Luke_Chen (Wed, 20 Jun 2018 08:31:22 GMT): infowolfe (Wed, 20 Jun 2018 08:32:37 GMT): infowolfe (Wed, 20 Jun 2018 08:32:48 GMT): Luke_Chen (Wed, 20 Jun 2018 08:33:52 GMT): Luke_Chen (Wed, 20 Jun 2018 08:34:05 GMT): infowolfe (Wed, 20 Jun 2018 08:34:08 GMT): infowolfe (Wed, 20 Jun 2018 08:34:10 GMT): Luke_Chen (Wed, 20 Jun 2018 08:39:21 GMT): infowolfe (Wed, 20 Jun 2018 08:39:37 GMT): hightall (Wed, 20 Jun 2018 08:47:53 GMT): aarthiKarthik (Wed, 20 Jun 2018 10:08:59 GMT): Tfabric (Wed, 20 Jun 2018 11:01:22 GMT): tongli (Wed, 20 Jun 2018 11:24:09 GMT): baohua (Wed, 20 Jun 2018 13:16:31 GMT): tongli (Wed, 20 Jun 2018 14:08:28 GMT): phulei (Wed, 20 Jun 2018 15:16:38 GMT): Katie_Wei (Wed, 20 Jun 2018 18:40:53 GMT): harsha (Wed, 20 Jun 2018 19:09:14 GMT): alexvicegrab (Wed, 20 Jun 2018 20:55:40 GMT): alexvicegrab (Wed, 20 Jun 2018 20:55:40 GMT): alexvicegrab (Wed, 20 Jun 2018 20:55:40 GMT): alexvicegrab (Wed, 20 Jun 2018 20:55:40 GMT): alexvicegrab (Wed, 20 Jun 2018 20:55:40 GMT): alexvicegrab (Wed, 20 Jun 2018 20:55:40 GMT): alexvicegrab (Wed, 20 Jun 2018 20:55:40 GMT): alexvicegrab (Wed, 20 Jun 2018 20:55:40 GMT): alexvicegrab (Wed, 20 Jun 2018 20:55:40 GMT): alexvicegrab (Wed, 20 Jun 2018 20:55:40 GMT): alexvicegrab (Wed, 20 Jun 2018 20:55:40 GMT): alexvicegrab (Wed, 20 Jun 2018 20:55:40 GMT): tongli (Wed, 20 Jun 2018 21:54:41 GMT): tongli (Wed, 20 Jun 2018 21:54:50 GMT): hightall (Thu, 21 Jun 2018 01:19:14 GMT): tongli (Thu, 21 Jun 2018 03:24:31 GMT): tongli (Thu, 21 Jun 2018 03:25:21 GMT): tongli (Thu, 21 Jun 2018 03:25:27 GMT): tongli (Thu, 21 Jun 2018 03:25:51 GMT): hightall (Thu, 21 Jun 2018 03:27:19 GMT): hightall (Thu, 21 Jun 2018 03:27:41 GMT): hightall (Thu, 21 Jun 2018 03:28:27 GMT): tongli (Thu, 21 Jun 2018 03:31:27 GMT): tongli (Thu, 21 Jun 2018 03:31:43 GMT): hightall (Thu, 21 Jun 2018 03:32:37 GMT): tongli (Thu, 21 Jun 2018 03:32:53 GMT): tongli (Thu, 21 Jun 2018 03:33:01 GMT): tongli (Thu, 21 Jun 2018 03:33:20 GMT): hightall (Thu, 21 Jun 2018 03:36:33 GMT): pankajcheema (Thu, 21 Jun 2018 04:26:35 GMT): aarthiKarthik (Thu, 21 Jun 2018 05:41:33 GMT): aarthiKarthik (Thu, 21 Jun 2018 05:42:29 GMT): aarthiKarthik (Thu, 21 Jun 2018 05:42:29 GMT): aarthiKarthik (Thu, 21 Jun 2018 05:42:29 GMT): aarthiKarthik (Thu, 21 Jun 2018 05:43:52 GMT): aarthiKarthik (Thu, 21 Jun 2018 05:43:52 GMT): pankajcheema (Thu, 21 Jun 2018 10:29:32 GMT): pankajcheema (Thu, 21 Jun 2018 10:29:48 GMT): pankajcheema (Thu, 21 Jun 2018 10:49:26 GMT): tongli (Thu, 21 Jun 2018 12:19:46 GMT): tongli (Thu, 21 Jun 2018 15:46:22 GMT): tongli (Thu, 21 Jun 2018 15:46:54 GMT): tongli (Thu, 21 Jun 2018 15:47:29 GMT): tongli (Thu, 21 Jun 2018 15:48:04 GMT): tongli (Thu, 21 Jun 2018 15:48:10 GMT): tongli (Thu, 21 Jun 2018 15:48:40 GMT): tongli (Thu, 21 Jun 2018 15:49:06 GMT): tongli (Thu, 21 Jun 2018 15:49:39 GMT): tongli (Thu, 21 Jun 2018 15:49:58 GMT): tongli (Thu, 21 Jun 2018 15:51:17 GMT): tongli (Thu, 21 Jun 2018 15:51:32 GMT): tongli (Thu, 21 Jun 2018 15:51:53 GMT): tongli (Thu, 21 Jun 2018 15:52:11 GMT): tongli (Thu, 21 Jun 2018 15:52:47 GMT): tongli (Thu, 21 Jun 2018 15:53:05 GMT): tongli (Thu, 21 Jun 2018 15:53:40 GMT): frustrated-chain (Thu, 21 Jun 2018 15:54:22 GMT): frustrated-chain (Thu, 21 Jun 2018 15:54:22 GMT): frustrated-chain (Thu, 21 Jun 2018 15:54:39 GMT): frustrated-chain (Thu, 21 Jun 2018 15:55:32 GMT): tongli (Thu, 21 Jun 2018 15:58:08 GMT): tongli (Thu, 21 Jun 2018 15:58:28 GMT): tongli (Thu, 21 Jun 2018 15:58:54 GMT): tongli (Thu, 21 Jun 2018 15:59:12 GMT): tongli (Thu, 21 Jun 2018 16:00:00 GMT): tongli (Thu, 21 Jun 2018 16:00:21 GMT): tongli (Thu, 21 Jun 2018 16:00:36 GMT): tongli (Thu, 21 Jun 2018 16:00:58 GMT): tongli (Thu, 21 Jun 2018 16:01:13 GMT): tongli (Thu, 21 Jun 2018 16:01:25 GMT): tongli (Thu, 21 Jun 2018 16:01:49 GMT): tongli (Thu, 21 Jun 2018 16:02:23 GMT): tongli (Thu, 21 Jun 2018 16:02:32 GMT): tongli (Thu, 21 Jun 2018 16:02:44 GMT): tongli (Thu, 21 Jun 2018 16:02:58 GMT): tongli (Thu, 21 Jun 2018 16:03:06 GMT): tongli (Thu, 21 Jun 2018 16:03:36 GMT): tongli (Thu, 21 Jun 2018 16:03:50 GMT): tongli (Thu, 21 Jun 2018 16:03:56 GMT): tongli (Thu, 21 Jun 2018 16:04:34 GMT): frustrated-chain (Thu, 21 Jun 2018 16:07:25 GMT): frustrated-chain (Thu, 21 Jun 2018 16:07:25 GMT): tongli (Thu, 21 Jun 2018 16:08:35 GMT): tongli (Thu, 21 Jun 2018 16:08:43 GMT): frustrated-chain (Thu, 21 Jun 2018 16:08:51 GMT): tongli (Thu, 21 Jun 2018 16:09:45 GMT): frustrated-chain (Thu, 21 Jun 2018 16:10:24 GMT): tongli (Thu, 21 Jun 2018 16:10:27 GMT): hightall (Fri, 22 Jun 2018 03:01:11 GMT): pankajcheema (Fri, 22 Jun 2018 04:16:24 GMT): pankajcheema (Fri, 22 Jun 2018 04:16:33 GMT): pankajcheema (Fri, 22 Jun 2018 04:16:39 GMT): pankajcheema (Fri, 22 Jun 2018 04:27:06 GMT): pankajcheema (Fri, 22 Jun 2018 04:27:10 GMT): pankajcheema (Fri, 22 Jun 2018 04:27:32 GMT): pankajcheema (Fri, 22 Jun 2018 04:27:38 GMT): pankajcheema (Fri, 22 Jun 2018 04:29:45 GMT): pankajcheema (Fri, 22 Jun 2018 04:48:02 GMT): pankajcheema (Fri, 22 Jun 2018 04:48:41 GMT): pankajcheema (Fri, 22 Jun 2018 05:02:11 GMT): pankajcheema (Fri, 22 Jun 2018 05:02:54 GMT): hightall (Fri, 22 Jun 2018 05:03:31 GMT): pankajcheema (Fri, 22 Jun 2018 05:03:59 GMT): hightall (Fri, 22 Jun 2018 05:05:02 GMT): hightall (Fri, 22 Jun 2018 05:05:42 GMT): pankajcheema (Fri, 22 Jun 2018 05:13:16 GMT): pankajcheema (Fri, 22 Jun 2018 05:13:21 GMT): pankajcheema (Fri, 22 Jun 2018 05:23:09 GMT): hightall (Fri, 22 Jun 2018 05:23:53 GMT): pankajcheema (Fri, 22 Jun 2018 05:24:11 GMT): pankajcheema (Fri, 22 Jun 2018 05:24:33 GMT): pankajcheema (Fri, 22 Jun 2018 05:28:54 GMT): pankajcheema (Fri, 22 Jun 2018 05:30:44 GMT): pankajcheema (Fri, 22 Jun 2018 05:30:52 GMT): pankajcheema (Fri, 22 Jun 2018 05:31:22 GMT): hightall (Fri, 22 Jun 2018 05:31:26 GMT): pankajcheema (Fri, 22 Jun 2018 05:31:32 GMT): pankajcheema (Fri, 22 Jun 2018 05:31:41 GMT): pankajcheema (Fri, 22 Jun 2018 05:33:12 GMT): hightall (Fri, 22 Jun 2018 05:33:38 GMT): pankajcheema (Fri, 22 Jun 2018 05:40:19 GMT): pankajcheema (Fri, 22 Jun 2018 05:50:50 GMT): hightall (Fri, 22 Jun 2018 06:00:38 GMT): pankajcheema (Fri, 22 Jun 2018 06:01:04 GMT): pankajcheema (Fri, 22 Jun 2018 06:01:11 GMT): pankajcheema (Fri, 22 Jun 2018 06:01:25 GMT): hightall (Fri, 22 Jun 2018 06:01:28 GMT): pankajcheema (Fri, 22 Jun 2018 06:01:45 GMT): pankajcheema (Fri, 22 Jun 2018 06:02:10 GMT): hightall (Fri, 22 Jun 2018 06:02:21 GMT): pankajcheema (Fri, 22 Jun 2018 06:02:28 GMT): hightall (Fri, 22 Jun 2018 06:19:22 GMT): infowolfe (Fri, 22 Jun 2018 06:56:05 GMT): hightall (Fri, 22 Jun 2018 06:58:16 GMT): infowolfe (Fri, 22 Jun 2018 06:58:33 GMT): shuailaotou (Fri, 22 Jun 2018 08:08:02 GMT): Gaurang (Fri, 22 Jun 2018 08:55:28 GMT): Gaurang (Fri, 22 Jun 2018 08:58:40 GMT): Gaurang (Fri, 22 Jun 2018 09:02:02 GMT): Tfabric (Fri, 22 Jun 2018 11:49:25 GMT): tongli (Fri, 22 Jun 2018 13:29:59 GMT): infowolfe (Fri, 22 Jun 2018 13:39:37 GMT): tongli (Fri, 22 Jun 2018 13:42:59 GMT): baohua (Fri, 22 Jun 2018 14:11:05 GMT): infowolfe (Fri, 22 Jun 2018 15:24:11 GMT): tongli (Fri, 22 Jun 2018 15:56:25 GMT): tongli (Fri, 22 Jun 2018 15:57:30 GMT): tongli (Fri, 22 Jun 2018 15:57:36 GMT): infowolfe (Fri, 22 Jun 2018 17:19:25 GMT): tongli (Fri, 22 Jun 2018 17:19:47 GMT): infowolfe (Fri, 22 Jun 2018 17:23:04 GMT): infowolfe (Fri, 22 Jun 2018 17:23:04 GMT): infowolfe (Fri, 22 Jun 2018 17:23:04 GMT): DmitryK4 (Sat, 23 Jun 2018 03:29:57 GMT): DmitryK4 (Sat, 23 Jun 2018 03:30:44 GMT): rudimk (Sat, 23 Jun 2018 04:14:47 GMT): mengan (Sun, 24 Jun 2018 09:55:09 GMT): Ram786 (Sun, 24 Jun 2018 11:21:11 GMT): luckydogchina (Mon, 25 Jun 2018 01:37:35 GMT): luckydogchina (Mon, 25 Jun 2018 01:37:35 GMT): baohua (Mon, 25 Jun 2018 02:26:37 GMT): luckydogchina (Mon, 25 Jun 2018 02:32:18 GMT): luckydogchina (Mon, 25 Jun 2018 02:32:18 GMT): baohua (Mon, 25 Jun 2018 02:36:27 GMT): qsmen (Mon, 25 Jun 2018 02:49:37 GMT): Jaline (Mon, 25 Jun 2018 02:58:51 GMT): luckydogchina (Mon, 25 Jun 2018 03:10:12 GMT): luckydogchina (Mon, 25 Jun 2018 03:10:12 GMT): Nikolas 11 (Mon, 25 Jun 2018 03:10:55 GMT): luckydogchina (Mon, 25 Jun 2018 03:13:19 GMT): baohua (Mon, 25 Jun 2018 03:17:15 GMT): luckydogchina (Mon, 25 Jun 2018 03:40:56 GMT): pankajcheema (Mon, 25 Jun 2018 04:58:45 GMT): hightall (Mon, 25 Jun 2018 06:27:08 GMT): YashGanthe (Mon, 25 Jun 2018 08:52:42 GMT): YashGanthe (Mon, 25 Jun 2018 08:52:42 GMT): pankajcheema (Mon, 25 Jun 2018 11:47:08 GMT): pankajcheema (Mon, 25 Jun 2018 11:47:26 GMT): pankajcheema (Mon, 25 Jun 2018 11:48:04 GMT): pankajcheema (Mon, 25 Jun 2018 11:48:15 GMT): pankajcheema (Mon, 25 Jun 2018 11:48:26 GMT): pankajcheema (Mon, 25 Jun 2018 11:54:21 GMT): pankajcheema (Mon, 25 Jun 2018 11:55:23 GMT): pankajcheema (Mon, 25 Jun 2018 11:55:53 GMT): pankajcheema (Mon, 25 Jun 2018 11:59:58 GMT): pankajcheema (Mon, 25 Jun 2018 12:01:18 GMT): pankajcheema (Mon, 25 Jun 2018 12:06:53 GMT): tongli (Mon, 25 Jun 2018 13:52:40 GMT): infowolfe (Mon, 25 Jun 2018 13:53:08 GMT): tongli (Mon, 25 Jun 2018 13:54:20 GMT): infowolfe (Mon, 25 Jun 2018 13:57:44 GMT): infowolfe (Mon, 25 Jun 2018 13:57:59 GMT): infowolfe (Mon, 25 Jun 2018 13:58:15 GMT): infowolfe (Mon, 25 Jun 2018 13:58:15 GMT): pankajcheema (Mon, 25 Jun 2018 14:00:40 GMT): pankajcheema (Mon, 25 Jun 2018 14:00:46 GMT): pankajcheema (Mon, 25 Jun 2018 14:01:35 GMT): pankajcheema (Mon, 25 Jun 2018 14:02:14 GMT): hightall (Mon, 25 Jun 2018 14:02:31 GMT): pankajcheema (Mon, 25 Jun 2018 14:02:53 GMT): pankajcheema (Mon, 25 Jun 2018 14:03:33 GMT): pankajcheema (Mon, 25 Jun 2018 14:03:49 GMT): hightall (Mon, 25 Jun 2018 14:04:08 GMT): pankajcheema (Mon, 25 Jun 2018 14:04:12 GMT): pankajcheema (Mon, 25 Jun 2018 14:04:14 GMT): hightall (Mon, 25 Jun 2018 14:04:16 GMT): hightall (Mon, 25 Jun 2018 14:04:22 GMT): pankajcheema (Mon, 25 Jun 2018 14:04:25 GMT): pankajcheema (Mon, 25 Jun 2018 14:05:41 GMT): pankajcheema (Mon, 25 Jun 2018 14:05:59 GMT): pankajcheema (Mon, 25 Jun 2018 14:06:08 GMT): pankajcheema (Mon, 25 Jun 2018 14:06:15 GMT): hightall (Mon, 25 Jun 2018 14:07:18 GMT): tongli (Mon, 25 Jun 2018 14:07:42 GMT): tongli (Mon, 25 Jun 2018 14:08:16 GMT): pankajcheema (Mon, 25 Jun 2018 14:08:32 GMT): pankajcheema (Mon, 25 Jun 2018 14:08:54 GMT): pankajcheema (Mon, 25 Jun 2018 14:09:03 GMT): tongli (Mon, 25 Jun 2018 14:09:27 GMT): pankajcheema (Mon, 25 Jun 2018 14:09:37 GMT): pankajcheema (Mon, 25 Jun 2018 14:09:37 GMT): tongli (Mon, 25 Jun 2018 14:09:42 GMT): pankajcheema (Mon, 25 Jun 2018 14:09:49 GMT): pankajcheema (Mon, 25 Jun 2018 14:09:52 GMT): pankajcheema (Mon, 25 Jun 2018 14:10:16 GMT): pankajcheema (Mon, 25 Jun 2018 14:10:20 GMT): pankajcheema (Mon, 25 Jun 2018 14:10:28 GMT): hightall (Mon, 25 Jun 2018 14:10:32 GMT): tongli (Mon, 25 Jun 2018 14:10:56 GMT): tongli (Mon, 25 Jun 2018 14:11:32 GMT): hightall (Mon, 25 Jun 2018 14:11:50 GMT): tongli (Mon, 25 Jun 2018 14:12:04 GMT): pankajcheema (Mon, 25 Jun 2018 14:12:11 GMT): hightall (Mon, 25 Jun 2018 14:13:06 GMT): tongli (Mon, 25 Jun 2018 14:13:23 GMT): pankajcheema (Mon, 25 Jun 2018 14:48:46 GMT): pankajcheema (Mon, 25 Jun 2018 14:48:55 GMT): hightall (Mon, 25 Jun 2018 14:49:06 GMT): pankajcheema (Mon, 25 Jun 2018 14:49:09 GMT): pankajcheema (Mon, 25 Jun 2018 14:49:15 GMT): aarthiKarthik (Tue, 26 Jun 2018 06:21:24 GMT): aarthiKarthik (Tue, 26 Jun 2018 06:21:34 GMT): aarthiKarthik (Tue, 26 Jun 2018 06:21:37 GMT): brendatian (Tue, 26 Jun 2018 09:08:33 GMT): brendatian (Tue, 26 Jun 2018 09:09:26 GMT): YashGanthe (Tue, 26 Jun 2018 10:15:11 GMT): mandeville135 (Tue, 26 Jun 2018 14:08:07 GMT): hightall (Tue, 26 Jun 2018 16:03:07 GMT): baohua (Tue, 26 Jun 2018 17:22:52 GMT): DavidPark (Tue, 26 Jun 2018 22:56:24 GMT): hightall (Wed, 27 Jun 2018 03:04:35 GMT): rkrish82 (Wed, 27 Jun 2018 03:41:17 GMT): rkrish82 (Wed, 27 Jun 2018 03:42:29 GMT): rkrish82 (Wed, 27 Jun 2018 03:45:28 GMT): rkrish82 (Wed, 27 Jun 2018 03:45:28 GMT): hightall (Wed, 27 Jun 2018 06:20:05 GMT): pankajcheema (Wed, 27 Jun 2018 06:21:05 GMT): hightall (Wed, 27 Jun 2018 06:21:09 GMT): hightall (Wed, 27 Jun 2018 06:21:49 GMT): pankajcheema (Wed, 27 Jun 2018 06:23:01 GMT): hightall (Wed, 27 Jun 2018 06:23:05 GMT): pankajcheema (Wed, 27 Jun 2018 06:23:14 GMT): hightall (Wed, 27 Jun 2018 06:23:22 GMT): pankajcheema (Wed, 27 Jun 2018 06:23:23 GMT): hightall (Wed, 27 Jun 2018 06:25:45 GMT): hightall (Wed, 27 Jun 2018 06:46:08 GMT): pankajcheema (Wed, 27 Jun 2018 07:04:51 GMT): hightall (Wed, 27 Jun 2018 07:05:05 GMT): pankajcheema (Wed, 27 Jun 2018 07:05:09 GMT): Tfabric (Wed, 27 Jun 2018 09:12:39 GMT): hightall (Wed, 27 Jun 2018 09:14:44 GMT): Tfabric (Wed, 27 Jun 2018 09:25:06 GMT): hightall (Wed, 27 Jun 2018 09:25:57 GMT): Tfabric (Wed, 27 Jun 2018 09:26:41 GMT): Tfabric (Wed, 27 Jun 2018 09:29:01 GMT): Tfabric (Wed, 27 Jun 2018 10:38:15 GMT): Tfabric (Wed, 27 Jun 2018 10:39:04 GMT): infowolfe (Wed, 27 Jun 2018 12:41:50 GMT): Tfabric (Wed, 27 Jun 2018 12:55:53 GMT): Tfabric (Wed, 27 Jun 2018 12:58:45 GMT): Tfabric (Wed, 27 Jun 2018 13:01:41 GMT): alain2sf (Wed, 27 Jun 2018 13:35:39 GMT): hightall (Thu, 28 Jun 2018 01:56:02 GMT): hightall (Thu, 28 Jun 2018 02:58:04 GMT): rkrish82 (Thu, 28 Jun 2018 03:26:39 GMT): rkrish82 (Thu, 28 Jun 2018 03:27:22 GMT): rkrish82 (Thu, 28 Jun 2018 03:27:50 GMT): hightall (Thu, 28 Jun 2018 03:28:06 GMT): rkrish82 (Thu, 28 Jun 2018 03:29:20 GMT): rkrish82 (Thu, 28 Jun 2018 03:29:20 GMT): tongli (Thu, 28 Jun 2018 07:23:55 GMT): tongli (Thu, 28 Jun 2018 07:28:40 GMT): tongli (Thu, 28 Jun 2018 07:29:21 GMT): rkrish82 (Thu, 28 Jun 2018 07:29:47 GMT): tongli (Thu, 28 Jun 2018 07:30:09 GMT): tongli (Thu, 28 Jun 2018 07:30:22 GMT): tongli (Thu, 28 Jun 2018 07:31:52 GMT): rkrish82 (Thu, 28 Jun 2018 07:33:06 GMT): hightall (Thu, 28 Jun 2018 07:33:49 GMT): kodepareek (Thu, 28 Jun 2018 08:35:05 GMT): Tfabric (Thu, 28 Jun 2018 10:01:43 GMT): tongli (Thu, 28 Jun 2018 14:41:11 GMT): jwagantall (Thu, 28 Jun 2018 22:07:14 GMT): luckydogchina (Fri, 29 Jun 2018 07:52:09 GMT): luckydogchina (Fri, 29 Jun 2018 07:52:09 GMT): luckydogchina (Fri, 29 Jun 2018 07:52:09 GMT): luckydogchina (Fri, 29 Jun 2018 07:52:09 GMT): leofantast (Fri, 29 Jun 2018 09:25:43 GMT): AbidiBassem (Fri, 29 Jun 2018 11:05:44 GMT): luckydogchina (Mon, 02 Jul 2018 03:46:23 GMT): luckydogchina (Mon, 02 Jul 2018 03:46:45 GMT): luckydogchina (Mon, 02 Jul 2018 03:46:45 GMT): luckydogchina (Mon, 02 Jul 2018 03:46:45 GMT): luckydogchina (Mon, 02 Jul 2018 03:46:45 GMT): hightall (Mon, 02 Jul 2018 08:04:28 GMT): n1zyz (Mon, 02 Jul 2018 12:24:06 GMT): baohua (Mon, 02 Jul 2018 12:58:29 GMT): nemo 8 (Mon, 02 Jul 2018 14:15:30 GMT): nemo 8 (Mon, 02 Jul 2018 14:15:35 GMT): dexhunter (Mon, 02 Jul 2018 14:25:06 GMT): hightall (Mon, 02 Jul 2018 14:31:25 GMT): frustrated-chain (Mon, 02 Jul 2018 18:35:56 GMT): frustrated-chain (Mon, 02 Jul 2018 18:35:56 GMT): titog (Mon, 02 Jul 2018 19:44:54 GMT): baohua (Tue, 03 Jul 2018 01:09:17 GMT): baohua (Tue, 03 Jul 2018 01:09:17 GMT): RealDeanZhao (Tue, 03 Jul 2018 01:51:55 GMT): hightall (Tue, 03 Jul 2018 06:39:06 GMT): qsmen (Tue, 03 Jul 2018 07:05:40 GMT): hightall (Tue, 03 Jul 2018 07:13:39 GMT): luckydogchina (Tue, 03 Jul 2018 07:14:08 GMT): hightall (Tue, 03 Jul 2018 07:15:12 GMT): luckydogchina (Tue, 03 Jul 2018 07:15:46 GMT): luckydogchina (Tue, 03 Jul 2018 07:15:46 GMT): hightall (Tue, 03 Jul 2018 07:18:32 GMT): hightall (Tue, 03 Jul 2018 07:18:54 GMT): hightall (Tue, 03 Jul 2018 07:19:09 GMT): hightall (Tue, 03 Jul 2018 07:19:30 GMT): hightall (Tue, 03 Jul 2018 07:21:42 GMT): luckydogchina (Tue, 03 Jul 2018 07:24:59 GMT): hightall (Tue, 03 Jul 2018 07:27:01 GMT): hightall (Tue, 03 Jul 2018 07:27:56 GMT): luckydogchina (Tue, 03 Jul 2018 07:33:09 GMT): qsmen (Tue, 03 Jul 2018 07:47:51 GMT): qsmen (Tue, 03 Jul 2018 07:47:51 GMT): hightall (Tue, 03 Jul 2018 07:49:30 GMT): qsmen (Tue, 03 Jul 2018 08:03:04 GMT): qsmen (Tue, 03 Jul 2018 08:26:54 GMT): qsmen (Tue, 03 Jul 2018 08:26:54 GMT): hightall (Tue, 03 Jul 2018 08:29:25 GMT): qsmen (Tue, 03 Jul 2018 08:39:43 GMT): qsmen (Tue, 03 Jul 2018 08:42:45 GMT): qsmen (Tue, 03 Jul 2018 08:42:45 GMT): hightall (Tue, 03 Jul 2018 08:45:17 GMT): qsmen (Tue, 03 Jul 2018 08:51:19 GMT): nemo 8 (Tue, 03 Jul 2018 10:26:06 GMT): nemo 8 (Tue, 03 Jul 2018 10:30:01 GMT): luckydogchina (Tue, 03 Jul 2018 11:24:01 GMT): tongli (Tue, 03 Jul 2018 11:33:48 GMT): nemo 8 (Tue, 03 Jul 2018 13:07:16 GMT): tripleOrange (Tue, 03 Jul 2018 17:31:57 GMT): Vanitha (Tue, 03 Jul 2018 18:57:44 GMT): Vanitha (Tue, 03 Jul 2018 18:57:49 GMT): baohua (Wed, 04 Jul 2018 03:13:44 GMT): baohua (Wed, 04 Jul 2018 03:14:01 GMT): luckydogchina (Wed, 04 Jul 2018 03:26:47 GMT): luckydogchina (Wed, 04 Jul 2018 04:01:32 GMT): luckydogchina (Wed, 04 Jul 2018 04:01:32 GMT): hightall (Wed, 04 Jul 2018 06:07:13 GMT): nemo 8 (Wed, 04 Jul 2018 09:05:45 GMT): guangyingyuan (Wed, 04 Jul 2018 13:21:38 GMT): guangyingyuan (Wed, 04 Jul 2018 13:21:40 GMT): guangyingyuan (Wed, 04 Jul 2018 13:21:40 GMT): guangyingyuan (Wed, 04 Jul 2018 13:21:40 GMT): BabuPallam (Wed, 04 Jul 2018 22:18:59 GMT): baohua (Thu, 05 Jul 2018 01:46:30 GMT): luckydogchina (Thu, 05 Jul 2018 02:22:24 GMT): guangyingyuan (Thu, 05 Jul 2018 08:59:53 GMT): guangyingyuan (Thu, 05 Jul 2018 09:19:32 GMT): Vanitha (Thu, 05 Jul 2018 12:01:17 GMT): Vanitha (Thu, 05 Jul 2018 12:02:00 GMT): Vanitha (Thu, 05 Jul 2018 12:02:14 GMT): frustrated-chain (Thu, 05 Jul 2018 15:59:59 GMT): QQ-ing (Fri, 06 Jul 2018 06:42:44 GMT): Vanitha (Fri, 06 Jul 2018 13:07:22 GMT): Vanitha (Fri, 06 Jul 2018 13:07:30 GMT): baohua (Sat, 07 Jul 2018 11:46:07 GMT): baohua (Sat, 07 Jul 2018 11:49:17 GMT): raymondfx (Sun, 08 Jul 2018 11:29:34 GMT): mna2016 (Sun, 08 Jul 2018 12:37:19 GMT): mna2016 (Sun, 08 Jul 2018 12:38:37 GMT): mna2016 (Sun, 08 Jul 2018 12:39:08 GMT): mna2016 (Sun, 08 Jul 2018 12:39:41 GMT): vmag (Sun, 08 Jul 2018 17:10:38 GMT): hightall (Mon, 09 Jul 2018 06:07:09 GMT): baohua (Mon, 09 Jul 2018 13:06:49 GMT): tongli (Mon, 09 Jul 2018 13:12:14 GMT): s2nazari (Mon, 09 Jul 2018 14:13:48 GMT): aileennguyen (Mon, 09 Jul 2018 18:52:07 GMT): Roeyc (Mon, 09 Jul 2018 23:35:29 GMT): rkrish82 (Tue, 10 Jul 2018 04:26:23 GMT): rkrish82 (Tue, 10 Jul 2018 04:26:23 GMT): rkrish82 (Tue, 10 Jul 2018 04:26:23 GMT): rkrish82 (Tue, 10 Jul 2018 04:27:39 GMT): Tfabric (Tue, 10 Jul 2018 09:03:58 GMT): tongli (Tue, 10 Jul 2018 15:22:11 GMT): WadeLu (Thu, 12 Jul 2018 07:27:28 GMT): tongli (Thu, 12 Jul 2018 16:27:56 GMT): baohua (Fri, 13 Jul 2018 01:06:09 GMT): ehsanhajian (Fri, 13 Jul 2018 09:32:55 GMT): WadeLu (Fri, 13 Jul 2018 09:57:36 GMT): WadeLu (Fri, 13 Jul 2018 09:59:23 GMT): WadeLu (Fri, 13 Jul 2018 10:00:44 GMT): WadeLu (Fri, 13 Jul 2018 10:01:35 GMT): jiulama (Sat, 14 Jul 2018 07:09:39 GMT): ageekymonk (Sat, 14 Jul 2018 22:56:37 GMT): ageekymonk (Sat, 14 Jul 2018 22:57:19 GMT): ageekymonk (Sat, 14 Jul 2018 22:57:19 GMT): baohua (Sun, 15 Jul 2018 01:29:15 GMT): ageekymonk (Sun, 15 Jul 2018 01:42:37 GMT): baohua (Sun, 15 Jul 2018 02:56:44 GMT): baohua (Sun, 15 Jul 2018 02:57:52 GMT): ageekymonk (Mon, 16 Jul 2018 23:41:16 GMT): ageekymonk (Mon, 16 Jul 2018 23:41:43 GMT): ageekymonk (Mon, 16 Jul 2018 23:42:25 GMT): ageekymonk (Mon, 16 Jul 2018 23:43:13 GMT): Luke_Chen (Tue, 17 Jul 2018 03:38:15 GMT): Luke_Chen (Tue, 17 Jul 2018 03:39:00 GMT): neewy (Tue, 17 Jul 2018 13:03:01 GMT): neewy (Tue, 17 Jul 2018 13:04:40 GMT): neewy (Tue, 17 Jul 2018 13:05:11 GMT): neewy (Tue, 17 Jul 2018 13:05:47 GMT): alokmatta (Tue, 17 Jul 2018 15:54:54 GMT): StevenP 4 (Tue, 17 Jul 2018 18:30:15 GMT): StevenP 4 (Tue, 17 Jul 2018 18:30:26 GMT): anusha100 (Wed, 18 Jul 2018 00:02:15 GMT): corshamax (Wed, 18 Jul 2018 00:05:45 GMT): anusha100 (Wed, 18 Jul 2018 00:05:53 GMT): anusha100 (Wed, 18 Jul 2018 00:05:53 GMT): anusha100 (Wed, 18 Jul 2018 00:05:53 GMT): FlavioSS (Wed, 18 Jul 2018 10:53:48 GMT): shrey.techie (Wed, 18 Jul 2018 12:23:03 GMT): shrey.techie (Wed, 18 Jul 2018 12:23:05 GMT): ranjan008 (Wed, 18 Jul 2018 12:41:16 GMT): baohua (Wed, 18 Jul 2018 13:13:11 GMT): baohua (Wed, 18 Jul 2018 13:13:35 GMT): tongli (Wed, 18 Jul 2018 13:15:05 GMT): tongli (Wed, 18 Jul 2018 13:16:00 GMT): tongli (Wed, 18 Jul 2018 13:17:03 GMT): tongli (Wed, 18 Jul 2018 13:17:28 GMT): tongli (Wed, 18 Jul 2018 13:18:16 GMT): tongli (Wed, 18 Jul 2018 13:18:34 GMT): StevenP 4 (Wed, 18 Jul 2018 13:20:22 GMT): latitiah (Wed, 18 Jul 2018 13:20:28 GMT): StevenP 4 (Wed, 18 Jul 2018 13:56:39 GMT): StevenP 4 (Wed, 18 Jul 2018 13:56:39 GMT): StevenP 4 (Wed, 18 Jul 2018 13:57:19 GMT): latitiah (Wed, 18 Jul 2018 14:47:42 GMT): latitiah (Wed, 18 Jul 2018 14:48:28 GMT): tongli (Thu, 19 Jul 2018 00:35:11 GMT): tongli (Thu, 19 Jul 2018 00:35:57 GMT): baohua (Thu, 19 Jul 2018 01:05:29 GMT): latitiah (Thu, 19 Jul 2018 01:08:31 GMT): latitiah (Thu, 19 Jul 2018 01:08:31 GMT): WadeLu (Thu, 19 Jul 2018 01:29:06 GMT): WadeLu (Thu, 19 Jul 2018 01:33:11 GMT): samir.tata (Thu, 19 Jul 2018 03:54:48 GMT): nao-someya (Thu, 19 Jul 2018 04:42:06 GMT): luckydogchina (Thu, 19 Jul 2018 05:27:05 GMT): luckydogchina (Thu, 19 Jul 2018 05:27:05 GMT): ranjan008 (Thu, 19 Jul 2018 07:38:48 GMT): SandeepPerugu (Thu, 19 Jul 2018 09:49:20 GMT): gokhru (Thu, 19 Jul 2018 11:04:19 GMT): baohua (Thu, 19 Jul 2018 14:01:22 GMT): ageekymonk (Thu, 19 Jul 2018 20:20:45 GMT): WadeLu (Fri, 20 Jul 2018 01:03:23 GMT): dhuseby (Fri, 20 Jul 2018 01:27:44 GMT): dhuseby (Fri, 20 Jul 2018 01:28:56 GMT): dhuseby (Fri, 20 Jul 2018 01:29:31 GMT): dhuseby (Fri, 20 Jul 2018 01:29:40 GMT): samir.tata (Fri, 20 Jul 2018 02:33:52 GMT): pandagopal (Fri, 20 Jul 2018 05:13:41 GMT): choco_coder (Fri, 20 Jul 2018 08:29:15 GMT): choco_coder (Fri, 20 Jul 2018 08:35:45 GMT): sathishNS (Fri, 20 Jul 2018 11:12:59 GMT): neewy (Fri, 20 Jul 2018 13:29:03 GMT): kevinmcmahon (Sat, 21 Jul 2018 19:42:20 GMT): JaswanthReddyV (Sun, 22 Jul 2018 10:44:30 GMT): Luke_Chen (Mon, 23 Jul 2018 03:41:52 GMT): ageekymonk (Mon, 23 Jul 2018 04:46:07 GMT): SineadKelly (Mon, 23 Jul 2018 11:33:25 GMT): bobzhao (Mon, 23 Jul 2018 15:24:30 GMT): baohua (Tue, 24 Jul 2018 02:47:42 GMT): baohua (Tue, 24 Jul 2018 02:47:50 GMT): luckydogchina (Tue, 24 Jul 2018 07:17:16 GMT): Luke_Chen (Tue, 24 Jul 2018 07:28:27 GMT): Luke_Chen (Tue, 24 Jul 2018 07:29:15 GMT): luckydogchina (Tue, 24 Jul 2018 07:31:16 GMT): luckydogchina (Tue, 24 Jul 2018 07:31:16 GMT): Luke_Chen (Tue, 24 Jul 2018 07:32:55 GMT): luckydogchina (Tue, 24 Jul 2018 07:34:51 GMT): luckydogchina (Tue, 24 Jul 2018 08:08:21 GMT): giniz (Tue, 24 Jul 2018 09:56:17 GMT): Luke_Chen (Tue, 24 Jul 2018 10:30:33 GMT): Luke_Chen (Tue, 24 Jul 2018 10:31:16 GMT): Russ.corsha (Tue, 24 Jul 2018 17:32:21 GMT): Russ.corsha (Tue, 24 Jul 2018 17:36:43 GMT): Russ.corsha (Tue, 24 Jul 2018 18:34:22 GMT): giniz (Wed, 25 Jul 2018 01:12:50 GMT): giniz (Wed, 25 Jul 2018 01:14:14 GMT): luckydogchina (Wed, 25 Jul 2018 02:14:44 GMT): luckydogchina (Wed, 25 Jul 2018 02:14:44 GMT): luckydogchina (Wed, 25 Jul 2018 02:19:04 GMT): Luke_Chen (Wed, 25 Jul 2018 02:34:01 GMT): Luke_Chen (Wed, 25 Jul 2018 02:34:15 GMT): Luke_Chen (Wed, 25 Jul 2018 02:35:04 GMT): luckydogchina (Wed, 25 Jul 2018 02:35:26 GMT): Luke_Chen (Wed, 25 Jul 2018 02:35:40 GMT): luckydogchina (Wed, 25 Jul 2018 02:36:13 GMT): Luke_Chen (Wed, 25 Jul 2018 02:36:27 GMT): luckydogchina (Wed, 25 Jul 2018 02:36:33 GMT): Luke_Chen (Wed, 25 Jul 2018 02:36:38 GMT): luckydogchina (Wed, 25 Jul 2018 02:36:55 GMT): luckydogchina (Wed, 25 Jul 2018 02:38:53 GMT): luckydogchina (Wed, 25 Jul 2018 02:39:22 GMT): luckydogchina (Wed, 25 Jul 2018 02:39:31 GMT): Luke_Chen (Wed, 25 Jul 2018 02:40:14 GMT): Luke_Chen (Wed, 25 Jul 2018 02:41:46 GMT): luckydogchina (Wed, 25 Jul 2018 02:56:45 GMT): luckydogchina (Wed, 25 Jul 2018 02:56:45 GMT): Luke_Chen (Wed, 25 Jul 2018 03:30:12 GMT): Luke_Chen (Wed, 25 Jul 2018 03:30:27 GMT): luckydogchina (Wed, 25 Jul 2018 03:37:49 GMT): luckydogchina (Wed, 25 Jul 2018 03:38:44 GMT): luckydogchina (Wed, 25 Jul 2018 03:38:44 GMT): kariyappals (Wed, 25 Jul 2018 07:06:15 GMT): baohua (Wed, 25 Jul 2018 08:41:38 GMT): tongli (Wed, 25 Jul 2018 13:01:35 GMT): kipharris (Wed, 25 Jul 2018 15:30:36 GMT): VenkataY (Wed, 25 Jul 2018 17:02:10 GMT): baohua (Thu, 26 Jul 2018 01:22:36 GMT): rkrish82 (Thu, 26 Jul 2018 03:46:00 GMT): naviat (Thu, 26 Jul 2018 08:58:15 GMT): baohua (Thu, 26 Jul 2018 09:07:44 GMT): baohua (Thu, 26 Jul 2018 09:08:07 GMT): baohua (Thu, 26 Jul 2018 14:40:24 GMT): luckydogchina (Fri, 27 Jul 2018 02:08:20 GMT): Krait82 (Fri, 27 Jul 2018 12:42:20 GMT): Russ.corsha (Fri, 27 Jul 2018 19:38:48 GMT): Russ.corsha (Fri, 27 Jul 2018 19:38:48 GMT): Russ.corsha (Fri, 27 Jul 2018 20:50:47 GMT): Russ.corsha (Fri, 27 Jul 2018 20:50:47 GMT): tongli (Fri, 27 Jul 2018 23:38:19 GMT): Russ.corsha (Fri, 27 Jul 2018 23:47:08 GMT): tongli (Sat, 28 Jul 2018 02:16:56 GMT): tongli (Sat, 28 Jul 2018 02:17:21 GMT): baohua (Sat, 28 Jul 2018 04:40:46 GMT): leondcastle (Sat, 28 Jul 2018 08:42:11 GMT): tongli (Sat, 28 Jul 2018 11:18:57 GMT): KeerthiJ (Sat, 28 Jul 2018 17:14:39 GMT): kipharris (Tue, 31 Jul 2018 00:00:27 GMT): kipharris (Tue, 31 Jul 2018 00:00:27 GMT): kipharris (Tue, 31 Jul 2018 00:00:27 GMT): baohua (Tue, 31 Jul 2018 02:23:16 GMT): angieology (Tue, 31 Jul 2018 17:48:09 GMT): kipharris (Tue, 31 Jul 2018 22:47:15 GMT): kipharris (Tue, 31 Jul 2018 23:09:53 GMT): kipharris (Tue, 31 Jul 2018 23:15:45 GMT): baohua (Wed, 01 Aug 2018 04:49:21 GMT): ameijer (Wed, 01 Aug 2018 13:36:10 GMT): iDix (Wed, 01 Aug 2018 14:28:05 GMT): jk.ananth (Thu, 02 Aug 2018 06:14:59 GMT): jk.ananth (Thu, 02 Aug 2018 06:16:34 GMT): owen.liu (Thu, 02 Aug 2018 08:36:23 GMT): owen.liu (Thu, 02 Aug 2018 08:40:05 GMT): owen.liu (Thu, 02 Aug 2018 08:40:58 GMT): owen.liu (Thu, 02 Aug 2018 08:43:49 GMT): jk.ananth (Thu, 02 Aug 2018 09:32:40 GMT): jk.ananth (Thu, 02 Aug 2018 09:33:41 GMT): yzhivkov (Thu, 02 Aug 2018 15:27:01 GMT): vinayakkumar (Fri, 03 Aug 2018 12:28:37 GMT): baohua (Fri, 03 Aug 2018 13:31:02 GMT): JayPandya (Fri, 03 Aug 2018 13:46:07 GMT): yzhivkov (Fri, 03 Aug 2018 21:17:35 GMT): yzhivkov (Fri, 03 Aug 2018 21:18:51 GMT): akshay.sood (Sat, 04 Aug 2018 06:45:39 GMT): rkrish82 (Mon, 06 Aug 2018 05:31:15 GMT): Luke_Chen (Mon, 06 Aug 2018 05:58:52 GMT): Luke_Chen (Mon, 06 Aug 2018 05:58:52 GMT): rkrish82 (Mon, 06 Aug 2018 07:13:23 GMT): hvandurme (Mon, 06 Aug 2018 11:51:32 GMT): Luke_Chen (Mon, 06 Aug 2018 12:59:19 GMT): Luke_Chen (Mon, 06 Aug 2018 12:59:19 GMT): Luke_Chen (Mon, 06 Aug 2018 12:59:19 GMT): Luke_Chen (Mon, 06 Aug 2018 12:59:34 GMT): janmpo (Mon, 06 Aug 2018 13:13:30 GMT): janmpo (Mon, 06 Aug 2018 13:28:32 GMT): janmpo (Mon, 06 Aug 2018 13:28:38 GMT): janmpo (Mon, 06 Aug 2018 13:32:51 GMT): janmpo (Mon, 06 Aug 2018 13:34:26 GMT): baohua (Tue, 07 Aug 2018 01:20:43 GMT): baohua (Tue, 07 Aug 2018 01:22:19 GMT): aguel (Tue, 07 Aug 2018 06:00:56 GMT): janmpo (Tue, 07 Aug 2018 06:22:41 GMT): janmpo (Tue, 07 Aug 2018 06:25:19 GMT): luckydogchina (Tue, 07 Aug 2018 06:50:50 GMT): luckydogchina (Tue, 07 Aug 2018 06:51:01 GMT): Aejnor (Tue, 07 Aug 2018 06:51:19 GMT): luckydogchina (Tue, 07 Aug 2018 06:51:27 GMT): luckydogchina (Tue, 07 Aug 2018 06:51:50 GMT): luckydogchina (Tue, 07 Aug 2018 06:52:27 GMT): luckydogchina (Tue, 07 Aug 2018 06:52:27 GMT): luckydogchina (Tue, 07 Aug 2018 07:01:47 GMT): luckydogchina (Tue, 07 Aug 2018 07:04:03 GMT): luckydogchina (Tue, 07 Aug 2018 07:04:03 GMT): luckydogchina (Tue, 07 Aug 2018 07:04:03 GMT): luckydogchina (Tue, 07 Aug 2018 07:04:03 GMT): luckydogchina (Tue, 07 Aug 2018 07:05:15 GMT): luckydogchina (Tue, 07 Aug 2018 07:05:15 GMT): luckydogchina (Tue, 07 Aug 2018 07:05:15 GMT): Luke_Chen (Tue, 07 Aug 2018 07:23:42 GMT): Luke_Chen (Tue, 07 Aug 2018 07:23:42 GMT): jk.ananth (Tue, 07 Aug 2018 08:15:31 GMT): VarunVerma (Tue, 07 Aug 2018 08:48:01 GMT): jk.ananth (Tue, 07 Aug 2018 08:55:16 GMT): jk.ananth (Tue, 07 Aug 2018 08:56:32 GMT): Luke_Chen (Tue, 07 Aug 2018 09:22:34 GMT): Luke_Chen (Tue, 07 Aug 2018 09:22:56 GMT): Luke_Chen (Tue, 07 Aug 2018 09:23:16 GMT): ameijer (Tue, 07 Aug 2018 19:04:05 GMT): jramirez (Tue, 07 Aug 2018 21:43:41 GMT): Luke_Chen (Wed, 08 Aug 2018 03:01:22 GMT): indira.kalagara (Wed, 08 Aug 2018 08:30:09 GMT): giniz (Wed, 08 Aug 2018 08:47:45 GMT): baohua (Wed, 08 Aug 2018 08:57:00 GMT): baohua (Wed, 08 Aug 2018 08:57:40 GMT): thPart (Wed, 08 Aug 2018 09:02:36 GMT): easeev (Wed, 08 Aug 2018 13:46:26 GMT): ameijer (Wed, 08 Aug 2018 18:28:10 GMT): baohua (Thu, 09 Aug 2018 02:10:06 GMT): Luke_Chen (Thu, 09 Aug 2018 05:31:34 GMT): Luke_Chen (Thu, 09 Aug 2018 05:33:17 GMT): jk.ananth (Thu, 09 Aug 2018 06:51:27 GMT): jk.ananth (Thu, 09 Aug 2018 06:52:07 GMT): jk.ananth (Thu, 09 Aug 2018 06:52:20 GMT): Luke_Chen (Thu, 09 Aug 2018 07:17:05 GMT): jk.ananth (Thu, 09 Aug 2018 07:20:41 GMT): jk.ananth (Thu, 09 Aug 2018 07:22:08 GMT): Luke_Chen (Thu, 09 Aug 2018 07:29:34 GMT): jk.ananth (Thu, 09 Aug 2018 07:32:00 GMT): Luke_Chen (Thu, 09 Aug 2018 07:38:14 GMT): uhaider (Thu, 09 Aug 2018 07:54:19 GMT): newthinker (Thu, 09 Aug 2018 11:54:18 GMT): jeevas (Thu, 09 Aug 2018 13:06:29 GMT): jeevas (Thu, 09 Aug 2018 13:16:52 GMT): jeevas (Thu, 09 Aug 2018 13:16:52 GMT): Luke_Chen (Thu, 09 Aug 2018 13:54:15 GMT): jeevas (Thu, 09 Aug 2018 16:35:59 GMT): luckydogchina (Fri, 10 Aug 2018 02:56:20 GMT): luckydogchina (Fri, 10 Aug 2018 02:56:20 GMT): newthinker (Fri, 10 Aug 2018 02:59:10 GMT): newthinker (Fri, 10 Aug 2018 03:00:01 GMT): newthinker (Fri, 10 Aug 2018 03:00:22 GMT): newthinker (Fri, 10 Aug 2018 03:00:41 GMT): newthinker (Fri, 10 Aug 2018 03:00:50 GMT): luckydogchina (Fri, 10 Aug 2018 03:03:59 GMT): luckydogchina (Fri, 10 Aug 2018 03:03:59 GMT): luckydogchina (Fri, 10 Aug 2018 03:03:59 GMT): luckydogchina (Fri, 10 Aug 2018 03:03:59 GMT): newthinker (Fri, 10 Aug 2018 03:16:23 GMT): newthinker (Fri, 10 Aug 2018 03:19:33 GMT): luckydogchina (Fri, 10 Aug 2018 04:06:55 GMT): newthinker (Fri, 10 Aug 2018 06:39:30 GMT): newthinker (Fri, 10 Aug 2018 06:46:37 GMT): jk.ananth (Fri, 10 Aug 2018 07:14:17 GMT): jk.ananth (Fri, 10 Aug 2018 07:14:17 GMT): Luke_Chen (Fri, 10 Aug 2018 07:32:56 GMT): luckydogchina (Fri, 10 Aug 2018 09:05:39 GMT): newthinker (Fri, 10 Aug 2018 09:45:03 GMT): newthinker (Fri, 10 Aug 2018 09:45:49 GMT): luckydogchina (Fri, 10 Aug 2018 09:50:11 GMT): luckydogchina (Fri, 10 Aug 2018 09:50:11 GMT): newthinker (Fri, 10 Aug 2018 09:53:54 GMT): newthinker (Fri, 10 Aug 2018 09:54:00 GMT): luckydogchina (Fri, 10 Aug 2018 09:57:28 GMT): newthinker (Fri, 10 Aug 2018 09:58:33 GMT): jeevas (Fri, 10 Aug 2018 12:24:35 GMT): jeevas (Fri, 10 Aug 2018 12:24:42 GMT): jeevas (Fri, 10 Aug 2018 12:29:54 GMT): KeerthiJ (Fri, 10 Aug 2018 14:39:08 GMT): YuanOwen (Sun, 12 Aug 2018 09:18:15 GMT): aroca (Sun, 12 Aug 2018 21:02:11 GMT): aroca (Sun, 12 Aug 2018 23:43:39 GMT): aroca (Sun, 12 Aug 2018 23:45:33 GMT): jk.ananth (Tue, 14 Aug 2018 07:41:02 GMT): Javi (Tue, 14 Aug 2018 15:43:48 GMT): Javi (Tue, 14 Aug 2018 15:49:09 GMT): Javi (Tue, 14 Aug 2018 15:49:18 GMT): Javi (Tue, 14 Aug 2018 15:49:56 GMT): Javi (Tue, 14 Aug 2018 15:50:23 GMT): Javi (Tue, 14 Aug 2018 15:51:50 GMT): Javi (Tue, 14 Aug 2018 16:44:17 GMT): Javi (Tue, 14 Aug 2018 16:45:28 GMT): baohua (Tue, 14 Aug 2018 19:57:06 GMT): Avinash_Kancharla (Tue, 14 Aug 2018 20:22:25 GMT): moyating (Wed, 15 Aug 2018 03:37:59 GMT): baohua (Wed, 15 Aug 2018 14:56:47 GMT): newthinker (Thu, 16 Aug 2018 02:34:14 GMT): newthinker (Thu, 16 Aug 2018 02:34:37 GMT): NicklasSandgren (Thu, 16 Aug 2018 08:24:37 GMT): NicklasSandgren (Thu, 16 Aug 2018 08:32:19 GMT): NicklasSandgren (Thu, 16 Aug 2018 08:35:42 GMT): jeevas (Thu, 16 Aug 2018 09:57:17 GMT): jeevas (Thu, 16 Aug 2018 09:57:17 GMT): jk.ananth (Thu, 16 Aug 2018 10:29:51 GMT): Luke_Chen (Thu, 16 Aug 2018 10:42:00 GMT): Luke_Chen (Thu, 16 Aug 2018 10:44:32 GMT): Javi (Thu, 16 Aug 2018 11:47:23 GMT): Javi (Thu, 16 Aug 2018 11:51:16 GMT): oleg_ (Fri, 17 Aug 2018 05:37:06 GMT): newthinker (Fri, 17 Aug 2018 05:44:15 GMT): Anshulmittal (Fri, 17 Aug 2018 06:09:31 GMT): jeevas (Fri, 17 Aug 2018 06:22:37 GMT): yzhivkov (Fri, 17 Aug 2018 17:41:12 GMT): shyammishra (Sat, 18 Aug 2018 07:39:47 GMT): YuanOwen (Sat, 18 Aug 2018 12:55:39 GMT): yzhivkov (Sat, 18 Aug 2018 13:09:37 GMT): YuanOwen (Sat, 18 Aug 2018 13:10:33 GMT): YuanOwen (Sat, 18 Aug 2018 13:10:51 GMT): yzhivkov (Sat, 18 Aug 2018 13:11:02 GMT): YuanOwen (Sat, 18 Aug 2018 13:12:17 GMT): YuanOwen (Sat, 18 Aug 2018 14:50:32 GMT): YuanOwen (Sat, 18 Aug 2018 14:52:55 GMT): YuanOwen (Sun, 19 Aug 2018 10:16:24 GMT): YuanOwen (Sun, 19 Aug 2018 10:16:24 GMT): lotty02cho (Mon, 20 Aug 2018 10:49:35 GMT): lotty02cho (Mon, 20 Aug 2018 10:53:39 GMT): baohua (Tue, 21 Aug 2018 01:37:26 GMT): baohua (Tue, 21 Aug 2018 01:38:32 GMT): baohua (Tue, 21 Aug 2018 02:14:00 GMT): vdods (Tue, 21 Aug 2018 14:10:44 GMT): NicklasSandgren (Tue, 21 Aug 2018 14:16:30 GMT): aanugu (Tue, 21 Aug 2018 21:13:43 GMT): bdjidi (Tue, 21 Aug 2018 22:51:03 GMT): Luke_Chen (Wed, 22 Aug 2018 02:15:24 GMT): MohammedR (Wed, 22 Aug 2018 07:06:36 GMT): baohua (Wed, 22 Aug 2018 08:27:46 GMT): kosullivan_sita (Wed, 22 Aug 2018 08:56:25 GMT): NicklasSandgren (Wed, 22 Aug 2018 09:22:41 GMT): baohua (Wed, 22 Aug 2018 09:27:02 GMT): NicklasSandgren (Wed, 22 Aug 2018 09:36:52 GMT): MohammedR (Wed, 22 Aug 2018 10:32:03 GMT): karthikmohan91 (Wed, 22 Aug 2018 12:14:06 GMT): chawlanikhil24 (Wed, 22 Aug 2018 13:20:11 GMT): chawlanikhil24 (Wed, 22 Aug 2018 13:20:35 GMT): knagware9 (Wed, 22 Aug 2018 13:24:46 GMT): chawlanikhil24 (Wed, 22 Aug 2018 13:25:39 GMT): chawlanikhil24 (Wed, 22 Aug 2018 13:26:03 GMT): chawlanikhil24 (Wed, 22 Aug 2018 13:26:35 GMT): knagware9 (Wed, 22 Aug 2018 13:27:49 GMT): chawlanikhil24 (Wed, 22 Aug 2018 13:28:51 GMT): knagware9 (Wed, 22 Aug 2018 13:29:24 GMT): knagware9 (Wed, 22 Aug 2018 13:29:41 GMT): chawlanikhil24 (Wed, 22 Aug 2018 13:30:20 GMT): chawlanikhil24 (Wed, 22 Aug 2018 13:30:44 GMT): chawlanikhil24 (Wed, 22 Aug 2018 13:33:11 GMT): knagware9 (Wed, 22 Aug 2018 13:35:41 GMT): chawlanikhil24 (Wed, 22 Aug 2018 13:40:09 GMT): tongli (Wed, 22 Aug 2018 23:29:45 GMT): tongli (Wed, 22 Aug 2018 23:30:17 GMT): tongli (Wed, 22 Aug 2018 23:30:34 GMT): HongweiPeng (Thu, 23 Aug 2018 01:01:36 GMT): HongweiPeng (Thu, 23 Aug 2018 01:05:23 GMT): HongweiPeng (Thu, 23 Aug 2018 01:06:44 GMT): baohua (Thu, 23 Aug 2018 01:48:08 GMT): baohua (Thu, 23 Aug 2018 02:33:10 GMT): tongli (Thu, 23 Aug 2018 02:42:09 GMT): HongweiPeng (Thu, 23 Aug 2018 02:43:09 GMT): chawlanikhil24 (Thu, 23 Aug 2018 04:38:45 GMT): chawlanikhil24 (Thu, 23 Aug 2018 04:39:27 GMT): chawlanikhil24 (Thu, 23 Aug 2018 04:46:46 GMT): baohua (Thu, 23 Aug 2018 04:57:35 GMT): baohua (Thu, 23 Aug 2018 04:57:40 GMT): tongli (Thu, 23 Aug 2018 13:29:26 GMT): tongli (Thu, 23 Aug 2018 13:29:26 GMT): MohammedR (Thu, 23 Aug 2018 14:14:31 GMT): MohammedR (Thu, 23 Aug 2018 14:16:26 GMT): MohammedR (Thu, 23 Aug 2018 14:18:06 GMT): MohammedR (Thu, 23 Aug 2018 14:37:44 GMT): MohammedR (Thu, 23 Aug 2018 14:39:14 GMT): baohua (Fri, 24 Aug 2018 02:56:32 GMT): baohua (Fri, 24 Aug 2018 07:39:20 GMT): Luke_Chen (Fri, 24 Aug 2018 09:46:55 GMT): Luke_Chen (Fri, 24 Aug 2018 09:47:42 GMT): VenkataY (Fri, 24 Aug 2018 10:23:06 GMT): VenkataY (Fri, 24 Aug 2018 10:28:09 GMT): VenkataY (Fri, 24 Aug 2018 10:29:17 GMT): Luke_Chen (Fri, 24 Aug 2018 12:28:49 GMT): baohua (Fri, 24 Aug 2018 13:19:56 GMT): baohua (Fri, 24 Aug 2018 13:20:30 GMT): VenkataY (Sat, 25 Aug 2018 17:11:51 GMT): StefanRehm (Sat, 25 Aug 2018 21:49:05 GMT): Luke_Chen (Sun, 26 Aug 2018 11:03:08 GMT): vanclief (Sun, 26 Aug 2018 14:35:38 GMT): baohua (Mon, 27 Aug 2018 01:58:46 GMT): Luke_Chen (Mon, 27 Aug 2018 02:02:28 GMT): Luke_Chen (Mon, 27 Aug 2018 11:31:16 GMT): Luke_Chen (Mon, 27 Aug 2018 11:39:51 GMT): baohua (Mon, 27 Aug 2018 13:22:13 GMT): baohua (Mon, 27 Aug 2018 13:23:39 GMT): VenkataY (Mon, 27 Aug 2018 13:58:44 GMT): Anshulmittal (Mon, 27 Aug 2018 17:33:44 GMT): Anshulmittal (Mon, 27 Aug 2018 17:34:27 GMT): Luke_Chen (Tue, 28 Aug 2018 03:19:52 GMT): Luke_Chen (Tue, 28 Aug 2018 03:22:55 GMT): Luke_Chen (Tue, 28 Aug 2018 06:13:40 GMT): Luke_Chen (Tue, 28 Aug 2018 06:24:30 GMT): knagware9 (Tue, 28 Aug 2018 12:40:55 GMT): Anil-Co-Create-Labs (Tue, 28 Aug 2018 13:18:03 GMT): Luke_Chen (Tue, 28 Aug 2018 18:17:16 GMT): Luke_Chen (Tue, 28 Aug 2018 18:17:23 GMT): Luke_Chen (Wed, 29 Aug 2018 03:30:06 GMT): Ibraxos (Wed, 29 Aug 2018 04:51:21 GMT): Ibraxos (Wed, 29 Aug 2018 04:59:34 GMT): Luke_Chen (Wed, 29 Aug 2018 05:34:26 GMT): Luke_Chen (Wed, 29 Aug 2018 05:36:42 GMT): Luke_Chen (Wed, 29 Aug 2018 05:38:16 GMT): knagware9 (Wed, 29 Aug 2018 06:01:16 GMT): Luke_Chen (Wed, 29 Aug 2018 06:46:46 GMT): Luke_Chen (Wed, 29 Aug 2018 06:46:54 GMT): knagware9 (Wed, 29 Aug 2018 06:51:50 GMT): Luke_Chen (Wed, 29 Aug 2018 06:54:33 GMT): Luke_Chen (Wed, 29 Aug 2018 06:56:03 GMT): knagware9 (Wed, 29 Aug 2018 07:08:51 GMT): knagware9 (Wed, 29 Aug 2018 07:09:00 GMT): Luke_Chen (Wed, 29 Aug 2018 07:14:07 GMT): Luke_Chen (Wed, 29 Aug 2018 07:14:14 GMT): hightall (Wed, 29 Aug 2018 07:44:54 GMT): baohua (Wed, 29 Aug 2018 08:06:36 GMT): knagware9 (Wed, 29 Aug 2018 09:15:12 GMT): knagware9 (Wed, 29 Aug 2018 09:16:10 GMT): knagware9 (Wed, 29 Aug 2018 09:16:27 GMT): knagware9 (Wed, 29 Aug 2018 09:16:34 GMT): Luke_Chen (Wed, 29 Aug 2018 09:50:21 GMT): Luke_Chen (Wed, 29 Aug 2018 09:50:21 GMT): Luke_Chen (Wed, 29 Aug 2018 09:54:18 GMT): Luke_Chen (Wed, 29 Aug 2018 09:54:18 GMT): knagware9 (Wed, 29 Aug 2018 10:20:39 GMT): Ibraxos (Wed, 29 Aug 2018 12:45:36 GMT): asaningmaxchain123 (Wed, 29 Aug 2018 13:04:25 GMT): asaningmaxchain123 (Wed, 29 Aug 2018 13:04:25 GMT): knagware9 (Wed, 29 Aug 2018 13:05:12 GMT): asaningmaxchain123 (Wed, 29 Aug 2018 13:19:33 GMT): knagware9 (Wed, 29 Aug 2018 13:44:25 GMT): JohnR (Wed, 29 Aug 2018 14:09:31 GMT): Luke_Chen (Wed, 29 Aug 2018 16:03:48 GMT): Luke_Chen (Wed, 29 Aug 2018 16:04:49 GMT): Luke_Chen (Wed, 29 Aug 2018 16:10:06 GMT): Luke_Chen (Wed, 29 Aug 2018 16:10:27 GMT): hightall (Thu, 30 Aug 2018 01:45:06 GMT): asaningmaxchain123 (Thu, 30 Aug 2018 01:55:10 GMT): asaningmaxchain123 (Thu, 30 Aug 2018 02:00:17 GMT): baohua (Thu, 30 Aug 2018 02:43:22 GMT): asaningmaxchain123 (Thu, 30 Aug 2018 03:14:06 GMT): asaningmaxchain123 (Thu, 30 Aug 2018 03:14:58 GMT): hightall (Thu, 30 Aug 2018 03:44:31 GMT): baohua (Thu, 30 Aug 2018 04:47:57 GMT): asaningmaxchain123 (Thu, 30 Aug 2018 04:51:39 GMT): asaningmaxchain123 (Thu, 30 Aug 2018 04:51:48 GMT): asaningmaxchain123 (Thu, 30 Aug 2018 04:51:53 GMT): asaningmaxchain123 (Thu, 30 Aug 2018 04:51:53 GMT): baohua (Thu, 30 Aug 2018 05:09:19 GMT): baohua (Thu, 30 Aug 2018 05:12:10 GMT): Ibraxos (Thu, 30 Aug 2018 05:15:50 GMT): asaningmaxchain123 (Thu, 30 Aug 2018 05:16:37 GMT): Ibraxos (Thu, 30 Aug 2018 05:19:05 GMT): asaningmaxchain123 (Thu, 30 Aug 2018 05:19:35 GMT): asaningmaxchain123 (Thu, 30 Aug 2018 05:19:53 GMT): Ibraxos (Thu, 30 Aug 2018 05:20:15 GMT): GowriR (Thu, 30 Aug 2018 05:20:41 GMT): GowriR (Thu, 30 Aug 2018 05:23:36 GMT): GowriR (Thu, 30 Aug 2018 05:23:42 GMT): Ibraxos (Thu, 30 Aug 2018 05:26:31 GMT): GowriR (Thu, 30 Aug 2018 05:27:55 GMT): Ibraxos (Thu, 30 Aug 2018 05:31:05 GMT): Ibraxos (Thu, 30 Aug 2018 05:32:08 GMT): GowriR (Thu, 30 Aug 2018 05:33:04 GMT): baohua (Thu, 30 Aug 2018 05:45:38 GMT): asaningmaxchain123 (Thu, 30 Aug 2018 05:49:13 GMT): baohua (Thu, 30 Aug 2018 06:05:25 GMT): Ibraxos (Thu, 30 Aug 2018 06:15:45 GMT): baohua (Thu, 30 Aug 2018 06:18:01 GMT): Ibraxos (Thu, 30 Aug 2018 06:18:55 GMT): Ibraxos (Thu, 30 Aug 2018 06:18:55 GMT): hightall (Thu, 30 Aug 2018 06:26:46 GMT): baohua (Thu, 30 Aug 2018 06:42:03 GMT): GowriR (Thu, 30 Aug 2018 07:04:45 GMT): Luke_Chen (Thu, 30 Aug 2018 07:31:31 GMT): Luke_Chen (Thu, 30 Aug 2018 07:31:31 GMT): Ibraxos (Thu, 30 Aug 2018 07:33:47 GMT): Luke_Chen (Thu, 30 Aug 2018 07:37:53 GMT): Luke_Chen (Thu, 30 Aug 2018 07:38:43 GMT): Luke_Chen (Thu, 30 Aug 2018 07:38:43 GMT): Luke_Chen (Thu, 30 Aug 2018 07:39:48 GMT): Ibraxos (Thu, 30 Aug 2018 07:43:01 GMT): Ibraxos (Thu, 30 Aug 2018 07:43:10 GMT): Luke_Chen (Thu, 30 Aug 2018 07:45:00 GMT): Ibraxos (Thu, 30 Aug 2018 07:45:52 GMT): Ibraxos (Thu, 30 Aug 2018 07:47:10 GMT): Luke_Chen (Thu, 30 Aug 2018 07:50:42 GMT): Ibraxos (Thu, 30 Aug 2018 07:51:14 GMT): Luke_Chen (Thu, 30 Aug 2018 07:53:21 GMT): Ibraxos (Thu, 30 Aug 2018 08:00:03 GMT): huxiangdong (Thu, 30 Aug 2018 08:59:27 GMT): miceiken (Thu, 30 Aug 2018 09:36:49 GMT): miceiken (Thu, 30 Aug 2018 09:38:34 GMT): miceiken (Thu, 30 Aug 2018 09:46:19 GMT): miceiken (Thu, 30 Aug 2018 09:48:58 GMT): Luke_Chen (Thu, 30 Aug 2018 09:56:53 GMT): Luke_Chen (Thu, 30 Aug 2018 09:56:53 GMT): Luke_Chen (Thu, 30 Aug 2018 09:57:18 GMT): miceiken (Thu, 30 Aug 2018 10:17:28 GMT): miceiken (Thu, 30 Aug 2018 10:30:11 GMT): Ibraxos (Thu, 30 Aug 2018 10:36:32 GMT): miceiken (Thu, 30 Aug 2018 10:36:45 GMT): Ibraxos (Thu, 30 Aug 2018 10:37:17 GMT): Ibraxos (Thu, 30 Aug 2018 10:37:45 GMT): miceiken (Thu, 30 Aug 2018 10:38:03 GMT): Ibraxos (Thu, 30 Aug 2018 10:38:34 GMT): miceiken (Thu, 30 Aug 2018 10:43:52 GMT): miceiken (Thu, 30 Aug 2018 10:45:58 GMT): Ibraxos (Thu, 30 Aug 2018 10:51:18 GMT): miceiken (Thu, 30 Aug 2018 10:52:28 GMT): miceiken (Thu, 30 Aug 2018 10:54:13 GMT): miceiken (Thu, 30 Aug 2018 11:08:35 GMT): Ibraxos (Thu, 30 Aug 2018 11:10:19 GMT): miceiken (Thu, 30 Aug 2018 11:11:26 GMT): Ibraxos (Thu, 30 Aug 2018 11:16:18 GMT): miceiken (Thu, 30 Aug 2018 12:52:03 GMT): Ibraxos (Thu, 30 Aug 2018 12:56:44 GMT): miceiken (Thu, 30 Aug 2018 12:57:54 GMT): miceiken (Thu, 30 Aug 2018 12:57:57 GMT): miceiken (Thu, 30 Aug 2018 12:58:39 GMT): Ibraxos (Thu, 30 Aug 2018 13:11:08 GMT): miceiken (Thu, 30 Aug 2018 13:18:05 GMT): miceiken (Thu, 30 Aug 2018 13:18:21 GMT): miceiken (Thu, 30 Aug 2018 13:29:00 GMT): mgalat (Thu, 30 Aug 2018 13:37:34 GMT): miceiken (Thu, 30 Aug 2018 13:55:34 GMT): Luke_Chen (Thu, 30 Aug 2018 13:57:59 GMT): Luke_Chen (Thu, 30 Aug 2018 13:58:37 GMT): miceiken (Thu, 30 Aug 2018 13:59:08 GMT): miceiken (Thu, 30 Aug 2018 13:59:28 GMT): hightall (Thu, 30 Aug 2018 15:59:07 GMT): mgalat (Thu, 30 Aug 2018 17:42:19 GMT): compleatang (Thu, 30 Aug 2018 20:33:13 GMT): compleatang (Thu, 30 Aug 2018 20:35:08 GMT): compleatang (Thu, 30 Aug 2018 20:37:35 GMT): compleatang (Thu, 30 Aug 2018 20:38:11 GMT): compleatang (Thu, 30 Aug 2018 20:38:11 GMT): baohua (Fri, 31 Aug 2018 02:09:07 GMT): baohua (Fri, 31 Aug 2018 02:09:14 GMT): hightall (Fri, 31 Aug 2018 05:52:17 GMT): tongli (Fri, 31 Aug 2018 12:26:28 GMT): VenkataY (Fri, 31 Aug 2018 12:48:21 GMT): Ibraxos (Fri, 31 Aug 2018 12:57:40 GMT): sureshtedla (Fri, 31 Aug 2018 13:42:12 GMT): sureshtedla (Fri, 31 Aug 2018 17:17:31 GMT): sureshtedla (Fri, 31 Aug 2018 17:18:12 GMT): asaningmaxchain123 (Sat, 01 Sep 2018 03:17:31 GMT): asaningmaxchain123 (Sat, 01 Sep 2018 03:17:36 GMT): asaningmaxchain123 (Sat, 01 Sep 2018 03:17:56 GMT): baohua (Sat, 01 Sep 2018 07:46:20 GMT): baohua (Sat, 01 Sep 2018 07:46:55 GMT): anchit (Sat, 01 Sep 2018 09:53:41 GMT): sureshtedla (Sat, 01 Sep 2018 10:22:37 GMT): sureshtedla (Sat, 01 Sep 2018 10:24:04 GMT): Luke_Chen (Sun, 02 Sep 2018 07:35:49 GMT): Luke_Chen (Sun, 02 Sep 2018 07:36:03 GMT): asaningmaxchain123 (Sun, 02 Sep 2018 09:31:16 GMT): VenkataY (Sun, 02 Sep 2018 10:09:51 GMT): hightall (Sun, 02 Sep 2018 14:28:44 GMT): baohua (Mon, 03 Sep 2018 02:14:21 GMT): baohua (Mon, 03 Sep 2018 02:14:46 GMT): baohua (Mon, 03 Sep 2018 02:14:58 GMT): asaningmaxchain123 (Mon, 03 Sep 2018 05:36:06 GMT): asaningmaxchain123 (Mon, 03 Sep 2018 05:36:24 GMT): asaningmaxchain123 (Mon, 03 Sep 2018 05:53:19 GMT): hightall (Mon, 03 Sep 2018 06:37:19 GMT): ManriqueGarcia (Mon, 03 Sep 2018 07:30:54 GMT): bryq3 (Mon, 03 Sep 2018 08:23:35 GMT): bryq3 (Mon, 03 Sep 2018 08:23:49 GMT): saurabharora (Mon, 03 Sep 2018 08:56:20 GMT): asaningmaxchain123 (Mon, 03 Sep 2018 15:41:27 GMT): baohua (Tue, 04 Sep 2018 06:21:17 GMT): baohua (Tue, 04 Sep 2018 06:24:54 GMT): asaningmaxchain123 (Tue, 04 Sep 2018 07:24:26 GMT): asaningmaxchain123 (Tue, 04 Sep 2018 07:26:06 GMT): asaningmaxchain123 (Tue, 04 Sep 2018 07:26:06 GMT): Ibraxos (Tue, 04 Sep 2018 09:00:57 GMT): Ibraxos (Tue, 04 Sep 2018 09:00:59 GMT): jk.ananth (Tue, 04 Sep 2018 09:44:02 GMT): ameijer (Tue, 04 Sep 2018 12:30:46 GMT): StevenP 4 (Tue, 04 Sep 2018 14:19:27 GMT): brycebudd (Tue, 04 Sep 2018 16:00:43 GMT): rodolfoleal (Tue, 04 Sep 2018 21:34:38 GMT): baohua (Wed, 05 Sep 2018 01:40:24 GMT): baohua (Wed, 05 Sep 2018 01:40:37 GMT): asaningmaxchain123 (Wed, 05 Sep 2018 02:08:12 GMT): baohua (Wed, 05 Sep 2018 05:07:10 GMT): sureshtedla (Wed, 05 Sep 2018 10:44:24 GMT): sureshtedla (Wed, 05 Sep 2018 14:03:01 GMT): sureshtedla (Wed, 05 Sep 2018 14:03:01 GMT): sureshtedla (Wed, 05 Sep 2018 14:03:01 GMT): theathibm (Wed, 05 Sep 2018 14:09:44 GMT): ameijer (Wed, 05 Sep 2018 14:15:14 GMT): ameijer (Wed, 05 Sep 2018 14:15:14 GMT): ameijer (Wed, 05 Sep 2018 14:15:14 GMT): theathibm (Wed, 05 Sep 2018 15:54:35 GMT): theathibm (Wed, 05 Sep 2018 15:54:35 GMT): theathibm (Wed, 05 Sep 2018 16:18:44 GMT): baohua (Thu, 06 Sep 2018 02:35:46 GMT): theathibm (Thu, 06 Sep 2018 03:00:58 GMT): baohua (Thu, 06 Sep 2018 03:24:01 GMT): Luke_Chen (Thu, 06 Sep 2018 12:52:23 GMT): mighty-pirate (Thu, 06 Sep 2018 17:53:34 GMT): nfrunza (Thu, 06 Sep 2018 21:29:34 GMT): Ibraxos (Fri, 07 Sep 2018 08:31:08 GMT): hightall (Fri, 07 Sep 2018 08:47:10 GMT): asaningmaxchain123 (Fri, 07 Sep 2018 08:58:44 GMT): hightall (Fri, 07 Sep 2018 08:59:39 GMT): akshay.sood (Fri, 07 Sep 2018 09:44:21 GMT): akshay.sood (Fri, 07 Sep 2018 09:44:31 GMT): akshay.sood (Fri, 07 Sep 2018 09:46:20 GMT): baohua (Fri, 07 Sep 2018 14:51:53 GMT): baohua (Fri, 07 Sep 2018 14:52:16 GMT): nfrunza (Fri, 07 Sep 2018 20:37:14 GMT): greg2git (Sun, 09 Sep 2018 21:22:55 GMT): baohua (Mon, 10 Sep 2018 08:32:37 GMT): jayeshjawale95 (Mon, 10 Sep 2018 12:00:15 GMT): Javi (Mon, 10 Sep 2018 12:04:17 GMT): baohua (Tue, 11 Sep 2018 01:38:00 GMT): MariaHenderson (Tue, 11 Sep 2018 02:23:23 GMT): baohua (Tue, 11 Sep 2018 02:44:56 GMT): tongli (Tue, 11 Sep 2018 02:45:49 GMT): harsha (Tue, 11 Sep 2018 02:53:52 GMT): tongli (Tue, 11 Sep 2018 06:45:12 GMT): tongli (Tue, 11 Sep 2018 06:45:23 GMT): tongli (Tue, 11 Sep 2018 06:45:26 GMT): Gvedam (Tue, 11 Sep 2018 06:52:43 GMT): Gvedam (Tue, 11 Sep 2018 06:52:43 GMT): tongli (Tue, 11 Sep 2018 07:03:17 GMT): luckydogchina (Tue, 11 Sep 2018 08:00:38 GMT): Luke_Chen (Tue, 11 Sep 2018 08:05:40 GMT): luckydogchina (Tue, 11 Sep 2018 08:06:44 GMT): Ibraxos (Tue, 11 Sep 2018 09:29:03 GMT): Bhanu (Tue, 11 Sep 2018 13:08:38 GMT): BhaskarRaju (Wed, 12 Sep 2018 06:38:52 GMT): BhaskarRaju (Wed, 12 Sep 2018 06:41:00 GMT): BhaskarRaju (Wed, 12 Sep 2018 06:41:00 GMT): baohua (Wed, 12 Sep 2018 07:09:58 GMT): baohua (Wed, 12 Sep 2018 07:11:07 GMT): baohua (Wed, 12 Sep 2018 07:11:30 GMT): BhaskarRaju (Wed, 12 Sep 2018 07:12:11 GMT): baohua (Wed, 12 Sep 2018 07:14:32 GMT): iMacTia (Wed, 12 Sep 2018 14:12:24 GMT): iMacTia (Wed, 12 Sep 2018 14:16:29 GMT): iMacTia (Wed, 12 Sep 2018 14:16:49 GMT): iMacTia (Wed, 12 Sep 2018 14:17:37 GMT): iMacTia (Wed, 12 Sep 2018 14:18:47 GMT): iMacTia (Wed, 12 Sep 2018 14:19:22 GMT): iMacTia (Wed, 12 Sep 2018 14:19:50 GMT): Luke_Chen (Thu, 13 Sep 2018 03:42:52 GMT): gzkoala (Thu, 13 Sep 2018 04:04:06 GMT): Vishal3152 (Thu, 13 Sep 2018 05:09:12 GMT): Vishal3152 (Thu, 13 Sep 2018 07:58:07 GMT): Vishal3152 (Thu, 13 Sep 2018 07:58:42 GMT): Vishal3152 (Thu, 13 Sep 2018 07:58:43 GMT): Vishal3152 (Thu, 13 Sep 2018 07:59:08 GMT): Vishal3152 (Thu, 13 Sep 2018 07:59:09 GMT): Vishal3152 (Thu, 13 Sep 2018 07:59:25 GMT): Vishal3152 (Thu, 13 Sep 2018 08:18:42 GMT): Vishal3152 (Thu, 13 Sep 2018 08:18:49 GMT): Vishal3152 (Thu, 13 Sep 2018 08:18:53 GMT): Vishal3152 (Thu, 13 Sep 2018 08:19:01 GMT): iMacTia (Thu, 13 Sep 2018 08:38:56 GMT): iMacTia (Thu, 13 Sep 2018 08:41:07 GMT): iMacTia (Thu, 13 Sep 2018 08:41:44 GMT): iMacTia (Thu, 13 Sep 2018 08:50:42 GMT): iMacTia (Thu, 13 Sep 2018 08:51:50 GMT): iMacTia (Thu, 13 Sep 2018 08:53:05 GMT): Luke_Chen (Thu, 13 Sep 2018 11:17:24 GMT): Luke_Chen (Thu, 13 Sep 2018 11:18:38 GMT): iMacTia (Thu, 13 Sep 2018 11:23:17 GMT): iMacTia (Thu, 13 Sep 2018 11:23:33 GMT): iMacTia (Thu, 13 Sep 2018 11:23:58 GMT): iMacTia (Thu, 13 Sep 2018 11:24:12 GMT): iMacTia (Thu, 13 Sep 2018 11:24:31 GMT): iMacTia (Thu, 13 Sep 2018 11:24:58 GMT): Luke_Chen (Thu, 13 Sep 2018 11:31:58 GMT): Luke_Chen (Thu, 13 Sep 2018 11:34:12 GMT): Luke_Chen (Thu, 13 Sep 2018 11:36:10 GMT): Luke_Chen (Thu, 13 Sep 2018 11:36:42 GMT): Luke_Chen (Thu, 13 Sep 2018 11:37:17 GMT): iMacTia (Thu, 13 Sep 2018 11:46:26 GMT): iMacTia (Thu, 13 Sep 2018 11:46:41 GMT): iMacTia (Thu, 13 Sep 2018 11:48:31 GMT): iMacTia (Thu, 13 Sep 2018 11:53:15 GMT): Ibraxos (Thu, 13 Sep 2018 12:13:18 GMT): iMacTia (Thu, 13 Sep 2018 12:29:53 GMT): iMacTia (Thu, 13 Sep 2018 14:04:31 GMT): iMacTia (Thu, 13 Sep 2018 14:04:50 GMT): iMacTia (Thu, 13 Sep 2018 14:04:56 GMT): baohua (Thu, 13 Sep 2018 14:41:24 GMT): surfer76 (Thu, 13 Sep 2018 16:54:41 GMT): Luke_Chen (Fri, 14 Sep 2018 07:47:25 GMT): iMacTia (Fri, 14 Sep 2018 09:26:05 GMT): iMacTia (Fri, 14 Sep 2018 09:26:24 GMT): iMacTia (Fri, 14 Sep 2018 09:26:53 GMT): iMacTia (Fri, 14 Sep 2018 09:27:26 GMT): iMacTia (Fri, 14 Sep 2018 09:28:58 GMT): iMacTia (Fri, 14 Sep 2018 09:29:00 GMT): tahaf10 (Fri, 14 Sep 2018 10:30:57 GMT): baohua (Fri, 14 Sep 2018 12:30:50 GMT): bperetti (Fri, 14 Sep 2018 12:55:07 GMT): bperetti (Fri, 14 Sep 2018 12:57:24 GMT): baohua (Fri, 14 Sep 2018 13:14:28 GMT): baohua (Fri, 14 Sep 2018 13:14:51 GMT): JohnR (Fri, 14 Sep 2018 13:23:12 GMT): JohnR (Fri, 14 Sep 2018 13:23:47 GMT): JohnR (Fri, 14 Sep 2018 13:23:59 GMT): baohua (Fri, 14 Sep 2018 13:25:06 GMT): JohnR (Fri, 14 Sep 2018 13:26:07 GMT): JohnR (Fri, 14 Sep 2018 13:27:01 GMT): silliman (Fri, 14 Sep 2018 13:28:25 GMT): JohnR (Fri, 14 Sep 2018 13:32:08 GMT): bperetti (Fri, 14 Sep 2018 13:32:08 GMT): JohnR (Fri, 14 Sep 2018 13:32:16 GMT): baohua (Fri, 14 Sep 2018 13:46:52 GMT): fsl (Mon, 17 Sep 2018 02:17:38 GMT): jackyu66 (Mon, 17 Sep 2018 06:16:30 GMT): Luke_Chen (Tue, 18 Sep 2018 12:12:18 GMT): baohua (Tue, 18 Sep 2018 12:51:41 GMT): youngsunglee (Wed, 19 Sep 2018 06:29:54 GMT): youngsunglee (Wed, 19 Sep 2018 06:30:58 GMT): iampo12 (Wed, 19 Sep 2018 09:29:30 GMT): callmeashok (Wed, 19 Sep 2018 09:39:19 GMT): Levilk (Wed, 19 Sep 2018 13:43:38 GMT): Levilk (Wed, 19 Sep 2018 13:47:15 GMT): asaningmaxchain123 (Thu, 20 Sep 2018 05:14:11 GMT): JohnR (Thu, 20 Sep 2018 08:35:18 GMT): asaningmaxchain123 (Thu, 20 Sep 2018 14:33:11 GMT): baohua (Thu, 20 Sep 2018 14:34:54 GMT): asaningmaxchain123 (Thu, 20 Sep 2018 14:36:24 GMT): baohua (Thu, 20 Sep 2018 14:46:17 GMT): ColeBoudreau (Fri, 21 Sep 2018 14:09:47 GMT): ColeBoudreau (Fri, 21 Sep 2018 14:11:37 GMT): Aswath8687 (Fri, 21 Sep 2018 16:15:19 GMT): compleatang (Fri, 21 Sep 2018 17:46:05 GMT): ColeBoudreau (Fri, 21 Sep 2018 20:04:50 GMT): ColeBoudreau (Fri, 21 Sep 2018 20:06:10 GMT): baohua (Fri, 21 Sep 2018 23:58:07 GMT): Randyshu2018 (Sun, 23 Sep 2018 12:13:20 GMT): nxng (Sun, 23 Sep 2018 21:02:01 GMT): nxng (Sun, 23 Sep 2018 21:03:32 GMT): geethanisp (Mon, 24 Sep 2018 00:55:34 GMT): geethanisp (Mon, 24 Sep 2018 00:57:15 GMT): geethanisp (Mon, 24 Sep 2018 00:57:15 GMT): geethanisp (Mon, 24 Sep 2018 00:57:15 GMT): geethanisp (Mon, 24 Sep 2018 01:00:06 GMT): Randyshu2018 (Mon, 24 Sep 2018 02:09:39 GMT): tahaf10 (Mon, 24 Sep 2018 09:48:01 GMT): geethanisp (Mon, 24 Sep 2018 15:26:49 GMT): geethanisp (Mon, 24 Sep 2018 15:26:49 GMT): geethanisp (Mon, 24 Sep 2018 15:26:49 GMT): greg2git (Mon, 24 Sep 2018 15:28:53 GMT): geethanisp (Mon, 24 Sep 2018 15:31:14 GMT): ColeBoudreau (Mon, 24 Sep 2018 17:37:04 GMT): ColeBoudreau (Mon, 24 Sep 2018 20:07:47 GMT): baohua (Tue, 25 Sep 2018 01:28:24 GMT): hightall (Tue, 25 Sep 2018 06:16:07 GMT): geethanisp (Tue, 25 Sep 2018 08:09:20 GMT): geethanisp (Tue, 25 Sep 2018 08:09:20 GMT): geethanisp (Tue, 25 Sep 2018 08:09:20 GMT): nxng (Tue, 25 Sep 2018 08:11:18 GMT): baohua (Tue, 25 Sep 2018 08:25:09 GMT): baohua (Tue, 25 Sep 2018 08:37:14 GMT): geethanisp (Tue, 25 Sep 2018 09:00:03 GMT): geethanisp (Tue, 25 Sep 2018 09:00:03 GMT): baohua (Tue, 25 Sep 2018 09:04:42 GMT): baohua (Tue, 25 Sep 2018 09:05:07 GMT): geethanisp (Tue, 25 Sep 2018 09:32:30 GMT): geethanisp (Tue, 25 Sep 2018 09:32:30 GMT): alexvicegrab (Tue, 25 Sep 2018 14:13:04 GMT): alexvicegrab (Tue, 25 Sep 2018 14:13:21 GMT): ColeBoudreau (Tue, 25 Sep 2018 14:35:36 GMT): ColeBoudreau (Tue, 25 Sep 2018 17:10:11 GMT): ColeBoudreau (Tue, 25 Sep 2018 19:15:51 GMT): Gvedam (Tue, 25 Sep 2018 21:48:45 GMT): baohua (Wed, 26 Sep 2018 01:09:09 GMT): baohua (Wed, 26 Sep 2018 01:10:46 GMT): Gvedam (Wed, 26 Sep 2018 04:39:08 GMT): Gvedam (Wed, 26 Sep 2018 04:47:12 GMT): baohua (Wed, 26 Sep 2018 09:33:56 GMT): Gvedam (Wed, 26 Sep 2018 09:39:18 GMT): nxng (Wed, 26 Sep 2018 10:02:18 GMT): nxng (Wed, 26 Sep 2018 10:03:26 GMT): nishanthkp (Wed, 26 Sep 2018 14:59:32 GMT): ColeBoudreau (Wed, 26 Sep 2018 16:59:32 GMT): ColeBoudreau (Wed, 26 Sep 2018 16:59:32 GMT): ColeBoudreau (Wed, 26 Sep 2018 16:59:35 GMT): baohua (Thu, 27 Sep 2018 00:34:26 GMT): baohua (Thu, 27 Sep 2018 00:40:18 GMT): nfrunza (Thu, 27 Sep 2018 01:48:25 GMT): pankajcheema (Thu, 27 Sep 2018 15:15:58 GMT): ColeBoudreau (Thu, 27 Sep 2018 15:17:46 GMT): pankajcheema (Thu, 27 Sep 2018 15:37:06 GMT): ColeBoudreau (Thu, 27 Sep 2018 16:32:41 GMT): pankajcheema (Thu, 27 Sep 2018 16:50:31 GMT): pankajcheema (Thu, 27 Sep 2018 16:51:02 GMT): DeepakMule (Fri, 28 Sep 2018 05:08:38 GMT): DeepakMule (Fri, 28 Sep 2018 05:09:25 GMT): DeepakMule (Fri, 28 Sep 2018 05:09:25 GMT): nishanthkp (Fri, 28 Sep 2018 10:09:31 GMT): zale144 (Fri, 28 Sep 2018 17:46:00 GMT): zale144 (Fri, 28 Sep 2018 17:49:02 GMT): zale144 (Fri, 28 Sep 2018 17:59:10 GMT): zale144 (Fri, 28 Sep 2018 17:59:28 GMT): forrunner (Sat, 29 Sep 2018 04:35:04 GMT): louisliu2048 (Sun, 30 Sep 2018 03:16:18 GMT): bairathirahul (Sun, 30 Sep 2018 19:23:49 GMT): Kby7 (Mon, 01 Oct 2018 11:19:50 GMT): Kby7 (Mon, 01 Oct 2018 11:26:38 GMT): VirendraSolanke (Mon, 01 Oct 2018 16:20:24 GMT): Ferrymania (Tue, 02 Oct 2018 04:45:36 GMT): Bartb0 (Tue, 02 Oct 2018 10:58:05 GMT): AltifBrown (Tue, 02 Oct 2018 23:08:33 GMT): greg2git (Wed, 03 Oct 2018 14:56:59 GMT): ffiore81 (Thu, 04 Oct 2018 15:05:19 GMT): MohitJuneja (Thu, 04 Oct 2018 16:01:01 GMT): Perebor (Sat, 06 Oct 2018 03:07:57 GMT): nishanthkp (Mon, 08 Oct 2018 14:22:34 GMT): nishanthkp (Mon, 08 Oct 2018 14:22:40 GMT): rajanashutosh (Tue, 09 Oct 2018 04:28:28 GMT): baohua (Tue, 09 Oct 2018 05:38:20 GMT): rajanashutosh (Tue, 09 Oct 2018 07:26:47 GMT): rajanashutosh (Tue, 09 Oct 2018 07:26:51 GMT): rajanashutosh (Tue, 09 Oct 2018 07:27:14 GMT): rajanashutosh (Tue, 09 Oct 2018 07:27:43 GMT): rajanashutosh (Tue, 09 Oct 2018 08:29:22 GMT): rajanashutosh (Tue, 09 Oct 2018 08:29:44 GMT): Surabhi1777 (Tue, 09 Oct 2018 08:49:32 GMT): asaningmaxchain123 (Wed, 10 Oct 2018 08:20:32 GMT): venkat1 2 (Thu, 11 Oct 2018 21:17:34 GMT): venkat1 2 (Thu, 11 Oct 2018 22:04:03 GMT): Russ.corsha (Fri, 12 Oct 2018 18:35:37 GMT): venkat1 2 (Fri, 12 Oct 2018 21:40:55 GMT): rjones (Fri, 12 Oct 2018 22:52:10 GMT): saadinator (Sat, 13 Oct 2018 08:06:26 GMT): ritel126 (Mon, 15 Oct 2018 02:15:28 GMT): luckydogchina (Mon, 15 Oct 2018 06:00:09 GMT): OviiyaDominic (Mon, 15 Oct 2018 06:37:23 GMT): fnst_zhangxin (Thu, 18 Oct 2018 07:44:06 GMT): fnst_zhangxin (Thu, 18 Oct 2018 07:46:11 GMT): fnst_zhangxin (Thu, 18 Oct 2018 07:52:34 GMT): fnst_zhangxin (Thu, 18 Oct 2018 07:53:26 GMT): fnst_zhangxin (Thu, 18 Oct 2018 07:54:20 GMT): fnst_zhangxin (Thu, 18 Oct 2018 07:54:30 GMT): fnst_zhangxin (Thu, 18 Oct 2018 08:04:40 GMT): baohua (Thu, 18 Oct 2018 13:57:18 GMT): fnst_zhangxin (Fri, 19 Oct 2018 01:03:05 GMT): baohua (Fri, 19 Oct 2018 01:58:13 GMT): baohua (Fri, 19 Oct 2018 01:58:17 GMT): baohua (Fri, 19 Oct 2018 01:59:06 GMT): angelsuarez (Fri, 19 Oct 2018 08:21:12 GMT): leixianting (Fri, 19 Oct 2018 08:26:49 GMT): luckydogchina (Fri, 19 Oct 2018 12:16:32 GMT): luckydogchina (Fri, 19 Oct 2018 12:16:38 GMT): shuailaotou (Mon, 22 Oct 2018 07:45:07 GMT): shuailaotou (Mon, 22 Oct 2018 07:46:19 GMT): luckydogchina (Mon, 22 Oct 2018 09:37:26 GMT): nishanthkp (Mon, 22 Oct 2018 12:24:07 GMT): shuailaotou (Tue, 23 Oct 2018 02:56:30 GMT): GowriR (Tue, 23 Oct 2018 07:07:08 GMT): shuailaotou (Tue, 23 Oct 2018 07:11:56 GMT): MohammadObaid (Tue, 23 Oct 2018 08:29:17 GMT): ninn (Tue, 23 Oct 2018 14:30:15 GMT): aalcantara (Tue, 23 Oct 2018 15:32:59 GMT): aalcantara (Tue, 23 Oct 2018 15:33:06 GMT): dlarukov (Tue, 23 Oct 2018 16:02:46 GMT): dlarukov (Tue, 23 Oct 2018 16:21:15 GMT): MRM (Tue, 23 Oct 2018 18:47:36 GMT): ninn (Wed, 24 Oct 2018 11:06:32 GMT): ninn (Wed, 24 Oct 2018 11:06:32 GMT): ninn (Wed, 24 Oct 2018 11:06:32 GMT): dlarukov (Wed, 24 Oct 2018 11:20:53 GMT): Luke_Chen (Wed, 24 Oct 2018 11:59:26 GMT): cagdast (Thu, 25 Oct 2018 07:33:13 GMT): dcasado (Thu, 25 Oct 2018 14:20:12 GMT): javaguirre (Thu, 25 Oct 2018 20:28:53 GMT): Luke_Chen (Fri, 26 Oct 2018 03:58:18 GMT): Luke_Chen (Fri, 26 Oct 2018 03:58:18 GMT): Luke_Chen (Fri, 26 Oct 2018 04:02:39 GMT): baohua (Fri, 26 Oct 2018 04:06:53 GMT): baohua (Fri, 26 Oct 2018 04:07:29 GMT): baohua (Fri, 26 Oct 2018 04:07:59 GMT): Luke_Chen (Fri, 26 Oct 2018 04:08:32 GMT): Luke_Chen (Fri, 26 Oct 2018 04:08:57 GMT): baohua (Fri, 26 Oct 2018 04:10:05 GMT): baohua (Fri, 26 Oct 2018 04:10:34 GMT): Luke_Chen (Fri, 26 Oct 2018 04:11:54 GMT): Luke_Chen (Fri, 26 Oct 2018 05:31:26 GMT): Luke_Chen (Fri, 26 Oct 2018 05:31:26 GMT): Luke_Chen (Fri, 26 Oct 2018 05:32:00 GMT): Luke_Chen (Fri, 26 Oct 2018 08:54:05 GMT): Luke_Chen (Fri, 26 Oct 2018 14:24:53 GMT): Luke_Chen (Fri, 26 Oct 2018 14:24:55 GMT): baohua (Sat, 27 Oct 2018 03:01:08 GMT): baohua (Sat, 27 Oct 2018 03:01:24 GMT): Luke_Chen (Sat, 27 Oct 2018 11:44:16 GMT): baohua (Sat, 27 Oct 2018 14:19:38 GMT): luckydogchina (Mon, 29 Oct 2018 02:38:16 GMT): luckydogchina (Mon, 29 Oct 2018 02:38:16 GMT): luckydogchina (Mon, 29 Oct 2018 02:38:16 GMT): luckydogchina (Mon, 29 Oct 2018 02:38:16 GMT): baohua (Mon, 29 Oct 2018 03:30:05 GMT): luckydogchina (Mon, 29 Oct 2018 03:32:29 GMT): pravindia (Mon, 29 Oct 2018 06:13:15 GMT): luckydogchina (Mon, 29 Oct 2018 08:03:47 GMT): Luke_Chen (Mon, 29 Oct 2018 09:12:16 GMT): Luke_Chen (Mon, 29 Oct 2018 09:12:16 GMT): SumanPapanaboina (Mon, 29 Oct 2018 09:44:57 GMT): SumanPapanaboina (Mon, 29 Oct 2018 09:45:51 GMT): tongli (Mon, 29 Oct 2018 12:20:48 GMT): rjones (Mon, 29 Oct 2018 14:52:21 GMT): Luke_Chen (Tue, 30 Oct 2018 03:24:54 GMT): Luke_Chen (Tue, 30 Oct 2018 03:25:42 GMT): luckydogchina (Tue, 30 Oct 2018 04:49:41 GMT): luckydogchina (Tue, 30 Oct 2018 04:50:00 GMT): rkrish82 (Tue, 30 Oct 2018 07:40:04 GMT): Sasadara (Wed, 31 Oct 2018 04:17:46 GMT): baohua (Wed, 31 Oct 2018 08:58:15 GMT): Luke_Chen (Wed, 31 Oct 2018 14:55:20 GMT): swemso (Thu, 01 Nov 2018 09:22:45 GMT): baohua (Thu, 01 Nov 2018 14:13:47 GMT): Luke_Chen (Fri, 02 Nov 2018 05:57:16 GMT): baohua (Fri, 02 Nov 2018 13:53:07 GMT): baohua (Fri, 02 Nov 2018 13:53:14 GMT): baohua (Mon, 05 Nov 2018 01:37:18 GMT): luckydogchina (Mon, 05 Nov 2018 05:51:06 GMT): sean (Mon, 05 Nov 2018 19:24:24 GMT): baohua (Tue, 06 Nov 2018 01:47:38 GMT): luckydogchina (Tue, 06 Nov 2018 03:32:02 GMT): tuanpvi (Tue, 06 Nov 2018 08:03:13 GMT): tuanpvi (Tue, 06 Nov 2018 08:05:07 GMT): tuanpvi (Tue, 06 Nov 2018 08:05:32 GMT): tuanpvi (Tue, 06 Nov 2018 08:05:40 GMT): tuanpvi (Tue, 06 Nov 2018 08:06:44 GMT): baohua (Tue, 06 Nov 2018 08:26:35 GMT): baohua (Tue, 06 Nov 2018 08:27:01 GMT): tuanpvi (Tue, 06 Nov 2018 08:42:35 GMT): tuanpvi (Tue, 06 Nov 2018 08:43:51 GMT): tuanpvi (Tue, 06 Nov 2018 08:46:12 GMT): sreeharsha_katta (Tue, 06 Nov 2018 10:27:11 GMT): tongli (Tue, 06 Nov 2018 19:08:19 GMT): tongli (Tue, 06 Nov 2018 19:08:26 GMT): tongli (Tue, 06 Nov 2018 19:56:41 GMT): baohua (Wed, 07 Nov 2018 01:46:50 GMT): Skprog (Thu, 08 Nov 2018 08:09:58 GMT): fnst_zhangxin (Thu, 08 Nov 2018 08:54:37 GMT): lcj (Thu, 08 Nov 2018 13:28:51 GMT): lcj (Thu, 08 Nov 2018 13:31:48 GMT): dcasado (Thu, 08 Nov 2018 15:41:28 GMT): tongli (Thu, 08 Nov 2018 17:13:20 GMT): Russ.corsha (Thu, 08 Nov 2018 18:09:32 GMT): baohua (Fri, 09 Nov 2018 01:36:18 GMT): meilier (Fri, 09 Nov 2018 06:25:36 GMT): meilier (Fri, 09 Nov 2018 06:32:24 GMT): fnst_zhangxin (Fri, 09 Nov 2018 07:29:21 GMT): meilier (Fri, 09 Nov 2018 08:40:26 GMT): meilier (Fri, 09 Nov 2018 08:40:26 GMT): meilier (Fri, 09 Nov 2018 08:40:26 GMT): tongli (Fri, 09 Nov 2018 12:13:06 GMT): meilier (Fri, 09 Nov 2018 13:07:22 GMT): tongli (Fri, 09 Nov 2018 14:25:47 GMT): nishanthkp (Fri, 09 Nov 2018 16:24:42 GMT): nishanthkp (Fri, 09 Nov 2018 16:24:50 GMT): nishanthkp (Fri, 09 Nov 2018 16:24:54 GMT): nemo 8 (Fri, 09 Nov 2018 16:36:05 GMT): LegoEngineering (Mon, 12 Nov 2018 19:30:36 GMT): nishanthkp (Wed, 14 Nov 2018 11:29:12 GMT): manoj485 (Thu, 15 Nov 2018 05:39:30 GMT): manoj485 (Thu, 15 Nov 2018 05:40:04 GMT): Luke_Chen (Thu, 15 Nov 2018 06:10:40 GMT): manoj485 (Thu, 15 Nov 2018 06:18:31 GMT): manoj485 (Thu, 15 Nov 2018 06:19:11 GMT): Luke_Chen (Thu, 15 Nov 2018 06:23:41 GMT): Luke_Chen (Thu, 15 Nov 2018 06:24:00 GMT): manoj485 (Thu, 15 Nov 2018 06:24:32 GMT): manoj485 (Thu, 15 Nov 2018 06:27:35 GMT): Luke_Chen (Thu, 15 Nov 2018 06:28:24 GMT): manoj485 (Thu, 15 Nov 2018 06:29:26 GMT): Luke_Chen (Thu, 15 Nov 2018 06:30:54 GMT): Luke_Chen (Thu, 15 Nov 2018 06:31:34 GMT): manoj485 (Thu, 15 Nov 2018 06:32:23 GMT): manoj485 (Thu, 15 Nov 2018 06:32:38 GMT): manoj485 (Thu, 15 Nov 2018 06:33:02 GMT): Luke_Chen (Thu, 15 Nov 2018 06:35:17 GMT): manoj485 (Thu, 15 Nov 2018 06:35:31 GMT): manoj485 (Thu, 15 Nov 2018 06:36:20 GMT): Luke_Chen (Thu, 15 Nov 2018 06:39:11 GMT): manoj485 (Thu, 15 Nov 2018 07:12:43 GMT): manoj485 (Thu, 15 Nov 2018 07:12:51 GMT): manoj485 (Thu, 15 Nov 2018 10:54:25 GMT): kannancet (Fri, 16 Nov 2018 09:37:49 GMT): manoj485 (Sat, 17 Nov 2018 11:10:00 GMT): Gaurang (Sun, 18 Nov 2018 05:11:20 GMT): Gaurang (Sun, 18 Nov 2018 05:11:22 GMT): Gaurang (Sun, 18 Nov 2018 05:11:50 GMT): Gaurang (Sun, 18 Nov 2018 05:12:49 GMT): Gaurang (Sun, 18 Nov 2018 05:12:50 GMT): Gaurang (Sun, 18 Nov 2018 05:12:50 GMT): Gaurang (Sun, 18 Nov 2018 05:12:50 GMT): Gaurang (Sun, 18 Nov 2018 05:12:50 GMT): Gaurang (Sun, 18 Nov 2018 05:15:04 GMT): meilier (Sun, 18 Nov 2018 05:56:09 GMT): meilier (Sun, 18 Nov 2018 06:03:04 GMT): Gaurang (Sun, 18 Nov 2018 07:17:45 GMT): meilier (Sun, 18 Nov 2018 08:17:46 GMT): meilier (Sun, 18 Nov 2018 08:17:46 GMT): meilier (Sun, 18 Nov 2018 08:17:46 GMT): Gaurang (Sun, 18 Nov 2018 09:05:34 GMT): Gaurang (Sun, 18 Nov 2018 09:05:34 GMT): Gaurang (Sun, 18 Nov 2018 09:10:07 GMT): hightall (Tue, 20 Nov 2018 07:04:40 GMT): marryton007 (Wed, 21 Nov 2018 03:41:35 GMT): Jim1 1 (Wed, 21 Nov 2018 12:32:58 GMT): Jim1 1 (Wed, 21 Nov 2018 12:33:45 GMT): Jim1 1 (Wed, 21 Nov 2018 12:34:43 GMT): Jim1 1 (Wed, 21 Nov 2018 12:35:31 GMT): Jim1 1 (Wed, 21 Nov 2018 12:35:37 GMT): baohua (Wed, 21 Nov 2018 14:33:42 GMT): baohua (Wed, 21 Nov 2018 14:33:55 GMT): a.ochs (Wed, 21 Nov 2018 14:47:26 GMT): greivinlopez (Wed, 21 Nov 2018 16:07:35 GMT): mehulshahnj (Wed, 21 Nov 2018 18:49:48 GMT): mehulshahnj (Wed, 21 Nov 2018 18:50:28 GMT): mehulshahnj (Wed, 21 Nov 2018 18:50:28 GMT): mehulshahnj (Wed, 21 Nov 2018 18:52:45 GMT): mehulshahnj (Wed, 21 Nov 2018 18:52:45 GMT): mehulshahnj (Wed, 21 Nov 2018 18:58:06 GMT): mehulshahnj (Wed, 21 Nov 2018 19:09:34 GMT): rjones (Wed, 21 Nov 2018 19:58:44 GMT): mehulshahnj (Wed, 21 Nov 2018 20:34:52 GMT): mehulshahnj (Wed, 21 Nov 2018 21:43:34 GMT): Jim1 1 (Thu, 22 Nov 2018 03:26:52 GMT): Jim1 1 (Thu, 22 Nov 2018 03:27:20 GMT): Jim1 1 (Thu, 22 Nov 2018 03:28:27 GMT): You should be redirected automatically to target URL: http://:8083/auth/realms/cello-realm/protocol/openid-connect/auth?client_id=cello-operator-dashboard&redirect_uri=http%3A%2F%2F10.101.110.94%3A8080%2Foidc_callback&scope=openid+email&access_type=offline&response_type=code&state=eyJjc3JmX3Rva2VuIjogIjFEbzE4and6YjAwNERReUEzaGxaR09DMlhodWtTcWl1IiwgImRlc3RpbmF0aW9uIjogImV5SmhiR2NpT2lKSVV6VXhNaUo5LkltaDBkSEE2THk4eE1DNHhNREV1TVRFd0xqazBPamd3T0RBdklnLjgxdC1jcEVXOC13WXVMRTN0dkNJeFdCN1pUaGswWmIwaUdfd2hGTTE1Qkd1U1laZTlBVkZtZlk0WV95MF9faDVJQ0gtazNmRlk1M1dNa2Q4b2dPNTl3In0%3D&openid.realm=http%3A%2F%2F%3A8080%2Foidc_callback. If not click the link.
Jim1 1 (Thu, 22 Nov 2018 03:32:01 GMT): Jim1 1 (Thu, 22 Nov 2018 03:32:07 GMT): baohua (Thu, 22 Nov 2018 03:34:33 GMT): Jim1 1 (Thu, 22 Nov 2018 03:50:05 GMT): Gaurang (Thu, 22 Nov 2018 05:20:02 GMT): Jim1 1 (Thu, 22 Nov 2018 11:50:56 GMT): Jim1 1 (Thu, 22 Nov 2018 11:52:02 GMT): Jim1 1 (Thu, 22 Nov 2018 11:52:32 GMT): Jim1 1 (Thu, 22 Nov 2018 11:52:38 GMT): Jim1 1 (Thu, 22 Nov 2018 11:52:43 GMT): Gaurang (Thu, 22 Nov 2018 11:55:46 GMT): Jim1 1 (Thu, 22 Nov 2018 11:56:28 GMT): SadOraz (Thu, 22 Nov 2018 11:59:02 GMT): SadOraz (Thu, 22 Nov 2018 12:02:33 GMT): Jim1 1 (Thu, 22 Nov 2018 12:07:28 GMT): Jim1 1 (Thu, 22 Nov 2018 12:07:35 GMT): bd (Thu, 22 Nov 2018 12:27:38 GMT): SadOraz (Thu, 22 Nov 2018 12:33:56 GMT): bd (Thu, 22 Nov 2018 12:39:39 GMT): Jim1 1 (Thu, 22 Nov 2018 16:03:15 GMT): Jim1 1 (Thu, 22 Nov 2018 16:03:20 GMT): SadOraz (Fri, 23 Nov 2018 03:50:16 GMT): Jim1 1 (Fri, 23 Nov 2018 05:05:02 GMT): Jim1 1 (Fri, 23 Nov 2018 05:05:07 GMT): Jim1 1 (Fri, 23 Nov 2018 05:05:18 GMT): debajyoti-xyz (Fri, 23 Nov 2018 07:56:35 GMT): PaddyBN (Fri, 23 Nov 2018 10:56:56 GMT): yinzhiwu (Sat, 24 Nov 2018 02:56:39 GMT): yinzhiwu (Sat, 24 Nov 2018 03:09:55 GMT): jack77121 (Sun, 25 Nov 2018 17:07:33 GMT): Levilk (Sun, 25 Nov 2018 21:50:58 GMT): OviiyaDominic (Mon, 26 Nov 2018 05:05:34 GMT): PaddyBN (Mon, 26 Nov 2018 10:17:55 GMT): Levilk (Mon, 26 Nov 2018 20:22:07 GMT): Levilk (Mon, 26 Nov 2018 20:24:50 GMT): ajeethkannan (Tue, 27 Nov 2018 04:31:17 GMT): JoanArcas (Tue, 27 Nov 2018 11:08:26 GMT): joaquimpedrooliveira (Tue, 27 Nov 2018 13:34:26 GMT): baohua (Wed, 28 Nov 2018 01:46:49 GMT): Jim1 1 (Wed, 28 Nov 2018 07:24:34 GMT): Jim1 1 (Wed, 28 Nov 2018 07:25:14 GMT): BhaskarMondal (Wed, 28 Nov 2018 08:02:09 GMT): BhaskarMondal (Wed, 28 Nov 2018 08:04:32 GMT): baohua (Wed, 28 Nov 2018 08:08:36 GMT): baohua (Wed, 28 Nov 2018 08:08:54 GMT): BhaskarMondal (Wed, 28 Nov 2018 12:57:52 GMT): TopJohn (Thu, 29 Nov 2018 06:15:32 GMT): haggis (Thu, 29 Nov 2018 08:07:28 GMT): baohua (Thu, 29 Nov 2018 15:06:17 GMT): nitishbhardwaj19 (Fri, 30 Nov 2018 05:31:02 GMT): nitishbhardwaj19 (Fri, 30 Nov 2018 05:32:33 GMT): nitishbhardwaj19 (Fri, 30 Nov 2018 05:32:33 GMT): Jim1 1 (Fri, 30 Nov 2018 05:47:17 GMT): AlphaAsteroid (Fri, 30 Nov 2018 12:30:04 GMT): AlphaAsteroid (Fri, 30 Nov 2018 12:30:48 GMT): baohua (Fri, 30 Nov 2018 13:07:00 GMT): baohua (Fri, 30 Nov 2018 13:07:54 GMT): AlphaAsteroid (Fri, 30 Nov 2018 16:02:33 GMT): AlphaAsteroid (Fri, 30 Nov 2018 16:03:05 GMT): NursultanMakhanov (Sat, 01 Dec 2018 15:26:41 GMT): baohua (Sun, 02 Dec 2018 01:28:33 GMT): AlphaAsteroid (Sun, 02 Dec 2018 17:44:26 GMT): AlphaAsteroid (Sun, 02 Dec 2018 17:44:44 GMT): AlphaAsteroid (Sun, 02 Dec 2018 17:46:14 GMT): AlphaAsteroid (Sun, 02 Dec 2018 17:46:24 GMT): AlphaAsteroid (Sun, 02 Dec 2018 17:46:58 GMT): AlphaAsteroid (Sun, 02 Dec 2018 17:47:09 GMT): AlphaAsteroid (Sun, 02 Dec 2018 17:47:28 GMT): AlphaAsteroid (Sun, 02 Dec 2018 17:47:36 GMT): AlphaAsteroid (Mon, 03 Dec 2018 08:36:38 GMT): AlphaAsteroid (Mon, 03 Dec 2018 08:36:43 GMT): ra_w (Mon, 03 Dec 2018 12:36:35 GMT): ra_w (Mon, 03 Dec 2018 12:39:56 GMT): Cliff.Huang (Tue, 04 Dec 2018 09:35:28 GMT): tongli (Tue, 04 Dec 2018 16:21:56 GMT): tongli (Tue, 04 Dec 2018 16:22:02 GMT): baohua (Wed, 05 Dec 2018 01:12:55 GMT): calvin.heo (Wed, 05 Dec 2018 16:58:56 GMT): tongli (Wed, 05 Dec 2018 20:09:20 GMT): baohua (Thu, 06 Dec 2018 03:43:24 GMT): baohua (Fri, 07 Dec 2018 09:20:11 GMT): asaningmaxchain123 (Fri, 07 Dec 2018 13:17:06 GMT): ra_w (Sun, 09 Dec 2018 19:01:34 GMT): MeenakshiSingh (Mon, 10 Dec 2018 17:56:15 GMT): sivak2018 (Tue, 11 Dec 2018 04:12:45 GMT): sivak2018 (Tue, 11 Dec 2018 04:12:57 GMT): sivak2018 (Tue, 11 Dec 2018 04:12:57 GMT): sivak2018 (Tue, 11 Dec 2018 04:16:00 GMT): thipuvaasan (Tue, 11 Dec 2018 04:41:58 GMT): lewislau86 (Tue, 11 Dec 2018 07:54:25 GMT): lewislau86 (Tue, 11 Dec 2018 07:56:39 GMT): lewislau86 (Tue, 11 Dec 2018 07:57:40 GMT): thipuvaasan (Tue, 11 Dec 2018 11:05:41 GMT): yinzhiwu (Wed, 12 Dec 2018 02:11:57 GMT): baohua (Wed, 12 Dec 2018 04:04:58 GMT): asaningmaxchain123 (Wed, 12 Dec 2018 06:11:19 GMT): baohua (Wed, 12 Dec 2018 06:14:35 GMT): asaningmaxchain123 (Wed, 12 Dec 2018 06:16:12 GMT): asaningmaxchain123 (Wed, 12 Dec 2018 06:16:45 GMT): asaningmaxchain123 (Wed, 12 Dec 2018 06:16:45 GMT): baohua (Wed, 12 Dec 2018 10:43:02 GMT): yinzhiwu (Wed, 12 Dec 2018 10:43:16 GMT): baohua (Wed, 12 Dec 2018 10:43:17 GMT): baohua (Wed, 12 Dec 2018 10:43:44 GMT): yinzhiwu (Wed, 12 Dec 2018 10:43:45 GMT): baohua (Wed, 12 Dec 2018 10:43:51 GMT): baohua (Wed, 12 Dec 2018 10:43:57 GMT): baohua (Wed, 12 Dec 2018 10:44:19 GMT): yinzhiwu (Wed, 12 Dec 2018 10:45:48 GMT): yinzhiwu (Wed, 12 Dec 2018 10:48:18 GMT): yinzhiwu (Wed, 12 Dec 2018 10:57:59 GMT): sivak2018 (Wed, 12 Dec 2018 18:38:16 GMT): sivak2018 (Wed, 12 Dec 2018 18:38:16 GMT): tongli (Thu, 13 Dec 2018 01:02:42 GMT): yinzhiwu (Thu, 13 Dec 2018 01:21:13 GMT): aviralwal (Thu, 13 Dec 2018 09:20:56 GMT): ashwin (Thu, 13 Dec 2018 10:53:50 GMT): Gaurang (Thu, 13 Dec 2018 13:08:08 GMT): Gaurang (Thu, 13 Dec 2018 13:12:44 GMT): Abhinavgrg074 (Fri, 14 Dec 2018 06:14:15 GMT): Rantwijk (Fri, 14 Dec 2018 09:49:54 GMT): gotschelm (Fri, 14 Dec 2018 13:34:25 GMT): gotschelm (Fri, 14 Dec 2018 13:44:09 GMT): sairak (Sat, 15 Dec 2018 08:30:46 GMT): sivak2018 (Sat, 15 Dec 2018 15:06:50 GMT): baohua (Sat, 15 Dec 2018 22:58:10 GMT): nitishbhardwaj19 (Mon, 17 Dec 2018 03:33:42 GMT): nitishbhardwaj19 (Mon, 17 Dec 2018 04:06:32 GMT): nitishbhardwaj19 (Mon, 17 Dec 2018 04:06:32 GMT): wuqilv (Mon, 17 Dec 2018 09:42:35 GMT): jazzyarchitects (Mon, 17 Dec 2018 10:24:56 GMT): jazzyarchitects (Mon, 17 Dec 2018 10:29:18 GMT): jazzyarchitects (Mon, 17 Dec 2018 10:30:23 GMT): jazzyarchitects (Mon, 17 Dec 2018 10:31:35 GMT): mkhraisha (Mon, 17 Dec 2018 18:17:42 GMT): mkhraisha (Mon, 17 Dec 2018 18:42:40 GMT): mkhraisha (Mon, 17 Dec 2018 18:42:40 GMT): mkhraisha (Mon, 17 Dec 2018 18:42:40 GMT): greg2git (Mon, 17 Dec 2018 21:17:10 GMT): liuruifei (Wed, 19 Dec 2018 01:50:13 GMT): liuruifei (Wed, 19 Dec 2018 02:07:40 GMT): whoneedi (Thu, 20 Dec 2018 03:22:45 GMT): asaningmaxchain123 (Thu, 20 Dec 2018 07:02:05 GMT): asaningmaxchain123 (Thu, 20 Dec 2018 07:03:08 GMT): asaningmaxchain123 (Thu, 20 Dec 2018 07:03:11 GMT): asaningmaxchain123 (Thu, 20 Dec 2018 07:03:23 GMT): asaningmaxchain123 (Thu, 20 Dec 2018 07:04:04 GMT): asaningmaxchain123 (Thu, 20 Dec 2018 07:04:27 GMT): asaningmaxchain123 (Thu, 20 Dec 2018 07:13:54 GMT): asaningmaxchain123 (Thu, 20 Dec 2018 07:13:54 GMT): asaningmaxchain123 (Thu, 20 Dec 2018 07:13:54 GMT): asaningmaxchain123 (Thu, 20 Dec 2018 07:26:04 GMT): asaningmaxchain123 (Thu, 20 Dec 2018 07:26:14 GMT): liuruifei (Thu, 20 Dec 2018 10:35:13 GMT): liuruifei (Thu, 20 Dec 2018 10:35:35 GMT): Gaurang (Thu, 20 Dec 2018 13:03:58 GMT): asaningmaxchain123 (Thu, 20 Dec 2018 14:40:46 GMT): asaningmaxchain123 (Fri, 21 Dec 2018 01:42:21 GMT): baohua (Fri, 21 Dec 2018 01:48:24 GMT): baohua (Fri, 21 Dec 2018 01:48:28 GMT): asaningmaxchain123 (Fri, 21 Dec 2018 01:48:58 GMT): Gaurang (Fri, 21 Dec 2018 05:19:46 GMT): asaningmaxchain123 (Fri, 21 Dec 2018 07:38:56 GMT): asaningmaxchain123 (Fri, 21 Dec 2018 07:39:21 GMT): asaningmaxchain123 (Fri, 21 Dec 2018 11:55:43 GMT): asaningmaxchain123 (Fri, 21 Dec 2018 11:55:43 GMT): Gaurang (Sun, 23 Dec 2018 06:43:36 GMT): Gaurang (Sun, 23 Dec 2018 07:16:20 GMT): kodepareek (Sun, 23 Dec 2018 09:18:13 GMT): tongli (Mon, 24 Dec 2018 21:38:42 GMT): baohua (Tue, 25 Dec 2018 03:48:04 GMT): sharma66mahesh (Tue, 25 Dec 2018 04:43:12 GMT): sharma66mahesh (Tue, 25 Dec 2018 04:57:26 GMT): sharma66mahesh (Tue, 25 Dec 2018 04:59:48 GMT): Gaurang (Tue, 25 Dec 2018 06:47:12 GMT): Gaurang (Tue, 25 Dec 2018 06:47:12 GMT): narayanprusty (Tue, 25 Dec 2018 22:00:29 GMT): narayanprusty (Tue, 25 Dec 2018 22:01:01 GMT): narayanprusty (Tue, 25 Dec 2018 22:01:01 GMT): sam3313 (Wed, 26 Dec 2018 06:21:49 GMT): sam3313 (Wed, 26 Dec 2018 06:25:55 GMT): sam3313 (Wed, 26 Dec 2018 06:30:46 GMT): baohua (Wed, 26 Dec 2018 08:47:32 GMT): suneel18 (Wed, 26 Dec 2018 09:55:26 GMT): suneel18 (Wed, 26 Dec 2018 09:55:31 GMT): suneel18 (Wed, 26 Dec 2018 09:55:31 GMT): theprodigy (Wed, 26 Dec 2018 09:57:10 GMT): suneel18 (Wed, 26 Dec 2018 09:58:26 GMT): suneel18 (Wed, 26 Dec 2018 10:07:22 GMT): Gaurang (Wed, 26 Dec 2018 11:55:38 GMT): greivinlopez (Wed, 26 Dec 2018 12:57:04 GMT): ertalemdar (Thu, 27 Dec 2018 19:34:45 GMT): ertalemdar (Thu, 27 Dec 2018 19:34:48 GMT): ertalemdar (Thu, 27 Dec 2018 19:34:48 GMT): ertalemdar (Thu, 27 Dec 2018 19:34:48 GMT): jackial (Fri, 28 Dec 2018 09:17:52 GMT): jackial (Fri, 28 Dec 2018 09:21:44 GMT): yousaf (Sat, 29 Dec 2018 11:16:21 GMT): yousaf (Sat, 29 Dec 2018 11:18:16 GMT): hteng (Sat, 29 Dec 2018 14:32:06 GMT): rjones (Mon, 31 Dec 2018 05:37:52 GMT): rjones (Mon, 31 Dec 2018 05:43:36 GMT): NedSnark (Wed, 02 Jan 2019 19:44:02 GMT): tongli (Thu, 03 Jan 2019 01:51:48 GMT): tongli (Thu, 03 Jan 2019 01:51:55 GMT): baohua (Thu, 03 Jan 2019 02:54:35 GMT): tongli (Thu, 03 Jan 2019 17:27:28 GMT): tongli (Thu, 03 Jan 2019 17:27:39 GMT): tongli (Thu, 03 Jan 2019 17:27:43 GMT): tongli (Thu, 03 Jan 2019 17:28:16 GMT): tongli (Thu, 03 Jan 2019 17:28:41 GMT): PatrickOrtell (Fri, 04 Jan 2019 00:25:36 GMT): agabiola (Fri, 04 Jan 2019 00:34:52 GMT): baohua (Fri, 04 Jan 2019 03:53:22 GMT): chip (Fri, 04 Jan 2019 04:16:58 GMT): zjltiger (Fri, 04 Jan 2019 06:52:37 GMT): Luke_Chen (Fri, 04 Jan 2019 07:32:58 GMT): zjltiger (Fri, 04 Jan 2019 08:01:54 GMT): baohua (Sun, 06 Jan 2019 03:35:00 GMT): baohua (Sun, 06 Jan 2019 03:36:15 GMT): cm_liu (Tue, 08 Jan 2019 03:04:54 GMT): zjltiger (Tue, 08 Jan 2019 03:11:57 GMT): sstone1 (Tue, 08 Jan 2019 16:01:53 GMT): gnostikos (Wed, 09 Jan 2019 12:00:01 GMT): gnostikos (Wed, 09 Jan 2019 12:01:38 GMT): baohua (Thu, 10 Jan 2019 01:24:33 GMT): eujungkim (Thu, 10 Jan 2019 02:12:11 GMT): eujungkim (Thu, 10 Jan 2019 02:28:28 GMT): eujungkim (Thu, 10 Jan 2019 11:10:35 GMT): DmitriPlakhov (Thu, 10 Jan 2019 12:06:22 GMT): rjones (Fri, 11 Jan 2019 19:15:03 GMT): tongli (Fri, 11 Jan 2019 20:16:28 GMT): tongli (Fri, 11 Jan 2019 20:16:29 GMT): rjones (Fri, 11 Jan 2019 21:54:08 GMT): ArnabChatterjee (Mon, 14 Jan 2019 09:19:35 GMT): clydedacruz (Tue, 15 Jan 2019 04:44:09 GMT): clydedacruz (Tue, 15 Jan 2019 04:48:43 GMT): clydedacruz (Tue, 15 Jan 2019 04:48:43 GMT): clydedacruz (Tue, 15 Jan 2019 04:48:43 GMT): clydedacruz (Tue, 15 Jan 2019 04:51:43 GMT): clydedacruz (Tue, 15 Jan 2019 05:06:20 GMT): gzjandpy (Tue, 15 Jan 2019 08:44:23 GMT): baohua (Wed, 16 Jan 2019 04:17:34 GMT): baohua (Wed, 16 Jan 2019 04:17:53 GMT): gzjandpy (Wed, 16 Jan 2019 06:49:05 GMT): clydedacruz (Wed, 16 Jan 2019 06:57:47 GMT): clydedacruz (Wed, 16 Jan 2019 07:15:32 GMT): clydedacruz (Wed, 16 Jan 2019 07:15:32 GMT): clydedacruz (Wed, 16 Jan 2019 07:15:32 GMT): clydedacruz (Wed, 16 Jan 2019 07:15:32 GMT): clydedacruz (Wed, 16 Jan 2019 07:15:32 GMT): clydedacruz (Wed, 16 Jan 2019 07:15:32 GMT): lewislau86 (Wed, 16 Jan 2019 08:36:00 GMT): clydedacruz (Wed, 16 Jan 2019 09:15:16 GMT): clydedacruz (Wed, 16 Jan 2019 11:04:09 GMT): baohua (Thu, 17 Jan 2019 04:01:09 GMT): clydedacruz (Thu, 17 Jan 2019 04:44:22 GMT): clydedacruz (Thu, 17 Jan 2019 04:44:22 GMT): clydedacruz (Thu, 17 Jan 2019 04:44:22 GMT): baohua (Thu, 17 Jan 2019 07:33:56 GMT): eujungkim (Thu, 17 Jan 2019 09:22:10 GMT): clydedacruz (Thu, 17 Jan 2019 09:31:05 GMT): clydedacruz (Thu, 17 Jan 2019 09:31:05 GMT): baohua (Thu, 17 Jan 2019 14:59:13 GMT): eujungkim (Fri, 18 Jan 2019 01:54:15 GMT): KokoKoko (Fri, 18 Jan 2019 03:21:47 GMT): clydedacruz (Fri, 18 Jan 2019 04:48:58 GMT): clydedacruz (Fri, 18 Jan 2019 04:48:58 GMT): clydedacruz (Fri, 18 Jan 2019 04:48:58 GMT): clydedacruz (Fri, 18 Jan 2019 07:25:32 GMT): Igor360 (Mon, 21 Jan 2019 12:04:09 GMT): tongli (Mon, 21 Jan 2019 16:45:49 GMT): tongli (Mon, 21 Jan 2019 16:45:54 GMT): tongli (Mon, 21 Jan 2019 16:46:03 GMT): baohua (Tue, 22 Jan 2019 01:48:32 GMT): Shyam_Pratap_Singh (Tue, 22 Jan 2019 07:16:07 GMT): tongli (Tue, 22 Jan 2019 16:34:48 GMT): tongli (Tue, 22 Jan 2019 16:35:35 GMT): baohua (Wed, 23 Jan 2019 01:01:41 GMT): tongli (Wed, 23 Jan 2019 03:48:45 GMT): tongli (Wed, 23 Jan 2019 03:49:24 GMT): tongli (Wed, 23 Jan 2019 03:50:03 GMT): tongli (Wed, 23 Jan 2019 03:50:29 GMT): tongli (Wed, 23 Jan 2019 03:50:40 GMT): baohua (Wed, 23 Jan 2019 04:00:22 GMT): clydedacruz (Wed, 23 Jan 2019 09:04:25 GMT): clydedacruz (Wed, 23 Jan 2019 09:04:28 GMT): clydedacruz (Wed, 23 Jan 2019 12:22:21 GMT): tongli (Wed, 23 Jan 2019 22:32:19 GMT): DeepakMule (Fri, 25 Jan 2019 06:38:18 GMT): eric_mesquita (Fri, 25 Jan 2019 11:53:26 GMT): eric_mesquita (Fri, 25 Jan 2019 11:56:14 GMT): eric_mesquita (Fri, 25 Jan 2019 11:56:14 GMT): william123 (Fri, 25 Jan 2019 15:13:56 GMT): peter.danko (Mon, 28 Jan 2019 11:41:56 GMT): mattmaru (Tue, 29 Jan 2019 11:04:09 GMT): tongli (Wed, 30 Jan 2019 15:00:40 GMT): baohua (Wed, 30 Jan 2019 21:55:33 GMT): eric_mesquita (Thu, 31 Jan 2019 05:03:38 GMT): netanyahuyasser (Thu, 31 Jan 2019 08:03:01 GMT): netanyahuyasser (Thu, 31 Jan 2019 08:12:52 GMT): netanyahuyasser (Thu, 31 Jan 2019 08:12:52 GMT): netanyahuyasser (Thu, 31 Jan 2019 08:12:52 GMT): Bentipe (Thu, 31 Jan 2019 15:12:20 GMT): Bentipe (Thu, 31 Jan 2019 15:18:50 GMT): rjones (Thu, 31 Jan 2019 19:00:43 GMT): tongli (Thu, 31 Jan 2019 20:21:59 GMT): baohua (Fri, 01 Feb 2019 14:19:11 GMT): zgc (Sat, 02 Feb 2019 03:13:07 GMT): bcook 1 (Tue, 05 Feb 2019 18:26:25 GMT): bcook 1 (Tue, 05 Feb 2019 20:00:25 GMT): bcook 1 (Tue, 05 Feb 2019 20:00:52 GMT): bcook 1 (Tue, 05 Feb 2019 20:01:32 GMT): nitishbhardwaj19 (Mon, 11 Feb 2019 07:15:36 GMT): tongli (Mon, 11 Feb 2019 14:23:05 GMT): tongli (Mon, 11 Feb 2019 14:23:21 GMT): tongli (Mon, 11 Feb 2019 14:23:26 GMT): jarvis26 (Mon, 11 Feb 2019 17:59:01 GMT): jarvis26 (Mon, 11 Feb 2019 18:10:57 GMT): jarvis26 (Mon, 11 Feb 2019 18:10:57 GMT): jarvis26 (Mon, 11 Feb 2019 18:10:57 GMT): jarvis26 (Mon, 11 Feb 2019 18:14:08 GMT): baohua (Wed, 13 Feb 2019 01:52:27 GMT): baohua (Wed, 13 Feb 2019 01:52:56 GMT): habpygo (Wed, 13 Feb 2019 06:37:17 GMT): chengloy (Thu, 14 Feb 2019 06:44:19 GMT): baohua (Thu, 14 Feb 2019 07:05:17 GMT): baohua (Thu, 14 Feb 2019 07:06:25 GMT): baohua (Thu, 14 Feb 2019 08:33:26 GMT): chengloy (Thu, 14 Feb 2019 12:09:05 GMT): chengloy (Thu, 14 Feb 2019 12:09:20 GMT): chengloy (Thu, 14 Feb 2019 12:09:33 GMT): chengloy (Thu, 14 Feb 2019 12:09:42 GMT): chengloy (Thu, 14 Feb 2019 12:10:05 GMT): chengloy (Fri, 15 Feb 2019 06:09:48 GMT): chengloy (Fri, 15 Feb 2019 06:15:17 GMT): chengloy (Fri, 15 Feb 2019 06:16:02 GMT): chengloy (Fri, 15 Feb 2019 06:16:46 GMT): jarvis26 (Fri, 15 Feb 2019 07:26:39 GMT): jarvis26 (Fri, 15 Feb 2019 07:26:46 GMT): jarvis26 (Fri, 15 Feb 2019 07:27:20 GMT): jarvis26 (Fri, 15 Feb 2019 07:27:25 GMT): jarvis26 (Fri, 15 Feb 2019 07:28:20 GMT): lamnh2410 (Fri, 15 Feb 2019 07:31:06 GMT): lamnh2410 (Fri, 15 Feb 2019 07:34:29 GMT): chengloy (Fri, 15 Feb 2019 07:55:48 GMT): chengloy (Fri, 15 Feb 2019 07:58:23 GMT): chengloy (Fri, 15 Feb 2019 07:58:36 GMT): chengloy (Fri, 15 Feb 2019 08:27:40 GMT): chengloy (Fri, 15 Feb 2019 08:27:53 GMT): chengloy (Fri, 15 Feb 2019 08:28:11 GMT): chengloy (Fri, 15 Feb 2019 08:28:25 GMT): baohua (Sat, 16 Feb 2019 06:40:54 GMT): baohua (Sat, 16 Feb 2019 23:41:18 GMT): baohua (Sun, 17 Feb 2019 05:14:04 GMT): baohua (Sun, 17 Feb 2019 10:57:46 GMT): chengloy (Mon, 18 Feb 2019 08:50:38 GMT): chengloy (Mon, 18 Feb 2019 08:50:50 GMT): chengloy (Mon, 18 Feb 2019 08:50:52 GMT): chengloy (Mon, 18 Feb 2019 08:51:21 GMT): chengloy (Mon, 18 Feb 2019 08:57:42 GMT): jgiron (Mon, 18 Feb 2019 09:10:59 GMT): chengloy (Mon, 18 Feb 2019 09:59:53 GMT): chengloy (Mon, 18 Feb 2019 10:00:14 GMT): chengloy (Tue, 19 Feb 2019 02:17:51 GMT): jarvis26 (Tue, 19 Feb 2019 10:27:52 GMT): chengloy (Tue, 19 Feb 2019 10:46:35 GMT): jarvis26 (Tue, 19 Feb 2019 10:56:21 GMT): baohua (Tue, 19 Feb 2019 11:39:25 GMT): chengloy (Tue, 19 Feb 2019 11:55:12 GMT): baohua (Wed, 20 Feb 2019 06:33:24 GMT): tongli (Wed, 20 Feb 2019 14:06:54 GMT): tongli (Wed, 20 Feb 2019 14:07:12 GMT): baohua (Wed, 20 Feb 2019 15:17:56 GMT): tongli (Wed, 20 Feb 2019 15:18:34 GMT): tongli (Wed, 20 Feb 2019 15:18:54 GMT): baohua (Wed, 20 Feb 2019 15:19:18 GMT): baohua (Wed, 20 Feb 2019 15:19:26 GMT): tongli (Wed, 20 Feb 2019 15:20:31 GMT): tongli (Wed, 20 Feb 2019 15:20:40 GMT): tongli (Wed, 20 Feb 2019 15:21:01 GMT): tongli (Wed, 20 Feb 2019 15:21:13 GMT): tongli (Wed, 20 Feb 2019 15:51:19 GMT): baohua (Thu, 21 Feb 2019 01:36:49 GMT): Jasonyou (Thu, 21 Feb 2019 19:48:44 GMT): baohua (Fri, 22 Feb 2019 11:36:27 GMT): arthurmsouza (Mon, 25 Feb 2019 00:36:37 GMT): Vikaskr08 (Tue, 26 Feb 2019 15:04:05 GMT): Vikaskr08 (Tue, 26 Feb 2019 15:07:59 GMT): pikvik (Tue, 26 Feb 2019 16:52:08 GMT): Keegan-Lee (Tue, 26 Feb 2019 20:00:46 GMT): chengloy (Wed, 27 Feb 2019 03:48:18 GMT): chengloy (Wed, 27 Feb 2019 04:07:19 GMT): baohua (Wed, 27 Feb 2019 04:45:16 GMT): bjwswang (Thu, 28 Feb 2019 07:06:20 GMT): bjwswang (Thu, 28 Feb 2019 07:08:31 GMT): AlbertCL (Thu, 28 Feb 2019 15:48:19 GMT): AlbertCL (Fri, 01 Mar 2019 11:48:48 GMT): baohua (Sun, 03 Mar 2019 09:32:15 GMT): jarvis26 (Tue, 05 Mar 2019 10:17:24 GMT): jarvis26 (Tue, 05 Mar 2019 10:17:24 GMT): jarvis26 (Tue, 05 Mar 2019 10:17:29 GMT): jarvis26 (Tue, 05 Mar 2019 10:17:29 GMT): jarvis26 (Tue, 05 Mar 2019 18:44:19 GMT): jarvis26 (Tue, 05 Mar 2019 18:44:19 GMT): jarvis26 (Tue, 05 Mar 2019 18:44:19 GMT): jarvis26 (Tue, 05 Mar 2019 18:44:56 GMT): tongli (Wed, 06 Mar 2019 02:58:00 GMT): Jaline (Wed, 06 Mar 2019 08:57:10 GMT): baohua (Wed, 06 Mar 2019 15:42:45 GMT): davidpetit (Thu, 07 Mar 2019 05:15:34 GMT): mattsauce2 (Thu, 07 Mar 2019 05:16:58 GMT): baohua (Thu, 07 Mar 2019 05:37:54 GMT): baohua (Thu, 07 Mar 2019 05:43:20 GMT): baohua (Thu, 07 Mar 2019 05:45:52 GMT): y-arai (Thu, 07 Mar 2019 07:03:57 GMT): cjml1982 (Thu, 07 Mar 2019 07:32:08 GMT): gzjandpy (Thu, 07 Mar 2019 07:59:12 GMT): baohua (Thu, 07 Mar 2019 08:21:16 GMT): baohua (Thu, 07 Mar 2019 08:21:30 GMT): jarvis26 (Thu, 07 Mar 2019 14:04:26 GMT): jarvis26 (Thu, 07 Mar 2019 14:04:26 GMT): Javi (Thu, 07 Mar 2019 17:03:26 GMT): baohua (Fri, 08 Mar 2019 01:01:42 GMT): baohua (Fri, 08 Mar 2019 05:49:05 GMT): Javi (Fri, 08 Mar 2019 12:34:03 GMT): pikvik (Fri, 08 Mar 2019 12:53:32 GMT): ArindamBhattacharjee (Sun, 10 Mar 2019 21:20:43 GMT): ArindamBhattacharjee (Sun, 10 Mar 2019 21:23:00 GMT): ArindamBhattacharjee (Sun, 10 Mar 2019 21:23:11 GMT): baohua (Mon, 11 Mar 2019 01:09:41 GMT): baohua (Mon, 11 Mar 2019 01:11:02 GMT): ArindamBhattacharjee (Mon, 11 Mar 2019 04:55:45 GMT): davidpetit (Mon, 11 Mar 2019 07:06:32 GMT): pikvik (Mon, 11 Mar 2019 11:21:21 GMT): ArindamBhattacharjee (Mon, 11 Mar 2019 11:47:33 GMT): pikvik (Mon, 11 Mar 2019 11:59:33 GMT): ArindamBhattacharjee (Mon, 11 Mar 2019 14:08:18 GMT): vikas.kundz (Tue, 12 Mar 2019 03:02:10 GMT): baohua (Tue, 12 Mar 2019 08:21:11 GMT): baohua (Tue, 12 Mar 2019 08:21:32 GMT): baohua (Tue, 12 Mar 2019 08:21:57 GMT): rranjan3 (Tue, 12 Mar 2019 08:35:57 GMT): rranjan3 (Tue, 12 Mar 2019 08:37:20 GMT): pikvik (Tue, 12 Mar 2019 08:56:34 GMT): pikvik (Tue, 12 Mar 2019 08:56:34 GMT): baohua (Wed, 13 Mar 2019 01:19:35 GMT): baohua (Wed, 13 Mar 2019 01:20:07 GMT): rranjan3 (Wed, 13 Mar 2019 04:03:14 GMT): rranjan3 (Wed, 13 Mar 2019 04:03:14 GMT): SatheeshNehru (Wed, 13 Mar 2019 06:16:54 GMT): rranjan3 (Wed, 13 Mar 2019 06:53:05 GMT): gzjandpy (Wed, 13 Mar 2019 08:35:34 GMT): ArindamBhattacharjee (Wed, 13 Mar 2019 13:14:58 GMT): ArindamBhattacharjee (Wed, 13 Mar 2019 13:28:38 GMT): NasserRahal (Wed, 13 Mar 2019 16:00:23 GMT): ArindamBhattacharjee (Wed, 13 Mar 2019 16:19:03 GMT): ArindamBhattacharjee (Wed, 13 Mar 2019 16:23:45 GMT): Gaurang (Thu, 14 Mar 2019 06:50:25 GMT): Gaurang (Thu, 14 Mar 2019 06:50:25 GMT): Caralicious (Thu, 14 Mar 2019 08:47:57 GMT): Caralicious (Thu, 14 Mar 2019 08:50:34 GMT): baohua (Thu, 14 Mar 2019 13:45:10 GMT): baohua (Thu, 14 Mar 2019 13:45:54 GMT): tongli (Thu, 14 Mar 2019 19:04:39 GMT): tongli (Thu, 14 Mar 2019 19:05:00 GMT): baohua (Fri, 15 Mar 2019 04:00:36 GMT): Gaurang (Fri, 15 Mar 2019 04:55:53 GMT): RealDeanZhao (Fri, 15 Mar 2019 07:43:01 GMT): akoita (Fri, 15 Mar 2019 10:14:36 GMT): pikvik (Fri, 15 Mar 2019 10:55:48 GMT): pikvik (Fri, 15 Mar 2019 10:58:21 GMT): pikvik (Fri, 15 Mar 2019 10:58:35 GMT): baohua (Sat, 16 Mar 2019 07:41:48 GMT): pikvik (Mon, 18 Mar 2019 09:56:43 GMT): pikvik (Mon, 18 Mar 2019 10:05:25 GMT): pikvik (Mon, 18 Mar 2019 10:06:51 GMT): pikvik (Mon, 18 Mar 2019 10:09:29 GMT): Gaurang (Mon, 18 Mar 2019 10:42:50 GMT): pikvik (Mon, 18 Mar 2019 10:53:47 GMT): pikvik (Mon, 18 Mar 2019 11:42:25 GMT): epappas (Mon, 18 Mar 2019 20:50:37 GMT): Gaurang (Wed, 20 Mar 2019 08:11:35 GMT): Gaurang (Wed, 20 Mar 2019 08:12:29 GMT): pikvik (Wed, 20 Mar 2019 11:19:20 GMT): Gaurang (Wed, 20 Mar 2019 11:41:01 GMT): pikvik (Wed, 20 Mar 2019 11:41:48 GMT): Gaurang (Fri, 22 Mar 2019 05:23:50 GMT): pikvik (Fri, 22 Mar 2019 07:05:38 GMT): pikvik (Fri, 22 Mar 2019 07:06:44 GMT): Gaurang (Fri, 22 Mar 2019 07:32:22 GMT): Xiaozhang (Fri, 22 Mar 2019 08:01:05 GMT): pikvik (Fri, 22 Mar 2019 08:01:33 GMT): Xiaozhang (Fri, 22 Mar 2019 08:39:54 GMT): Xiaozhang (Fri, 22 Mar 2019 08:45:58 GMT): Xiaozhang (Fri, 22 Mar 2019 08:50:03 GMT): Gaurang (Fri, 22 Mar 2019 10:12:13 GMT): Gaurang (Fri, 22 Mar 2019 10:14:47 GMT): Gaurang (Fri, 22 Mar 2019 11:26:54 GMT): Gaurang (Fri, 22 Mar 2019 12:09:04 GMT): pikvik (Fri, 22 Mar 2019 12:34:57 GMT): KalyaniPullela (Fri, 22 Mar 2019 16:29:05 GMT): Gaurang (Sat, 23 Mar 2019 17:16:13 GMT): Xiaozhang (Sun, 24 Mar 2019 15:57:48 GMT): Vanitha (Mon, 25 Mar 2019 12:21:27 GMT): JohnR (Mon, 25 Mar 2019 12:38:19 GMT): Vanitha (Mon, 25 Mar 2019 12:45:46 GMT): Vanitha (Tue, 26 Mar 2019 10:58:07 GMT): Vanitha (Tue, 26 Mar 2019 10:58:31 GMT): Vanitha (Tue, 26 Mar 2019 11:26:34 GMT): Vanitha (Tue, 26 Mar 2019 11:27:11 GMT): JohnR (Tue, 26 Mar 2019 12:20:17 GMT): Vanitha (Tue, 26 Mar 2019 12:20:37 GMT): JohnR (Tue, 26 Mar 2019 12:21:17 GMT): Vanitha (Tue, 26 Mar 2019 12:24:34 GMT): Vanitha (Tue, 26 Mar 2019 12:29:34 GMT): pikvik (Wed, 27 Mar 2019 05:21:16 GMT): Vanitha (Wed, 27 Mar 2019 08:48:32 GMT): pikvik (Wed, 27 Mar 2019 08:50:00 GMT): Vanitha (Wed, 27 Mar 2019 08:54:32 GMT): pikvik (Wed, 27 Mar 2019 09:49:48 GMT): pikvik (Wed, 27 Mar 2019 09:49:57 GMT): Vanitha (Wed, 27 Mar 2019 10:10:28 GMT): ShrmaAmrit (Wed, 27 Mar 2019 10:18:38 GMT): ShrmaAmrit (Wed, 27 Mar 2019 10:30:37 GMT): KalyaniPullela (Wed, 27 Mar 2019 10:35:24 GMT): KalyaniPullela (Wed, 27 Mar 2019 17:10:21 GMT): Xiaozhang (Thu, 28 Mar 2019 07:39:49 GMT): Xiaozhang (Fri, 29 Mar 2019 03:53:02 GMT): alokkv (Fri, 29 Mar 2019 05:55:26 GMT): Gaurang (Fri, 29 Mar 2019 07:02:56 GMT): Vanitha (Fri, 29 Mar 2019 09:04:01 GMT): Vanitha (Fri, 29 Mar 2019 09:46:15 GMT): paul.sitoh (Fri, 29 Mar 2019 14:27:30 GMT): bhanu.ratnakaram (Fri, 29 Mar 2019 15:51:54 GMT): bhanukandregula (Fri, 29 Mar 2019 19:03:17 GMT): Xiaozhang (Mon, 01 Apr 2019 10:54:48 GMT): Xiaozhang (Tue, 02 Apr 2019 02:25:12 GMT): jarvis26 (Fri, 05 Apr 2019 07:16:43 GMT): baohua (Fri, 05 Apr 2019 13:31:19 GMT): baohua (Fri, 05 Apr 2019 13:31:25 GMT): Xiaozhang (Tue, 09 Apr 2019 06:35:35 GMT): baohua (Wed, 10 Apr 2019 00:18:36 GMT): Xiaozhang (Wed, 10 Apr 2019 01:16:40 GMT): Xiaozhang (Wed, 10 Apr 2019 01:17:10 GMT): Xiaozhang (Wed, 10 Apr 2019 07:21:28 GMT): Xiaozhang (Wed, 10 Apr 2019 17:24:41 GMT): prabu3192 (Thu, 11 Apr 2019 07:08:52 GMT): pipharrison (Fri, 12 Apr 2019 08:05:07 GMT): pipharrison (Fri, 12 Apr 2019 08:05:31 GMT): jarvis26 (Fri, 12 Apr 2019 08:26:51 GMT): jarvis26 (Fri, 12 Apr 2019 08:26:51 GMT): nitishbhardwaj19 (Fri, 12 Apr 2019 14:15:21 GMT): nitishbhardwaj19 (Fri, 12 Apr 2019 14:15:21 GMT): nitishbhardwaj19 (Fri, 12 Apr 2019 14:15:21 GMT): nitishbhardwaj19 (Fri, 12 Apr 2019 14:15:21 GMT): nitishbhardwaj19 (Fri, 12 Apr 2019 14:15:21 GMT): baohua (Sat, 13 Apr 2019 09:15:15 GMT): HiraSiddiqui (Mon, 15 Apr 2019 13:03:25 GMT): rameshthoomu (Wed, 24 Apr 2019 14:38:12 GMT): rameshthoomu (Wed, 24 Apr 2019 14:38:30 GMT): xoco (Thu, 25 Apr 2019 07:23:50 GMT): lzh 3 (Sat, 27 Apr 2019 06:07:41 GMT): Gnakale (Sun, 28 Apr 2019 11:17:00 GMT): Gnakale (Sun, 28 Apr 2019 11:18:39 GMT): Gnakale (Sun, 28 Apr 2019 11:18:39 GMT): awattez (Sun, 28 Apr 2019 20:24:55 GMT): baohua (Mon, 29 Apr 2019 00:58:47 GMT): zjltiger (Mon, 29 Apr 2019 02:04:19 GMT): zjltiger (Mon, 29 Apr 2019 02:42:18 GMT): Gnakale (Mon, 29 Apr 2019 06:47:31 GMT): MujtabaIdrees (Tue, 30 Apr 2019 14:04:47 GMT): MujtabaIdrees (Tue, 30 Apr 2019 14:13:26 GMT): MujtabaIdrees (Tue, 30 Apr 2019 14:13:26 GMT): MujtabaIdrees (Tue, 30 Apr 2019 14:13:26 GMT): MujtabaIdrees (Tue, 30 Apr 2019 14:14:34 GMT): MujtabaIdrees (Tue, 30 Apr 2019 14:14:34 GMT): MujtabaIdrees (Tue, 30 Apr 2019 14:14:34 GMT): MujtabaIdrees (Tue, 30 Apr 2019 14:18:25 GMT): MujtabaIdrees (Tue, 30 Apr 2019 14:18:25 GMT): MujtabaIdrees (Tue, 30 Apr 2019 14:34:05 GMT): MujtabaIdrees (Tue, 30 Apr 2019 14:34:20 GMT): rcchan (Fri, 03 May 2019 14:24:42 GMT): Ulysses (Sun, 05 May 2019 07:18:24 GMT): Ulysses (Sun, 05 May 2019 07:30:42 GMT): baohua (Mon, 06 May 2019 08:38:51 GMT): baohua (Mon, 06 May 2019 08:39:19 GMT): Ulysses (Mon, 06 May 2019 08:48:10 GMT): baclama (Tue, 07 May 2019 09:06:09 GMT): baclama (Tue, 07 May 2019 09:18:18 GMT): HLFPOC (Wed, 08 May 2019 17:37:00 GMT): HLFPOC (Wed, 08 May 2019 17:41:10 GMT): HLFPOC (Wed, 08 May 2019 17:41:10 GMT): tongli (Thu, 09 May 2019 11:22:45 GMT): tongli (Thu, 09 May 2019 11:22:56 GMT): tongli (Thu, 09 May 2019 11:23:33 GMT): tongli (Thu, 09 May 2019 11:23:53 GMT): tongli (Thu, 09 May 2019 11:24:16 GMT): HLFPOC (Thu, 09 May 2019 12:07:33 GMT): MujtabaIdrees (Thu, 09 May 2019 13:18:36 GMT): AlongGod (Fri, 10 May 2019 08:48:15 GMT): AlongGod (Fri, 10 May 2019 08:49:33 GMT): AlongGod (Mon, 13 May 2019 08:11:40 GMT): baclama (Wed, 15 May 2019 07:47:58 GMT): baohua (Thu, 16 May 2019 06:32:42 GMT): MujtabaIdrees (Fri, 17 May 2019 12:29:20 GMT): YutaoSong (Tue, 21 May 2019 00:41:15 GMT): YutaoSong (Tue, 21 May 2019 00:41:19 GMT): dsanchezseco (Tue, 21 May 2019 10:02:01 GMT): RicardoMurillo (Wed, 22 May 2019 17:59:18 GMT): RicardoMurillo (Wed, 22 May 2019 17:59:19 GMT): baohua (Thu, 23 May 2019 01:46:43 GMT): tongli (Thu, 23 May 2019 13:31:14 GMT): tongli (Thu, 23 May 2019 13:31:17 GMT): barry_liang (Mon, 27 May 2019 08:04:10 GMT): ooojerryooo (Tue, 28 May 2019 05:54:57 GMT): Man-Jain (Wed, 05 Jun 2019 07:08:32 GMT): jarvis26 (Tue, 11 Jun 2019 06:42:37 GMT): jarvis26 (Tue, 11 Jun 2019 06:42:37 GMT): jarvis26 (Tue, 11 Jun 2019 06:43:17 GMT): jarvis26 (Tue, 11 Jun 2019 06:43:17 GMT): jarvis26 (Tue, 11 Jun 2019 06:47:29 GMT): Bentipe (Tue, 11 Jun 2019 14:25:43 GMT): Bentipe (Tue, 11 Jun 2019 14:28:04 GMT): MHBauer (Wed, 12 Jun 2019 19:50:11 GMT): tatsu-sato (Fri, 14 Jun 2019 20:18:26 GMT): jarvis26 (Sun, 16 Jun 2019 11:57:59 GMT): BrajeshKumar (Wed, 19 Jun 2019 08:56:44 GMT): richardmurillo (Mon, 24 Jun 2019 20:59:49 GMT): qsmen (Thu, 27 Jun 2019 05:56:36 GMT): qsmen (Thu, 27 Jun 2019 05:57:02 GMT): qsmen (Thu, 27 Jun 2019 06:21:41 GMT): tongli (Fri, 28 Jun 2019 04:08:34 GMT): bjwswang (Tue, 09 Jul 2019 03:43:26 GMT): bjwswang (Tue, 09 Jul 2019 03:43:41 GMT): baohua (Tue, 09 Jul 2019 08:56:05 GMT): bjwswang (Wed, 10 Jul 2019 03:31:41 GMT): bjwswang (Wed, 10 Jul 2019 03:31:41 GMT): jarvis26 (Wed, 10 Jul 2019 11:09:52 GMT): jarvis26 (Wed, 10 Jul 2019 11:09:52 GMT): baohua (Wed, 10 Jul 2019 13:57:49 GMT): jarvis26 (Wed, 10 Jul 2019 13:58:38 GMT): baohua (Wed, 10 Jul 2019 13:59:44 GMT): jarvis26 (Wed, 10 Jul 2019 14:00:34 GMT): jarvis26 (Wed, 10 Jul 2019 14:01:12 GMT): baohua (Wed, 10 Jul 2019 14:01:45 GMT): jarvis26 (Wed, 10 Jul 2019 14:05:22 GMT): baohua (Wed, 10 Jul 2019 14:07:20 GMT): jarvis26 (Wed, 10 Jul 2019 14:08:27 GMT): jarvis26 (Wed, 10 Jul 2019 14:08:53 GMT): jarvis26 (Wed, 10 Jul 2019 14:11:25 GMT): jarvis26 (Wed, 10 Jul 2019 14:14:11 GMT): Bentipe (Wed, 10 Jul 2019 15:04:10 GMT): Bentipe (Wed, 10 Jul 2019 15:30:25 GMT): baohua (Thu, 11 Jul 2019 02:17:46 GMT): WCamaly (Thu, 11 Jul 2019 19:32:01 GMT): WCamaly (Thu, 11 Jul 2019 19:32:03 GMT): Relax20190715 (Mon, 15 Jul 2019 12:41:58 GMT): HarryLovesCode (Wed, 17 Jul 2019 15:27:57 GMT): HarryLovesCode (Wed, 17 Jul 2019 15:28:02 GMT): sureshtedla (Wed, 17 Jul 2019 16:18:01 GMT): tongli (Wed, 17 Jul 2019 19:24:58 GMT): tongli (Wed, 17 Jul 2019 19:25:12 GMT): bjwswang (Thu, 18 Jul 2019 08:24:18 GMT): bjwswang (Thu, 18 Jul 2019 08:36:09 GMT): bjwswang (Thu, 18 Jul 2019 08:36:09 GMT): sureshtedla (Thu, 18 Jul 2019 13:12:12 GMT): RealDeanZhao (Tue, 23 Jul 2019 03:00:46 GMT): RealDeanZhao (Tue, 23 Jul 2019 04:58:53 GMT): mallikarjunasai995 (Tue, 23 Jul 2019 12:05:13 GMT): mallikarjunasai995 (Tue, 23 Jul 2019 17:21:12 GMT): mallikarjunasai995 (Tue, 23 Jul 2019 17:21:14 GMT): mallikarjunasai995 (Tue, 23 Jul 2019 17:36:36 GMT): mallikarjunasai995 (Tue, 23 Jul 2019 17:48:31 GMT): mallikarjunasai995 (Tue, 23 Jul 2019 17:48:36 GMT): mallikarjunasai995 (Wed, 24 Jul 2019 05:12:54 GMT): mallikarjunasai995 (Wed, 24 Jul 2019 05:24:38 GMT): mallikarjunasai995 (Wed, 24 Jul 2019 09:26:08 GMT): mallikarjunasai995 (Wed, 24 Jul 2019 09:26:13 GMT): mallikarjunasai995 (Wed, 24 Jul 2019 09:26:24 GMT): mwklein (Thu, 25 Jul 2019 14:50:32 GMT): jarvis26 (Thu, 01 Aug 2019 17:54:53 GMT): tongli (Thu, 01 Aug 2019 17:56:58 GMT): jarvis26 (Thu, 01 Aug 2019 17:58:40 GMT): tongli (Thu, 01 Aug 2019 17:59:56 GMT): jarvis26 (Thu, 01 Aug 2019 17:59:59 GMT): tongli (Thu, 01 Aug 2019 18:05:07 GMT): tongli (Thu, 01 Aug 2019 18:05:09 GMT): tongli (Thu, 01 Aug 2019 18:05:19 GMT): jarvis26 (Thu, 01 Aug 2019 18:05:49 GMT): tongli (Thu, 01 Aug 2019 18:07:17 GMT): tongli (Thu, 01 Aug 2019 18:07:23 GMT): tongli (Thu, 01 Aug 2019 18:07:29 GMT): tongli (Thu, 01 Aug 2019 18:08:44 GMT): tongli (Thu, 01 Aug 2019 18:09:00 GMT): tongli (Thu, 01 Aug 2019 18:09:06 GMT): jarvis26 (Thu, 01 Aug 2019 18:10:25 GMT): tongli (Thu, 01 Aug 2019 18:11:05 GMT): tongli (Thu, 01 Aug 2019 18:11:21 GMT): tongli (Thu, 01 Aug 2019 18:11:29 GMT): tongli (Thu, 01 Aug 2019 18:11:36 GMT): tongli (Thu, 01 Aug 2019 18:11:57 GMT): tongli (Thu, 01 Aug 2019 18:12:06 GMT): jarvis26 (Thu, 01 Aug 2019 18:13:00 GMT): tongli (Thu, 01 Aug 2019 18:13:59 GMT): tongli (Thu, 01 Aug 2019 18:14:20 GMT): tongli (Thu, 01 Aug 2019 18:14:34 GMT): tongli (Thu, 01 Aug 2019 18:15:11 GMT): tongli (Thu, 01 Aug 2019 18:15:17 GMT): tongli (Thu, 01 Aug 2019 18:15:24 GMT): jarvis26 (Thu, 01 Aug 2019 18:15:48 GMT): jarvis26 (Thu, 01 Aug 2019 18:16:23 GMT): tongli (Thu, 01 Aug 2019 18:17:50 GMT): tongli (Thu, 01 Aug 2019 18:17:57 GMT): jarvis26 (Thu, 01 Aug 2019 18:18:24 GMT): tongli (Thu, 01 Aug 2019 18:18:57 GMT): tongli (Thu, 01 Aug 2019 18:19:08 GMT): tongli (Thu, 01 Aug 2019 18:20:50 GMT): tongli (Thu, 01 Aug 2019 18:21:03 GMT): jarvis26 (Thu, 01 Aug 2019 18:21:12 GMT): tongli (Thu, 01 Aug 2019 18:21:19 GMT): jarvis26 (Thu, 01 Aug 2019 18:21:39 GMT): tongli (Thu, 01 Aug 2019 18:21:50 GMT): jarvis26 (Thu, 01 Aug 2019 18:22:16 GMT): tongli (Thu, 01 Aug 2019 18:22:31 GMT): tongli (Thu, 01 Aug 2019 18:22:35 GMT): tongli (Thu, 01 Aug 2019 18:24:09 GMT): jarvis26 (Thu, 01 Aug 2019 18:24:10 GMT): tongli (Thu, 01 Aug 2019 18:25:01 GMT): tongli (Thu, 01 Aug 2019 18:25:17 GMT): tongli (Thu, 01 Aug 2019 18:25:40 GMT): jarvis26 (Thu, 01 Aug 2019 18:25:59 GMT): jarvis26 (Thu, 01 Aug 2019 18:27:18 GMT): tongli (Thu, 01 Aug 2019 18:35:00 GMT): tongli (Thu, 01 Aug 2019 18:35:12 GMT): tongli (Thu, 01 Aug 2019 18:35:21 GMT): jarvis26 (Thu, 01 Aug 2019 18:35:28 GMT): tongli (Thu, 01 Aug 2019 18:35:31 GMT): tongli (Thu, 01 Aug 2019 18:35:45 GMT): tongli (Thu, 01 Aug 2019 18:36:03 GMT): tongli (Thu, 01 Aug 2019 18:36:18 GMT): jarvis26 (Thu, 01 Aug 2019 18:40:26 GMT): tongli (Thu, 01 Aug 2019 18:44:55 GMT): tongli (Thu, 01 Aug 2019 18:45:05 GMT): tongli (Thu, 01 Aug 2019 18:45:14 GMT): tongli (Thu, 01 Aug 2019 18:45:21 GMT): jarvis26 (Thu, 01 Aug 2019 18:48:34 GMT): jarvis26 (Thu, 01 Aug 2019 18:49:14 GMT): jarvis26 (Thu, 01 Aug 2019 18:49:34 GMT): tongli (Thu, 01 Aug 2019 18:50:17 GMT): jarvis26 (Thu, 01 Aug 2019 18:51:05 GMT): tongli (Thu, 01 Aug 2019 18:51:20 GMT): tongli (Thu, 01 Aug 2019 18:51:25 GMT): tongli (Thu, 01 Aug 2019 18:51:31 GMT): jarvis26 (Thu, 01 Aug 2019 18:52:07 GMT): jarvis26 (Thu, 01 Aug 2019 18:52:28 GMT): tongli (Thu, 01 Aug 2019 18:56:55 GMT): tongli (Thu, 01 Aug 2019 18:56:58 GMT): tongli (Thu, 01 Aug 2019 18:57:22 GMT): jarvis26 (Thu, 01 Aug 2019 18:57:37 GMT): jarvis26 (Thu, 01 Aug 2019 18:58:21 GMT): tongli (Thu, 01 Aug 2019 18:59:17 GMT): jarvis26 (Thu, 01 Aug 2019 18:59:36 GMT): tongli (Thu, 01 Aug 2019 19:00:51 GMT): jarvis26 (Thu, 01 Aug 2019 19:01:18 GMT): tongli (Thu, 01 Aug 2019 19:01:31 GMT): tongli (Thu, 01 Aug 2019 19:01:35 GMT): tongli (Thu, 01 Aug 2019 19:02:16 GMT): jarvis26 (Thu, 01 Aug 2019 19:03:14 GMT): tongli (Thu, 01 Aug 2019 19:03:32 GMT): jarvis26 (Thu, 01 Aug 2019 19:05:55 GMT): tongli (Thu, 01 Aug 2019 19:06:41 GMT): tongli (Thu, 01 Aug 2019 19:06:50 GMT): tongli (Thu, 01 Aug 2019 19:07:19 GMT): tongli (Thu, 01 Aug 2019 19:07:32 GMT): tongli (Thu, 01 Aug 2019 19:07:43 GMT): tongli (Thu, 01 Aug 2019 19:08:13 GMT): jarvis26 (Thu, 01 Aug 2019 19:11:16 GMT): tongli (Thu, 01 Aug 2019 19:27:36 GMT): jarvis26 (Thu, 01 Aug 2019 19:38:03 GMT): jarvis26 (Thu, 01 Aug 2019 19:38:41 GMT): jarvis26 (Thu, 01 Aug 2019 19:38:41 GMT): jarvis26 (Thu, 01 Aug 2019 19:38:41 GMT): jarvis26 (Thu, 01 Aug 2019 19:39:43 GMT): tongli (Thu, 01 Aug 2019 19:55:09 GMT): tongli (Thu, 01 Aug 2019 19:56:09 GMT): tongli (Thu, 01 Aug 2019 19:56:43 GMT): tongli (Thu, 01 Aug 2019 19:58:01 GMT): tongli (Thu, 01 Aug 2019 19:58:10 GMT): tongli (Thu, 01 Aug 2019 19:58:26 GMT): tongli (Thu, 01 Aug 2019 19:58:46 GMT): tongli (Thu, 01 Aug 2019 19:59:00 GMT): adineshreddy1 (Fri, 02 Aug 2019 06:34:05 GMT): jarvis26 (Fri, 02 Aug 2019 06:40:38 GMT): jarvis26 (Fri, 02 Aug 2019 06:40:58 GMT): jarvis26 (Fri, 02 Aug 2019 06:41:26 GMT): jarvis26 (Fri, 02 Aug 2019 06:42:45 GMT): jarvis26 (Fri, 02 Aug 2019 06:43:34 GMT): mallikarjunasai995 (Fri, 02 Aug 2019 10:20:30 GMT): mallikarjunasai995 (Fri, 02 Aug 2019 12:33:32 GMT): mallikarjunasai995 (Fri, 02 Aug 2019 12:34:10 GMT): mallikarjunasai995 (Fri, 02 Aug 2019 12:40:51 GMT): tongli (Fri, 02 Aug 2019 14:29:05 GMT): liurain (Thu, 08 Aug 2019 04:54:29 GMT): baohua (Thu, 08 Aug 2019 08:07:06 GMT): sanket1211 (Fri, 09 Aug 2019 05:05:17 GMT): jarvis26 (Wed, 14 Aug 2019 09:13:03 GMT): soumyanayak (Thu, 15 Aug 2019 09:21:09 GMT): tongli (Thu, 15 Aug 2019 11:18:25 GMT): jarvis26 (Fri, 16 Aug 2019 06:40:08 GMT): tongli (Fri, 16 Aug 2019 14:21:18 GMT): tongli (Fri, 16 Aug 2019 14:21:26 GMT): tongli (Fri, 16 Aug 2019 14:21:35 GMT): schtho (Sat, 17 Aug 2019 15:15:33 GMT): schtho (Sat, 17 Aug 2019 15:15:34 GMT): schtho (Sat, 17 Aug 2019 15:15:51 GMT): schtho (Sat, 17 Aug 2019 15:15:52 GMT): Bentipe (Mon, 19 Aug 2019 14:21:05 GMT): Bentipe (Mon, 19 Aug 2019 14:21:05 GMT): Bentipe (Tue, 20 Aug 2019 13:42:54 GMT): jarvis26 (Wed, 21 Aug 2019 06:59:38 GMT): tongli (Wed, 21 Aug 2019 18:31:17 GMT): tongli (Wed, 21 Aug 2019 18:32:11 GMT): shitaibin (Thu, 22 Aug 2019 12:21:03 GMT): rodolfoleal (Thu, 29 Aug 2019 00:37:28 GMT): jarvis26 (Mon, 02 Sep 2019 10:42:17 GMT): jarvis26 (Mon, 02 Sep 2019 10:42:17 GMT): tongli (Tue, 03 Sep 2019 13:42:08 GMT): rjones (Wed, 04 Sep 2019 15:32:56 GMT): rjones (Wed, 04 Sep 2019 15:34:48 GMT): rjones (Wed, 04 Sep 2019 15:35:22 GMT): rjones (Wed, 04 Sep 2019 15:37:38 GMT): jarvis26 (Wed, 04 Sep 2019 17:12:15 GMT): rjones (Wed, 04 Sep 2019 22:28:53 GMT): baohua (Sun, 08 Sep 2019 23:21:19 GMT): rjones (Mon, 09 Sep 2019 00:04:01 GMT): rjones (Mon, 09 Sep 2019 00:04:01 GMT): baohua (Tue, 10 Sep 2019 20:35:43 GMT): baohua (Tue, 10 Sep 2019 20:35:43 GMT): baohua (Tue, 10 Sep 2019 20:35:45 GMT): AlbertCL (Thu, 12 Sep 2019 07:19:03 GMT): baohua (Thu, 12 Sep 2019 17:57:09 GMT): rjones (Thu, 12 Sep 2019 18:03:57 GMT): rjones (Thu, 12 Sep 2019 18:04:04 GMT): rjones (Thu, 12 Sep 2019 18:04:17 GMT): tongli (Thu, 12 Sep 2019 18:05:06 GMT): rjones (Thu, 12 Sep 2019 18:05:50 GMT): baohua (Thu, 12 Sep 2019 18:08:34 GMT): baohua (Thu, 12 Sep 2019 18:08:37 GMT): rjones (Mon, 16 Sep 2019 15:34:56 GMT): rjones (Thu, 19 Sep 2019 23:34:41 GMT): rjones (Thu, 19 Sep 2019 23:34:53 GMT): baohua (Mon, 23 Sep 2019 18:32:03 GMT): baohua (Wed, 25 Sep 2019 16:22:32 GMT): baohua (Wed, 25 Sep 2019 16:22:32 GMT): baohua (Fri, 27 Sep 2019 18:34:53 GMT): KellyCooper (Sun, 29 Sep 2019 18:13:13 GMT): kodepareek (Mon, 30 Sep 2019 12:54:27 GMT): kodepareek (Mon, 30 Sep 2019 12:55:21 GMT): tongli (Tue, 01 Oct 2019 12:16:15 GMT): rjones (Tue, 01 Oct 2019 20:38:59 GMT): rjones (Tue, 01 Oct 2019 20:39:18 GMT): rjones (Wed, 02 Oct 2019 01:16:08 GMT): tongli (Wed, 02 Oct 2019 10:09:59 GMT): rjones (Wed, 02 Oct 2019 21:03:55 GMT): tongli (Thu, 03 Oct 2019 17:55:14 GMT): tongli (Thu, 03 Oct 2019 17:55:27 GMT): tongli (Thu, 03 Oct 2019 17:56:47 GMT): rjones (Thu, 03 Oct 2019 21:47:01 GMT): rjones (Thu, 03 Oct 2019 21:47:05 GMT): tongli (Fri, 04 Oct 2019 12:52:59 GMT): tongli (Mon, 07 Oct 2019 18:04:01 GMT): tongli (Mon, 07 Oct 2019 18:09:38 GMT): baohua (Mon, 07 Oct 2019 22:18:35 GMT): rjones (Mon, 07 Oct 2019 22:36:18 GMT): rjones (Mon, 07 Oct 2019 22:37:11 GMT): rjones (Mon, 07 Oct 2019 22:37:39 GMT): tongli (Tue, 08 Oct 2019 13:56:06 GMT): tongli (Tue, 08 Oct 2019 13:56:59 GMT): rjones (Tue, 08 Oct 2019 21:45:51 GMT): tongli (Wed, 09 Oct 2019 14:33:30 GMT): tongli (Wed, 09 Oct 2019 14:33:45 GMT): tongli (Wed, 09 Oct 2019 14:34:15 GMT): rjones (Wed, 09 Oct 2019 21:48:34 GMT): rjones (Wed, 09 Oct 2019 21:48:56 GMT): rjones (Thu, 10 Oct 2019 00:10:59 GMT): rjones (Thu, 10 Oct 2019 00:10:59 GMT): baohua (Thu, 10 Oct 2019 17:21:30 GMT): baohua (Thu, 10 Oct 2019 17:21:30 GMT): baohua (Thu, 10 Oct 2019 17:21:30 GMT): tongli (Thu, 10 Oct 2019 17:28:43 GMT): baohua (Thu, 10 Oct 2019 17:35:00 GMT): rjones (Thu, 10 Oct 2019 17:41:32 GMT): tongli (Thu, 10 Oct 2019 18:42:57 GMT): tongli (Thu, 10 Oct 2019 18:43:12 GMT): tongli (Thu, 10 Oct 2019 19:08:53 GMT): tongli (Thu, 10 Oct 2019 20:53:24 GMT): tongli (Thu, 10 Oct 2019 20:53:51 GMT): tongli (Thu, 10 Oct 2019 20:54:11 GMT): tongli (Thu, 10 Oct 2019 20:54:23 GMT): tongli (Thu, 10 Oct 2019 20:54:51 GMT): tongli (Thu, 10 Oct 2019 20:55:12 GMT): rjones (Fri, 11 Oct 2019 01:18:41 GMT): rjones (Fri, 11 Oct 2019 01:18:50 GMT): rjones (Fri, 11 Oct 2019 01:18:50 GMT): tongli (Fri, 11 Oct 2019 04:27:31 GMT): rjones (Fri, 11 Oct 2019 04:51:54 GMT): rjones (Fri, 11 Oct 2019 04:51:57 GMT): baohua (Fri, 11 Oct 2019 20:03:30 GMT): tongli (Sun, 13 Oct 2019 00:56:20 GMT): tongli (Wed, 16 Oct 2019 14:46:07 GMT): tongli (Wed, 16 Oct 2019 14:46:42 GMT): rjones (Wed, 16 Oct 2019 22:02:58 GMT): rjones (Thu, 17 Oct 2019 00:42:11 GMT): rjones (Thu, 17 Oct 2019 00:42:12 GMT): baohua (Fri, 18 Oct 2019 02:58:44 GMT): baohua (Fri, 18 Oct 2019 02:58:48 GMT): nkl199 (Fri, 18 Oct 2019 08:14:54 GMT): AbhijeetSamanta (Mon, 21 Oct 2019 07:12:11 GMT): AbhijeetSamanta (Mon, 21 Oct 2019 07:12:12 GMT): AbhijeetSamanta (Mon, 21 Oct 2019 07:16:49 GMT): Koushik (Mon, 21 Oct 2019 19:15:04 GMT): baohua (Wed, 23 Oct 2019 23:04:39 GMT): lotty02cho (Thu, 24 Oct 2019 05:42:26 GMT): cDown (Fri, 25 Oct 2019 08:01:13 GMT): cDown (Fri, 25 Oct 2019 08:16:01 GMT): guptasndp10 (Fri, 25 Oct 2019 08:51:14 GMT): guptasndp10 (Fri, 25 Oct 2019 08:51:16 GMT): rjones (Fri, 25 Oct 2019 09:31:08 GMT): baohua (Fri, 25 Oct 2019 18:56:07 GMT): baohua (Fri, 25 Oct 2019 18:56:28 GMT): tongli (Fri, 25 Oct 2019 18:59:44 GMT): baohua (Tue, 29 Oct 2019 21:20:31 GMT): baohua (Wed, 30 Oct 2019 20:41:35 GMT): baohua (Fri, 01 Nov 2019 17:27:20 GMT): baohua (Fri, 01 Nov 2019 18:19:09 GMT): sureshtedla (Wed, 06 Nov 2019 13:36:15 GMT): sureshtedla (Wed, 06 Nov 2019 13:36:56 GMT): baohua (Fri, 08 Nov 2019 02:28:31 GMT): baohua (Fri, 08 Nov 2019 04:46:05 GMT): guptasndp10 (Fri, 08 Nov 2019 13:08:52 GMT): baohua (Thu, 14 Nov 2019 20:13:52 GMT): Man-Jain (Fri, 15 Nov 2019 07:32:53 GMT): baohua (Fri, 15 Nov 2019 17:36:41 GMT): Man-Jain (Fri, 15 Nov 2019 17:39:52 GMT): Man-Jain (Fri, 15 Nov 2019 17:39:52 GMT): gwalsh (Mon, 18 Nov 2019 19:01:29 GMT): AbhijeetSamanta (Wed, 20 Nov 2019 05:45:17 GMT): AbhijeetSamanta (Wed, 20 Nov 2019 05:45:52 GMT): AbhijeetSamanta (Wed, 20 Nov 2019 07:43:31 GMT): AbhijeetSamanta (Wed, 20 Nov 2019 07:43:31 GMT): AbhijeetSamanta (Wed, 20 Nov 2019 11:43:59 GMT): AbhijeetSamanta (Wed, 20 Nov 2019 11:47:49 GMT): tongli (Wed, 20 Nov 2019 15:52:31 GMT): tongli (Wed, 20 Nov 2019 15:53:10 GMT): tongli (Wed, 20 Nov 2019 15:53:19 GMT): tongli (Wed, 20 Nov 2019 15:53:57 GMT): AbhijeetSamanta (Wed, 20 Nov 2019 19:17:44 GMT): tongli (Wed, 20 Nov 2019 19:19:45 GMT): tongli (Wed, 20 Nov 2019 19:19:57 GMT): tongli (Wed, 20 Nov 2019 19:20:14 GMT): AbhijeetSamanta (Wed, 20 Nov 2019 19:20:23 GMT): tongli (Wed, 20 Nov 2019 19:20:29 GMT): tongli (Wed, 20 Nov 2019 19:20:33 GMT): tongli (Wed, 20 Nov 2019 19:20:53 GMT): AbhijeetSamanta (Wed, 20 Nov 2019 19:21:22 GMT): tongli (Wed, 20 Nov 2019 19:21:35 GMT): tongli (Wed, 20 Nov 2019 19:22:23 GMT): AbhijeetSamanta (Wed, 20 Nov 2019 19:22:38 GMT): tongli (Wed, 20 Nov 2019 19:22:45 GMT): AbhijeetSamanta (Wed, 20 Nov 2019 19:22:53 GMT): tongli (Wed, 20 Nov 2019 19:23:11 GMT): tongli (Wed, 20 Nov 2019 19:23:24 GMT): tongli (Wed, 20 Nov 2019 19:23:31 GMT): AbhijeetSamanta (Wed, 20 Nov 2019 19:23:41 GMT): tongli (Wed, 20 Nov 2019 19:23:56 GMT): tongli (Wed, 20 Nov 2019 19:24:07 GMT): tongli (Wed, 20 Nov 2019 19:24:13 GMT): AbhijeetSamanta (Wed, 20 Nov 2019 19:24:42 GMT): tongli (Wed, 20 Nov 2019 19:24:56 GMT): tongli (Wed, 20 Nov 2019 19:25:09 GMT): AbhijeetSamanta (Wed, 20 Nov 2019 19:25:19 GMT): tongli (Wed, 20 Nov 2019 19:25:36 GMT): tongli (Wed, 20 Nov 2019 19:25:44 GMT): AbhijeetSamanta (Wed, 20 Nov 2019 19:25:44 GMT): AbhijeetSamanta (Wed, 20 Nov 2019 19:26:00 GMT): AbhijeetSamanta (Wed, 20 Nov 2019 19:26:18 GMT): tongli (Wed, 20 Nov 2019 19:26:29 GMT): AbhijeetSamanta (Wed, 20 Nov 2019 19:26:46 GMT): AbhijeetSamanta (Wed, 20 Nov 2019 19:26:46 GMT): tongli (Wed, 20 Nov 2019 19:26:53 GMT): AbhijeetSamanta (Wed, 20 Nov 2019 19:26:58 GMT): AbhijeetSamanta (Wed, 20 Nov 2019 19:36:07 GMT): tongli (Thu, 21 Nov 2019 02:56:46 GMT): AbhijeetSamanta (Thu, 21 Nov 2019 13:41:00 GMT): AbhijeetSamanta (Thu, 21 Nov 2019 13:41:00 GMT): AbhijeetSamanta (Thu, 21 Nov 2019 13:44:16 GMT): AbhijeetSamanta (Thu, 21 Nov 2019 13:44:16 GMT): AbhijeetSamanta (Thu, 21 Nov 2019 13:58:11 GMT): tongli (Thu, 21 Nov 2019 16:24:42 GMT): tongli (Thu, 21 Nov 2019 16:25:02 GMT): tongli (Thu, 21 Nov 2019 16:25:12 GMT): tongli (Thu, 21 Nov 2019 16:25:32 GMT): tongli (Thu, 21 Nov 2019 16:48:24 GMT): tongli (Thu, 21 Nov 2019 16:48:27 GMT): tongli (Thu, 21 Nov 2019 17:28:06 GMT): tongli (Thu, 21 Nov 2019 17:28:42 GMT): tongli (Thu, 21 Nov 2019 17:28:53 GMT): tongli (Thu, 21 Nov 2019 17:28:57 GMT): tongli (Thu, 21 Nov 2019 17:38:44 GMT): tongli (Thu, 21 Nov 2019 17:38:48 GMT): tongli (Thu, 21 Nov 2019 17:39:45 GMT): tongli (Thu, 21 Nov 2019 17:40:17 GMT): tongli (Thu, 21 Nov 2019 17:49:15 GMT): tongli (Thu, 21 Nov 2019 17:49:50 GMT): tongli (Thu, 21 Nov 2019 17:51:10 GMT): tongli (Thu, 21 Nov 2019 17:51:39 GMT): tongli (Thu, 21 Nov 2019 17:52:14 GMT): AbhijeetSamanta (Thu, 21 Nov 2019 18:37:13 GMT): AbhijeetSamanta (Thu, 21 Nov 2019 18:37:13 GMT): tongli (Thu, 21 Nov 2019 19:04:24 GMT): tongli (Thu, 21 Nov 2019 19:04:59 GMT): tongli (Thu, 21 Nov 2019 19:05:17 GMT): tongli (Thu, 21 Nov 2019 19:27:56 GMT): AbhijeetSamanta (Thu, 21 Nov 2019 19:32:15 GMT): AbhijeetSamanta (Thu, 21 Nov 2019 19:35:59 GMT): AbhijeetSamanta (Thu, 21 Nov 2019 19:35:59 GMT): AbhijeetSamanta (Thu, 21 Nov 2019 19:37:04 GMT): AbhijeetSamanta (Thu, 21 Nov 2019 20:20:34 GMT): AbhijeetSamanta (Thu, 21 Nov 2019 20:20:34 GMT): AbhijeetSamanta (Thu, 21 Nov 2019 20:42:37 GMT): AbhijeetSamanta (Thu, 21 Nov 2019 20:43:34 GMT): tongli (Fri, 22 Nov 2019 01:38:35 GMT): tongli (Fri, 22 Nov 2019 01:39:13 GMT): AbhijeetSamanta (Fri, 22 Nov 2019 16:24:43 GMT): tongli (Sat, 23 Nov 2019 19:36:14 GMT): tongli (Sat, 23 Nov 2019 19:36:52 GMT): tongli (Sat, 23 Nov 2019 19:37:07 GMT): AbhijeetSamanta (Mon, 25 Nov 2019 15:49:25 GMT): tongli (Mon, 25 Nov 2019 15:50:07 GMT): tongli (Mon, 25 Nov 2019 15:50:39 GMT): AbhijeetSamanta (Mon, 25 Nov 2019 16:08:38 GMT): AbhijeetSamanta (Mon, 25 Nov 2019 16:10:15 GMT): tongli (Mon, 25 Nov 2019 16:22:21 GMT): cmgabriel (Tue, 26 Nov 2019 16:47:29 GMT): tongli (Tue, 26 Nov 2019 19:50:55 GMT): tongli (Tue, 26 Nov 2019 19:51:53 GMT): AbhijeetSamanta (Mon, 02 Dec 2019 17:42:59 GMT): AbhijeetSamanta (Tue, 03 Dec 2019 13:58:27 GMT): AbhijeetSamanta (Tue, 03 Dec 2019 13:59:15 GMT): mrudav.shukla (Wed, 04 Dec 2019 16:19:47 GMT): mrudav.shukla (Wed, 04 Dec 2019 16:21:35 GMT): tongli (Wed, 04 Dec 2019 18:14:56 GMT): tongli (Wed, 04 Dec 2019 18:15:24 GMT): vinayakkumar (Thu, 05 Dec 2019 09:50:35 GMT): rjones (Thu, 05 Dec 2019 18:41:58 GMT): rjones (Thu, 05 Dec 2019 18:42:00 GMT): tengc (Thu, 05 Dec 2019 19:46:44 GMT): tongli (Fri, 06 Dec 2019 02:48:33 GMT): tengc (Mon, 09 Dec 2019 17:14:28 GMT): tengc (Mon, 09 Dec 2019 17:14:28 GMT): palra (Tue, 10 Dec 2019 09:33:53 GMT): palra (Tue, 10 Dec 2019 09:36:42 GMT): rjones (Tue, 10 Dec 2019 18:00:14 GMT): baohua (Thu, 12 Dec 2019 17:58:56 GMT): nitishbhardwaj19 (Sun, 15 Dec 2019 11:52:31 GMT): nitishbhardwaj19 (Sun, 15 Dec 2019 11:53:36 GMT): ozbek (Mon, 16 Dec 2019 12:03:39 GMT): JHamilton (Mon, 16 Dec 2019 19:45:21 GMT): JHamilton (Mon, 16 Dec 2019 19:45:22 GMT): AbhijeetSamanta (Wed, 18 Dec 2019 12:46:39 GMT): nitishbhardwaj19 (Wed, 18 Dec 2019 16:02:52 GMT): baohua (Thu, 19 Dec 2019 17:31:57 GMT): ddhulla (Fri, 27 Dec 2019 14:02:35 GMT): ddhulla (Fri, 27 Dec 2019 14:02:37 GMT): ddhulla (Fri, 27 Dec 2019 14:02:37 GMT): tengc (Tue, 31 Dec 2019 03:22:35 GMT): RaphD (Thu, 02 Jan 2020 15:15:26 GMT): RaphD (Thu, 02 Jan 2020 15:15:26 GMT): ahmetbilal (Mon, 06 Jan 2020 11:53:01 GMT): ahmetbilal (Mon, 06 Jan 2020 11:53:01 GMT): ahmetbilal (Mon, 06 Jan 2020 11:53:55 GMT): tongli (Mon, 06 Jan 2020 17:24:01 GMT): tongli (Mon, 06 Jan 2020 17:24:17 GMT): tongli (Mon, 06 Jan 2020 17:24:46 GMT): Psingh (Tue, 07 Jan 2020 05:17:15 GMT): tengc (Fri, 10 Jan 2020 15:18:54 GMT): tengc (Fri, 10 Jan 2020 15:18:54 GMT): rjones (Fri, 10 Jan 2020 19:14:19 GMT): dineshthemacho1 (Wed, 22 Jan 2020 06:05:21 GMT): Silona (Wed, 22 Jan 2020 21:31:04 GMT): nitishbhardwaj19 (Sun, 26 Jan 2020 08:22:46 GMT): Silona (Mon, 27 Jan 2020 22:25:55 GMT): saanvijay (Tue, 28 Jan 2020 10:39:50 GMT): baohua (Wed, 29 Jan 2020 18:26:46 GMT): lavish 1 (Mon, 03 Feb 2020 11:24:42 GMT): stone-ch (Tue, 11 Feb 2020 08:23:16 GMT): Silona (Thu, 13 Feb 2020 18:23:27 GMT): Silona (Mon, 17 Feb 2020 22:17:38 GMT): abu (Wed, 19 Feb 2020 10:48:17 GMT): abu (Wed, 19 Feb 2020 10:48:18 GMT): abu (Wed, 19 Feb 2020 10:48:46 GMT): tongli (Wed, 19 Feb 2020 15:41:48 GMT): tongli (Wed, 19 Feb 2020 15:42:41 GMT): ahmetbilal (Fri, 21 Feb 2020 08:33:21 GMT): abu (Fri, 21 Feb 2020 12:07:44 GMT): tongli (Fri, 21 Feb 2020 12:16:34 GMT): abu (Fri, 21 Feb 2020 12:20:56 GMT): tongli (Fri, 21 Feb 2020 12:52:23 GMT): frankouyang (Sat, 22 Feb 2020 04:33:20 GMT): japidei (Sat, 22 Feb 2020 12:03:23 GMT): dexhunter (Sun, 23 Feb 2020 21:13:36 GMT): dexhunter (Sun, 23 Feb 2020 21:13:44 GMT): dexhunter (Sun, 23 Feb 2020 21:14:30 GMT): rjones (Sun, 23 Feb 2020 23:08:13 GMT): rjones (Sun, 23 Feb 2020 23:10:30 GMT): rjones (Sun, 23 Feb 2020 23:10:43 GMT): rjones (Sun, 23 Feb 2020 23:12:30 GMT): tongli (Mon, 24 Feb 2020 16:49:47 GMT): tongli (Mon, 24 Feb 2020 16:52:49 GMT): tongli (Mon, 24 Feb 2020 16:55:47 GMT): tongli (Mon, 24 Feb 2020 16:57:17 GMT): tongli (Mon, 24 Feb 2020 16:57:51 GMT): tongli (Mon, 24 Feb 2020 16:58:53 GMT): tongli (Mon, 24 Feb 2020 16:58:53 GMT): rjones (Mon, 24 Feb 2020 18:10:08 GMT): tongli (Mon, 24 Feb 2020 18:10:19 GMT): tongli (Mon, 24 Feb 2020 18:10:24 GMT): tongli (Mon, 24 Feb 2020 18:11:01 GMT): rjones (Mon, 24 Feb 2020 18:12:07 GMT): rjones (Mon, 24 Feb 2020 18:12:49 GMT): rjones (Mon, 24 Feb 2020 18:21:45 GMT): rjones (Mon, 24 Feb 2020 18:24:38 GMT): rjones (Mon, 24 Feb 2020 19:01:34 GMT): rjones (Mon, 24 Feb 2020 19:01:47 GMT): tongli (Mon, 24 Feb 2020 19:02:38 GMT): rjones (Mon, 24 Feb 2020 19:02:56 GMT): rjones (Mon, 24 Feb 2020 19:03:49 GMT): tongli (Mon, 24 Feb 2020 19:07:13 GMT): rjones (Mon, 24 Feb 2020 19:08:10 GMT): tongli (Mon, 24 Feb 2020 19:08:16 GMT): tongli (Mon, 24 Feb 2020 19:08:51 GMT): rjones (Mon, 24 Feb 2020 19:11:03 GMT): rjones (Mon, 24 Feb 2020 19:11:08 GMT): tongli (Mon, 24 Feb 2020 19:11:18 GMT): tongli (Mon, 24 Feb 2020 19:11:30 GMT): rjones (Mon, 24 Feb 2020 19:11:42 GMT): tongli (Mon, 24 Feb 2020 19:11:49 GMT): rjones (Mon, 24 Feb 2020 19:12:05 GMT): Koushik (Thu, 27 Feb 2020 01:59:47 GMT): cmonkeydo (Tue, 03 Mar 2020 21:02:58 GMT): cmonkeydo (Fri, 06 Mar 2020 20:51:00 GMT): rjones (Fri, 06 Mar 2020 20:58:21 GMT): rjones (Fri, 06 Mar 2020 20:58:39 GMT): cmonkeydo (Fri, 06 Mar 2020 20:58:46 GMT): cmonkeydo (Fri, 06 Mar 2020 21:11:10 GMT): rjones (Fri, 06 Mar 2020 21:13:08 GMT): cmonkeydo (Tue, 10 Mar 2020 19:42:30 GMT): cmonkeydo (Tue, 10 Mar 2020 19:45:48 GMT): cmonkeydo (Tue, 10 Mar 2020 20:31:06 GMT): cmonkeydo (Tue, 10 Mar 2020 20:32:34 GMT): cmonkeydo (Tue, 10 Mar 2020 20:32:34 GMT): rjones (Tue, 10 Mar 2020 20:51:04 GMT): rjones (Tue, 10 Mar 2020 20:51:04 GMT): rjones (Tue, 10 Mar 2020 20:51:34 GMT): rjones (Tue, 10 Mar 2020 20:52:17 GMT): rjones (Tue, 10 Mar 2020 20:52:39 GMT): cmonkeydo (Tue, 10 Mar 2020 20:55:00 GMT): cmonkeydo (Tue, 10 Mar 2020 20:56:20 GMT): rjones (Tue, 10 Mar 2020 20:58:14 GMT): rjones (Tue, 10 Mar 2020 20:59:22 GMT): cmonkeydo (Tue, 10 Mar 2020 21:01:14 GMT): rjones (Tue, 10 Mar 2020 21:01:46 GMT): cmonkeydo (Tue, 10 Mar 2020 21:02:46 GMT): silliman (Wed, 11 Mar 2020 02:46:01 GMT): cmgabriel (Wed, 11 Mar 2020 17:13:43 GMT): baohua (Fri, 13 Mar 2020 22:06:14 GMT): DayalMukati (Mon, 16 Mar 2020 13:05:28 GMT): DayalMukati (Mon, 16 Mar 2020 13:05:30 GMT): cmonkeydo (Mon, 16 Mar 2020 17:56:32 GMT): baohua (Fri, 20 Mar 2020 00:22:29 GMT): randyshu (Fri, 20 Mar 2020 02:25:57 GMT): gokulalex (Fri, 20 Mar 2020 05:30:13 GMT): baohua (Sat, 28 Mar 2020 04:26:27 GMT): rjones (Tue, 07 Apr 2020 19:34:55 GMT): MHBauer (Thu, 16 Apr 2020 21:26:58 GMT): baohua (Mon, 20 Apr 2020 01:25:07 GMT): SUSHOBHAN (Mon, 04 May 2020 09:39:15 GMT): spartucus (Fri, 08 May 2020 06:57:57 GMT): spartucus (Fri, 08 May 2020 06:57:58 GMT): spartucus (Fri, 08 May 2020 10:21:41 GMT): rjones (Fri, 08 May 2020 19:10:59 GMT): tongli (Fri, 08 May 2020 19:22:56 GMT): rjones (Fri, 08 May 2020 19:25:05 GMT): rjones (Sat, 09 May 2020 02:20:38 GMT): spartucus (Sat, 09 May 2020 02:34:24 GMT): tinywell (Tue, 12 May 2020 03:12:29 GMT): spartucus (Tue, 12 May 2020 04:18:24 GMT): rjones (Tue, 12 May 2020 16:17:06 GMT): samarth9201 (Tue, 19 May 2020 04:56:56 GMT): sownak (Tue, 19 May 2020 09:52:36 GMT): jagpreet (Wed, 20 May 2020 09:17:24 GMT): jagpreet (Wed, 20 May 2020 09:17:25 GMT): jagpreet (Thu, 21 May 2020 11:12:42 GMT): ooojerryooo (Tue, 26 May 2020 01:57:02 GMT): baohua (Thu, 28 May 2020 23:17:16 GMT): rjones (Sat, 30 May 2020 13:15:10 GMT): spartucus (Tue, 02 Jun 2020 08:43:25 GMT): vglafirov (Tue, 02 Jun 2020 20:37:29 GMT): HunterJW (Sun, 07 Jun 2020 03:33:09 GMT): HunterJW (Sun, 07 Jun 2020 03:33:11 GMT): rjones (Sun, 07 Jun 2020 17:46:26 GMT): HunterJW (Thu, 11 Jun 2020 09:04:38 GMT): sillysachin (Thu, 11 Jun 2020 14:45:16 GMT): pouya (Sun, 21 Jun 2020 10:05:04 GMT): iamksseo (Tue, 23 Jun 2020 07:51:19 GMT): iamksseo (Tue, 23 Jun 2020 07:51:19 GMT): iamksseo (Tue, 23 Jun 2020 08:11:09 GMT): iamksseo (Wed, 24 Jun 2020 01:24:09 GMT): iamksseo (Wed, 24 Jun 2020 01:24:15 GMT): iamksseo (Wed, 24 Jun 2020 01:26:01 GMT): iamksseo (Wed, 24 Jun 2020 01:26:03 GMT): pikvik (Wed, 01 Jul 2020 20:31:39 GMT): randyshu (Tue, 21 Jul 2020 08:22:10 GMT): randyshu (Wed, 22 Jul 2020 04:47:35 GMT): randyshu (Wed, 22 Jul 2020 04:49:52 GMT): Sushilpaneru (Tue, 28 Jul 2020 18:26:30 GMT): christo 244 (Sun, 09 Aug 2020 13:54:44 GMT): christo 244 (Sun, 09 Aug 2020 13:54:44 GMT): jagpreet (Thu, 13 Aug 2020 12:56:49 GMT): HeMBaD (Thu, 20 Aug 2020 10:45:38 GMT): AshutoshKumar7 (Thu, 20 Aug 2020 11:15:32 GMT): AlejandroCortesLeal (Fri, 21 Aug 2020 00:14:56 GMT): jcldnatv (Sat, 22 Aug 2020 17:29:35 GMT): AshutoshKumar7 (Thu, 27 Aug 2020 14:48:45 GMT): AshutoshKumar7 (Fri, 28 Aug 2020 06:06:50 GMT): rjones (Fri, 28 Aug 2020 17:10:02 GMT): dexhunter (Mon, 31 Aug 2020 15:38:54 GMT): dexhunter (Mon, 31 Aug 2020 15:38:54 GMT): george9898 (Wed, 09 Sep 2020 19:47:32 GMT): baohua (Fri, 11 Sep 2020 19:44:56 GMT): nimeshpr (Mon, 14 Sep 2020 14:28:48 GMT): nimeshpr (Mon, 14 Sep 2020 14:31:30 GMT): baohua (Thu, 17 Sep 2020 16:13:15 GMT): baohua (Thu, 17 Sep 2020 16:31:10 GMT): baohua (Thu, 17 Sep 2020 16:31:33 GMT): baohua (Thu, 17 Sep 2020 16:31:33 GMT): vioking (Fri, 18 Sep 2020 16:52:45 GMT): Sandeepk40 (Wed, 07 Oct 2020 04:29:32 GMT): skulos (Fri, 09 Oct 2020 07:21:48 GMT): dengjiali (Wed, 04 Nov 2020 08:00:44 GMT): dongge531 (Wed, 04 Nov 2020 11:01:17 GMT): dongge531 (Wed, 04 Nov 2020 11:01:20 GMT): dongge531 (Wed, 04 Nov 2020 11:02:05 GMT): reviz0r (Mon, 16 Nov 2020 20:25:03 GMT): zhangshuai (Thu, 19 Nov 2020 06:33:56 GMT): piyushmaheshwari65 (Wed, 25 Nov 2020 04:18:17 GMT): Suki (Tue, 01 Dec 2020 07:41:05 GMT): Suki (Wed, 02 Dec 2020 02:19:45 GMT): Suki (Wed, 02 Dec 2020 09:32:31 GMT): yixinhuo (Tue, 22 Dec 2020 16:24:31 GMT): zhukai 1 (Thu, 31 Dec 2020 00:54:43 GMT): zhukai 1 (Thu, 31 Dec 2020 00:54:43 GMT): user216 (Wed, 03 Feb 2021 12:10:26 GMT): Helen_Garneau (Thu, 04 Feb 2021 15:10:40 GMT): chengshuangquan (Mon, 01 Mar 2021 08:28:55 GMT): chengshuangquan (Mon, 01 Mar 2021 08:29:41 GMT): chengshuangquan (Mon, 01 Mar 2021 08:30:53 GMT): chengshuangquan (Mon, 01 Mar 2021 08:31:35 GMT): chengshuangquan (Mon, 01 Mar 2021 08:35:49 GMT): chengshuangquan (Mon, 01 Mar 2021 08:45:34 GMT): rjones (Wed, 03 Mar 2021 18:41:27 GMT): IvanV6 (Thu, 04 Mar 2021 10:34:14 GMT): IvanV6 (Thu, 04 Mar 2021 10:34:14 GMT): baohua (Thu, 25 Mar 2021 18:02:11 GMT): mohjam2004 (Wed, 31 Mar 2021 04:34:10 GMT): mohjam2004 (Wed, 31 Mar 2021 04:34:10 GMT): mohjam2004 (Wed, 31 Mar 2021 04:34:53 GMT): mohjam2004 (Wed, 31 Mar 2021 04:35:16 GMT): mohjam2004 (Wed, 31 Mar 2021 04:35:26 GMT): mohjam2004 (Wed, 31 Mar 2021 04:36:03 GMT): mohjam2004 (Wed, 31 Mar 2021 04:36:03 GMT): mohjam2004 (Wed, 31 Mar 2021 04:36:38 GMT): mohjam2004 (Wed, 31 Mar 2021 04:37:33 GMT): mohjam2004 (Wed, 31 Mar 2021 04:55:57 GMT): mohjam2004 (Wed, 31 Mar 2021 04:55:57 GMT): mohjam2004 (Wed, 31 Mar 2021 04:56:07 GMT): mohjam2004 (Wed, 31 Mar 2021 04:56:31 GMT): mohjam2004 (Wed, 31 Mar 2021 04:56:31 GMT): mohjam2004 (Wed, 31 Mar 2021 05:36:10 GMT): mohjam2004 (Wed, 31 Mar 2021 05:36:12 GMT): mohjam2004 (Wed, 31 Mar 2021 05:36:34 GMT): mohjam2004 (Wed, 31 Mar 2021 05:54:07 GMT): mohjam2004 (Wed, 31 Mar 2021 05:54:09 GMT): mohjam2004 (Wed, 31 Mar 2021 06:59:48 GMT): mohjam2004 (Wed, 31 Mar 2021 06:59:50 GMT): mohjam2004 (Wed, 31 Mar 2021 11:47:27 GMT): mohjam2004 (Wed, 31 Mar 2021 11:47:43 GMT): mohjam2004 (Wed, 31 Mar 2021 11:48:37 GMT): mohjam2004 (Wed, 31 Mar 2021 11:48:37 GMT): mohjam2004 (Mon, 05 Apr 2021 06:28:35 GMT): mohjam2004 (Mon, 05 Apr 2021 06:29:11 GMT): nikolas (Mon, 12 Apr 2021 15:15:36 GMT): SomeshBhalsing (Mon, 19 Apr 2021 12:49:13 GMT): charith1994 (Sat, 15 May 2021 17:04:31 GMT): digitalq (Sat, 22 May 2021 20:27:42 GMT): digitalq (Sat, 22 May 2021 20:27:42 GMT): digitalq (Sat, 22 May 2021 20:28:00 GMT): digitalq (Sat, 22 May 2021 20:28:09 GMT): digitalq (Sat, 22 May 2021 20:28:14 GMT): cmhacker (Sat, 22 May 2021 21:48:53 GMT): cmhacker (Sat, 22 May 2021 21:48:53 GMT): jorgeRodriguez (Sat, 12 Jun 2021 22:38:11 GMT): Djx (Mon, 14 Jun 2021 00:08:55 GMT): dezign (Fri, 18 Jun 2021 09:20:24 GMT): dezign (Fri, 18 Jun 2021 09:20:24 GMT): dezign (Fri, 18 Jun 2021 09:23:06 GMT): IoanSlavu (Sun, 04 Jul 2021 17:24:57 GMT): IoanSlavu (Sun, 04 Jul 2021 17:32:25 GMT): raychan11111 (Mon, 05 Jul 2021 07:29:29 GMT): raychan11111 (Mon, 05 Jul 2021 07:29:29 GMT): tkuhrt (Tue, 13 Jul 2021 15:54:23 GMT): aravindrajck (Thu, 19 Aug 2021 07:54:50 GMT): aravindrajck (Thu, 19 Aug 2021 07:54:50 GMT): aravindrajck (Thu, 19 Aug 2021 07:56:18 GMT): nkaramolegos (Fri, 03 Sep 2021 09:30:49 GMT): nkaramolegos (Fri, 03 Sep 2021 09:30:49 GMT): nkaramolegos (Fri, 03 Sep 2021 09:31:26 GMT): nkaramolegos (Fri, 03 Sep 2021 09:31:59 GMT): rlnrajesh (Thu, 09 Sep 2021 21:48:45 GMT): igorkrupczynski (Thu, 16 Sep 2021 10:09:09 GMT): kongwu 2 (Wed, 22 Sep 2021 08:58:59 GMT): kongwu 2 (Wed, 22 Sep 2021 08:58:59 GMT): kongwu 2 (Wed, 22 Sep 2021 08:59:54 GMT): gameprofits (Fri, 24 Sep 2021 19:15:02 GMT): gameprofits (Fri, 24 Sep 2021 19:15:02 GMT): gameprofits (Fri, 24 Sep 2021 19:16:13 GMT): gameprofits (Wed, 29 Sep 2021 15:13:03 GMT): gameprofits (Wed, 29 Sep 2021 15:13:18 GMT): gameprofits (Wed, 29 Sep 2021 15:13:28 GMT): gameprofits (Wed, 29 Sep 2021 15:16:12 GMT): gameprofits (Wed, 29 Sep 2021 15:19:35 GMT): gameprofits (Wed, 29 Sep 2021 15:19:43 GMT): gameprofits (Wed, 29 Sep 2021 15:20:29 GMT): gameprofits (Wed, 29 Sep 2021 15:23:10 GMT): gameprofits (Wed, 29 Sep 2021 15:26:33 GMT): gameprofits (Wed, 29 Sep 2021 15:28:34 GMT): gameprofits (Wed, 29 Sep 2021 15:29:07 GMT): gameprofits (Wed, 29 Sep 2021 15:30:11 GMT): gameprofits (Wed, 29 Sep 2021 15:36:05 GMT): gameprofits (Wed, 29 Sep 2021 15:39:45 GMT): anil8753 (Sun, 03 Oct 2021 12:29:49 GMT): FrancescoUrbano (Wed, 13 Oct 2021 08:18:35 GMT): FrancescoUrbano (Wed, 13 Oct 2021 08:18:35 GMT): thejas.pavithran (Sat, 16 Oct 2021 01:11:33 GMT): suisingLai (Sun, 07 Nov 2021 14:18:49 GMT): suisingLai (Sun, 07 Nov 2021 14:18:49 GMT): peter_somogyvari (Mon, 13 Dec 2021 18:23:43 GMT): peter_somogyvari (Mon, 13 Dec 2021 18:27:29 GMT): SeanBohan (Fri, 17 Dec 2021 19:59:17 GMT): KavyaR (Tue, 28 Dec 2021 09:08:00 GMT): KavyaR (Tue, 28 Dec 2021 09:10:00 GMT): KavyaR (Tue, 28 Dec 2021 09:10:00 GMT): KavyaR (Tue, 28 Dec 2021 09:10:00 GMT): KavyaR (Tue, 28 Dec 2021 09:10:03 GMT): SkandaGurunathanR (Tue, 28 Dec 2021 10:30:11 GMT): SkandaGurunathanR (Tue, 28 Dec 2021 10:30:11 GMT): amit1nayak (Tue, 28 Dec 2021 13:29:57 GMT): amit1nayak (Tue, 28 Dec 2021 13:49:40 GMT): amit1nayak (Tue, 28 Dec 2021 13:49:53 GMT): amit1nayak (Tue, 28 Dec 2021 13:50:58 GMT): amit1nayak (Tue, 28 Dec 2021 13:52:32 GMT): amit1nayak (Tue, 28 Dec 2021 13:53:00 GMT): PureBlack (Thu, 06 Jan 2022 03:15:41 GMT): KavyaR (Fri, 07 Jan 2022 10:58:09 GMT): KavyaR (Fri, 07 Jan 2022 10:58:38 GMT): SkandaGurunathanR (Mon, 10 Jan 2022 09:46:56 GMT): PureBlack (Fri, 14 Jan 2022 07:00:58 GMT): PureBlack (Fri, 14 Jan 2022 07:00:59 GMT): sbohanlf (Tue, 25 Jan 2022 15:17:18 GMT): bjwswangmail (Tue, 08 Feb 2022 03:34:47 GMT): rjones (Sat, 12 Feb 2022 02:37:54 GMT):
Hello everyone, I am building "n" node cluster where 5
Basically swarm is easy to start and supported by docker inc
k8s is more complicated and powerful, design for app management and is spported by google
if I are not familiar with container cluster the. Suggest start from swarm
to support hyperledger, both should be capable
@sravs can u share the log with err msg when u add new hosts?
ur shared logs don’t include that part
@baohua i got it, i added host, thankyou.
Hello everyone iam a beginner to cello, a host status is 'inactive' still docker is running on that host, how to make it active ? please help me. Thank you in advance
Has joined the channel.
Has joined the channel.
can u someone suggest me the deployment process for hyperledger fabric v1.0?
hi all, Can anyone help me to integrate cello with explorer?
@rajasekharpippalla if you look for a brand new fabric network, please look at this doc.
https://github.com/hyperledger/cello/blob/master/docs/ansible-agent-all-in-one.md
this doc takes you step-by-step to stand up a fabric network on one single node.
which by default uses 1.0.2
If you want to stand up multiple nodes fabric network, then look at this https://github.com/hyperledger/cello/blob/master/docs/ansible-agent-how-to.md
If you want to stand up fabric network on AWS cloud , or OpenStack cloud or Azure, then look at this https://github.com/hyperledger/cello/blob/master/src/agent/ansible/README.md
Hey @tongli I observed that in this ca file https://github.com/hyperledger/cello/blob/master/src/agent/ansible/roles/deploy_compose/fabricsetup/templates/ca-compose.j2 there is no port mapping as we seen in normal docker compose file for hyperledger fabric https://github.com/hyperledger/fabric/blob/release/examples/e2e_cli/docker-compose-e2e.yaml#L18 . Is this intentional?
@MohammadObaid yes. we built overlay network so that each container has its own IP address.
so no port conflict at all. Each will simply use their default port.
if you do docker inspect, you will find each container has different IP address.
and default port is ? 7054 ? or with respect to particular containers?
each type uses a different port I believe.
check files in this directory https://github.com/hyperledger/cello/tree/master/src/agent/ansible/roles/deploy_compose/fabricsetup/templates
such as peer-compose.j2, which tells you the port for peers.
look at order-compose.j2 to see port for orderers.
hmm, I take that back.
orderer-compose does not show orderer port. check the orderer container compose file for exposed default port.
The point is that each container has its own IP, no port conflict possible.
Has joined the channel.
Hi there, Does anyone know if cello is capable of deploying fabric across multiple cloud instances? Im working on a project looking to go to production in about 6 months and we need redundancy in our network incase one of the instances goes down.
Has joined the channel.
@zonkedhazmat cello doesn't add limit to support multiple instances, and need support from the underly cluster platform. Currently we support various cloud platforms. Take docker swarm/k8s for example, it can across multi-datacenters, and on top of it, cello can easily deploy fabric to multiple datacenters, too.
@baohua https://gerrit.hyperledger.org/r/#/c/15689/ request for review
Hi @tongli I beleive this configuration file is used to create fabric-ca container https://github.com/hyperledger/cello/blob/master/src/agent/ansible/roles/deploy_compose/fabricsetup/templates/fabric-ca-server-config.j2 which is entirely different from normal fabric ca config file . Dont you think config file should be similar to as this one http://hyperledger-fabric-ca.readthedocs.io/en/latest/serverconfig.html .?
Hi @tongli @baohua I beleive this configuration file is used to create fabric-ca container https://github.com/hyperledger/cello/blob/master/src/agent/ansible/roles/deploy_compose/fabricsetup/templates/fabric-ca-server-config.j2 which is entirely different from normal fabric ca config file . Dont you think config file should be similar to as this one http://hyperledger-fabric-ca.readthedocs.io/en/latest/serverconfig.html .?
I am asking this because fabric-ca should contain admin:adminpw dataset in their fabric-ca-server.db exist in `/etc/hyperledger/allorgs/orga/ca` folder in each peer container but incase of ansible fabric ca database is empty due to which user cannot enroll into fabric-ca
I tried to replace contents of file but when I do ca container couldnt start
@harsha thanks!
@tongli would u help have a look at @MohammadObaid 's question? Thanks!
@baohua Updated the PR per your comments.
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/15689/
Has joined the channel.
hi..is this a good place to start with, https://medium.com/wearetheledger/a-first-attempt-at-hyperledger-fabric-kubernetes-66e43b12a211 Would want to work with k8s..at hlf deployments...
@MohammadObaid do not think I understand your ca question above, can you elaborate a bit more so that I can help.
@tongli I am trying to understand the role of this file https://github.com/hyperledger/cello/blob/master/src/agent/ansible/roles/deploy_compose/fabricsetup/templates/fabric-ca-server-config.j2 . I believe this file is used to configure fabric-ca right?
@tongli I am trying to understand the role of this file https://github.com/hyperledger/cello/blob/master/src/agent/ansible/roles/deploy_compose/fabricsetup/templates/fabric-ca-server-config.j2 . I believe this file is used to configure fabric-ca right? If yes then there is no org added in affiliation section which will lead to create empty sqlite db for fabric ca .
@ambatigaan If you deploy Fabric on pure kubernetes environment, that will be a good place to start.
If you deploy Fabric on pure kubernetes environment, this will be a good place to start http://www.think-foundry.com/deploy-hyperledger-fabric-on-kubernetes-part-1
@Luke_Chen sure, will try this on a local k8s environment
@MohammadObaid that is the configuration file in addition to this file https://github.com/hyperledger/cello/blob/master/src/agent/ansible/roles/deploy_compose/fabricsetup/templates/ca-compose.j2
when the ca gets started, it maps to a directory which contains all necessary config files.
look at that file should give you hint.
@tongli The problem is with that all configuration , I am unable to enroll users in any certificate authority (ca's). On ca logs I got following
admin.png
@MohammadObaid , our testing team has tested enrolling new users against the cas setting up by using cello, it worked fine for them.
seems to me that your ca name is empty.
cadatabase.png
Hmm Seems like issue with my cello configuration. My ca server database is also empty with no admin entity
make sure that you are hitting the right database.
Ok. btw I just cloned fresh cello repository and without making any changes ran ansible script and then tries to enroll user from ca container and got same `authorization error` .
freshlogs.png
hmm. @MohammadObaid the init should have already run.
you should not need to do that again.
check the ca container log files, you should see something.
Yup . I got the same thing in ca container log as my previous screenshot. Regarding init command I ran as last resort to see if it works but no luck . I will try again tomorrow.
@MohammadObaid can you add the following into your ca configuration file and restart the container and try again?
```registry:
# Maximum number of times a password/secret can be reused for enrollment
# (default: -1, which means there is no limit)
maxenrollments: -1
# Contains identity information which is used when LDAP is disabled
identities:
- name: admin
pass: adminpw
type: client
affiliation: ""
maxenrollments: -1
attrs:
hf.Registrar.Roles: "client,user,peer,validator,auditor"
hf.Registrar.DelegateRoles: "client,user,validator,auditor"
hf.Revoker: true
hf.IntermediateCA: true```
I was told that that section was needed. If that works, I will make a change to cello to include that.
@MohammadObaid I have created an jira issue to track this and have also submitted a patch set to fix this problem. you can see the patch set here https://gerrit.hyperledger.org/r/#/c/15755/
if you download this patch set and was able to enroll and register user and peers, please +1 the patch set.
Has left the channel.
Has joined the channel.
Hi Baohua, I am working on CE-136, which is Integrate blockchain-explorer into the started network, and I have some modification need to let you know.
1. Currently cello/src/agent/docker/_compose_files/fabric-1.0/local/fabric-solo-4.yaml has defined a ca container, however, neither org1 nor org2 is relative to this ca, so I remove ca.example.com, and add ca.org1.example.com and ca.org2.example.com, just like this:
```
# ca.example.com:
# image: hyperledger/fabric-ca
# container_name: ${COMPOSE_PROJECT_NAME}_fabric-ca
# hostname: ca
# command: /go/src/github.com/hyperledger/fabric-ca/bin/ca server start -ca testdata/ec.pem -ca-key testdata/ec-key.pem -config testdata/testconfig.json
# expose:
# - "7054"
# command: fabric-ca-server start -b admin:adminpw
ca.org1.example.com:
image: hyperledger/fabric-ca
container_name: ${COMPOSE_PROJECT_NAME}_fabric-ca-org1
hostname: ca.org1.example.com
environment:
- FABRIC_CA_HOME=/etc/hyperledger/fabric-ca-server
- FABRIC_CA_SERVER_CA_NAME=ca.org1.example.com
- FABRIC_CA_SERVER_CA_CERTFILE=/etc/hyperledger/fabric-ca-server-config/ca.org1.example.com-cert.pem
- FABRIC_CA_SERVER_CA_KEYFILE=/etc/hyperledger/fabric-ca-server-config/f2f5a08d5969b4a21f99b5667a426564a3e640a268d2f6bfe77a0e066efb426f_sk
expose:
- "7054"
volumes:
- ${COMPOSE_PROJECT_PATH}/crypto-config/peerOrganizations/org1.example.com/ca/:/etc/hyperledger/fabric-ca-server-config
command: fabric-ca-server start -b admin:adminpw
ca.org2.example.com:
image: hyperledger/fabric-ca
container_name: ${COMPOSE_PROJECT_NAME}_fabric-ca-org2
hostname: ca.org2.example.com
environment:
- FABRIC_CA_HOME=/etc/hyperledger/fabric-ca-server
- FABRIC_CA_SERVER_CA_NAME=ca.org2.example.com
- FABRIC_CA_SERVER_CA_CERTFILE=/etc/hyperledger/fabric-ca-server-config/ca.org2.example.com-cert.pem
- FABRIC_CA_SERVER_CA_KEYFILE=/etc/hyperledger/fabric-ca-server-config/4a56bcf64913e49a5d6a05de9a71f3fa1c39ff1c4371fbb3d923cd8e3e0f75bd_sk
expose:
- "7054"
volumes:
- ${COMPOSE_PROJECT_PATH}/crypto-config/peerOrganizations/org2.example.com/ca/:/etc/hyperledger/fabric-ca-server-config
command: fabric-ca-server start -b admin:adminpw
```
2. I found some useless private-key files under cello/src/agent/docker/_compose_files/fabric-1.0/local/crypto-config/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp/keystore
and
cello/src/agent/docker/_compose_files/fabric-1.0/local/crypto-config/peerOrganizations/org2.example.com/users/Admin@org1.example.com/msp/keystore
I removed them and left the one that correspond to public key.
3. It seems we don’t mount any chiancode into cli container, so I created a ‘chiancodes’ folder with some chaincode examples in it, and mount this folder into cli container.
4. Since we have mount TLS certificates into Fabric cluster, so I enable TLS by default.
I also found a bug during working on CE-136. For every “fabric-1.0” cluster, we assign a set of worker ports for them, however, we are not actually mapping the container port to worker port, furthermore, we don’t expose the hole cluster to external, only one peer and one ca was exposed. As far as I concerned, we should refine the mapping solution, make every component of “fabric-1.0” cluster can be accessed from external.
@baohua
Hi, I'm trying to set up my HF network on a set of hosts using ansible. I have manually modified hosts file and the next step is to initialize and prepare the servers using the following command: `ansible-playbook -i hosts -e "mode=apply env=os env_type=flanneld" initcluster.yml`. But I am not setting it up on any cloud, so what `env` value should I choose?
Has joined the channel.
@SimonOberzan you can use vars/os.yml
but you have to change few things in that file to match your env.
@SimonOberzan if you tell me a bit more about your existing machines, I can certainly help.
@baohua @hightall can you please take a look at this patch to fix ca configuration issue?
https://gerrit.hyperledger.org/r/#/c/15755/
Oh, I see. What info should I give you? I run it on ubuntu 16.04
I have 3 host and I want to deploy bc1st example
I have 3 hosts and I want to deploy bc1st example
@tongli I see, I have tried to run it but I have a problem regarding keys. There are private_key_file and public_key_file atributes in the cluster configs. I only have my key in .ssh/authorized_keys, no key files. Should I generate a new pair on hosts?
the private key and public key should only available on your ansible controller.
which is the machine where you run the ansible scripts.
Oh so the paths are to the keys on my local machine?
on the machines which you will install fabric , you do not need private key.
if "local machine" means where you will run ansible commands, yes
these keys will be used for communication with the target machines (which fabric will be deployed onto)
Yeah thats what I meant thank you
you do need the public key in the target machine .ssh/authorized_keys file.
otherwise, ssh won't work.
also make sure that your target machine has python installed.
I have done that
ok, great.
and your target machines should be ubuntu 16.04 as OS.
check :)
ok. your target machine should not have golang or docker installed.
if you do, it may fail.
the ansible scripts will install all of them.
It has passed through that already so there should be no problem
Hi @tongli thanks a lot for the patch. I am now been able to enroll users, peer easily :)
@MohammadObaid happy to hear that. thanks for finding this issue.
hope we can get it merged soon.
@baohua @hightall can you take a look at this? it is a simple ca configuration template change. https://gerrit.hyperledger.org/r/#/c/15755/
done
@tongli would u like to help have a look at https://jira.hyperledger.org/browse/CE-173? Thanks
Has joined the channel.
@baohua I will take a look at the issue.
@Hi. I have now successfully ran the initcluster on my three peers. Than I have also ran the setupfabric, but got errors in kafka saying it cannot connect to other kafkas, even though IPs have been resolved successfuly. I then went back and checked logs in etcd.servicer which was the following: https://pastebin.com/5QAqASuG (I think that is OK). Then I checked the ogs of flanneld.service and I kept getting `Nov 29 08:22:36 blockchainIIC flanneld[10209]: E1129 08:22:36.679522 10209 watch.go:43] Watch subnets: delete "/coreos.com": not a subnet, skipping`. I than used `systemctl restart flanneld which resulted in: https://pastebin.com/Mi2rNGwU (Looks OK, but I shoudldn't do that manually). Now I pinged container on a host that is running it and it was a success but when I have pinged it from another host it was unsuccessfull. Is that expected or should I be able to?
I have tried that because i then ran setupfabric and zookeeper kept getting connection timeout errors when trying to connect to zookeepers on other hosts.
@Hi. I have now successfully ran the initcluster on my three peers. I then checked logs in etcd.servicer which was the following: https://pastebin.com/5QAqASuG (I think that is OK). Then I checked the ogs of flanneld.service and I kept getting `Nov 29 08:22:36 blockchainIIC flanneld[10209]: E1129 08:22:36.679522 10209 watch.go:43] Watch subnets: delete "/coreos.com": not a subnet, skipping`. I than used `systemctl restart flanneld which resulted in: https://pastebin.com/Mi2rNGwU (Looks OK, but I shoudldn't do that manually). Now I pinged container on a host that is running it and it was a success but when I have pinged it from another host it was unsuccessfull. Is that expected or should I be able to?
I have tried that because i then ran setupfabric and zookeeper kept getting connection timeout errors when trying to connect to zookeepers on other hosts.
@Hi. I have now successfully ran the initcluster on my three peers. I then checked logs in etcd.servicer which was the following: https://pastebin.com/5QAqASuG (I think that is OK). Then I checked the ogs of flanneld.service and I kept getting `Nov 29 08:22:36 blockchainIIC flanneld[10209]: E1129 08:22:36.679522 10209 watch.go:43] Watch subnets: delete "/coreos.com": not a subnet, skipping`. I than used `systemctl restart flanneld which resulted in: https://pastebin.com/Mi2rNGwU (Looks OK, but I shoudldn't do that manually). Now I pinged container on a host that is running it and it was a success but when I have pinged it from another host it was unsuccessfull. Is that expected or should I be able to connect to it?
I have tried that because i then ran setupfabric and zookeeper kept getting connection timeout errors when trying to connect to zookeepers on other hosts.
@tongli Hi. I have now successfully ran the initcluster on my three peers. I then checked logs in etcd.servicer which was the following: https://pastebin.com/5QAqASuG (I think that is OK). Then I checked the ogs of flanneld.service and I kept getting `Nov 29 08:22:36 blockchainIIC flanneld[10209]: E1129 08:22:36.679522 10209 watch.go:43] Watch subnets: delete "/coreos.com": not a subnet, skipping`. I than used `systemctl restart flanneld which resulted in: https://pastebin.com/Mi2rNGwU (Looks OK, but I shoudldn't do that manually). Now I pinged container on a host that is running it and it was a success but when I have pinged it from another host it was unsuccessfull. Is that expected or should I be able to connect to it?
I have tried that because i then ran setupfabric and zookeeper kept getting connection timeout errors when trying to connect to zookeepers on other hosts.
@tongli Hi. I have now successfully ran the initcluster on my three peers. I then checked logs in etcd.servicer which was the following: https://pastebin.com/5QAqASuG (I think that is OK). Then I checked the ogs of flanneld.service and I kept getting `Nov 29 08:22:36 blockchainIIC flanneld[10209]: E1129 08:22:36.679522 10209 watch.go:43] Watch subnets: delete "/coreos.com": not a subnet, skipping`. I than used `systemctl restart flanneld which resulted in: https://pastebin.com/Mi2rNGwU (Looks OK, but I shoudldn't do that manually). Now I pinged the registarator container on a host that is running it and it was a success but when I have pinged it from another host it was unsuccessfull. Is that expected or should I be able to connect to it?
I have tried that because i then ran setupfabric and zookeeper kept getting connection timeout errors when trying to connect to zookeepers on other hosts.
@tongli Hi. I have now successfully ran the initcluster on my three peers. I then checked logs in etcd.servicer which was the following: https://pastebin.com/5QAqASuG (I think that is OK). Then I checked the ogs of flanneld.service and I kept getting `Nov 29 08:22:36 blockchainIIC flanneld[10209]: E1129 08:22:36.679522 10209 watch.go:43] Watch subnets: delete "/coreos.com": not a subnet, skipping`. I than used `systemctl restart flanneld which resulted in: https://pastebin.com/Mi2rNGwU (Looks OK, but I shoudldn't do that manually). Now I pinged the registarator container on a host that is running it and it was a success but when I have pinged it from another host it was unsuccessfull. That is probably expected, right?
I have tried that because i then ran setupfabric and zookeeper kept getting connection timeout errors when trying to connect to zookeepers on other hosts.
@tongli Hi. I have now successfully ran the initcluster on my three peers. I then checked logs in etcd.servicer which was the following: https://pastebin.com/5QAqASuG (I think that is OK). Then I checked the ogs of flanneld.service and I kept getting `Nov 29 08:22:36 blockchainIIC flanneld[10209]: E1129 08:22:36.679522 10209 watch.go:43] Watch subnets: delete "/coreos.com": not a subnet, skipping`. I than used `systemctl restart flanneld which resulted in: https://pastebin.com/Mi2rNGwU (Looks OK, but I shoudldn't do that manually). Now I pinged the registarator container on a host that is running it and it was a success but when I have pinged it from another host it was unsuccessfull. That is probably expected, right?
I have tried that because i then ran setupfabric and zookeeper kept getting connection timeout errors when trying to connect to zookeepers on other hosts. How would I go about debuging this*
@tongli Hi. I have now successfully ran the initcluster on my three peers. I then checked logs in etcd.servicer which was the following: https://pastebin.com/5QAqASuG (I think that is OK). Then I checked the ogs of flanneld.service and I kept getting `Nov 29 08:22:36 blockchainIIC flanneld[10209]: E1129 08:22:36.679522 10209 watch.go:43] Watch subnets: delete "/coreos.com": not a subnet, skipping`. I than used `systemctl restart flanneld which resulted in: https://pastebin.com/Mi2rNGwU (Looks OK, but I shoudldn't do that manually). Now I pinged the registarator container on a host that is running it and it was a success but when I have pinged it from another host it was unsuccessfull. That is probably expected, right?
I have tried that because i then ran setupfabric and zookeeper kept getting connection timeout errors when trying to connect to zookeepers on other hosts. How would I go about debuging this?
I have now successfully ran the initcluster on my three peers. I then checked logs in etcd.servicer which was the following: https://pastebin.com/5QAqASuG (I think that is OK). Then I checked the ogs of flanneld.service and I kept getting `Nov 29 08:22:36 blockchainIIC flanneld[10209]: E1129 08:22:36.679522 10209 watch.go:43] Watch subnets: delete "/coreos.com": not a subnet, skipping`. I than used `systemctl restart flanneld which resulted in: https://pastebin.com/Mi2rNGwU (Looks OK, but I shoudldn't do that manually). Now I pinged the registarator container on a host that is running it and it was a success but when I have pinged it from another host it was unsuccessfull. That is probably expected, right?
I have tried that because i then ran setupfabric and zookeeper kept getting connection timeout errors when trying to connect to zookeepers on other hosts. How would I go about debuging this?
BTW my hosts are 172.16.67.240, 172.16.67.239 and 172.16.67.238 so maybe container-network configs are somehow wrong. My os.yaml file: https://pastebin.com/ZtZk0EBM
@tongli
@SimonOberzan hi.
Hi
@SimonOberzan you pinged me earlier, do you need something?
Yeah the message above explained my problem :)
Yeah the message above explains my problem :)
@SimonOberzan I think you run these apply command multiple times without cleaning up appropriately.
I ran ansible-playbook -e "mode=destroy env=bc1st deploy_type=compose" setupfabric.yml -K, befor running it again. Should I also run `ansible-playbook -e "mode=destroy env=os password=XXXXX cloud_type=os" provcluster.yml` but then can I just omit password parameter?
I ran `ansible-playbook -e "mode=destroy env=bc1st deploy_type=compose" setupfabric.yml -K`, before running it again. Should I also run `ansible-playbook -e "mode=destroy env=os password=XXXXX cloud_type=os" provcluster.yml` but then can I just omit password parameter?
each time you deploy things on, you will need to destroy it before you run it again especially when you change something in your configuration file.
since get it up running does not take long time, I would strongly destroy everything and start over.
provcluster.yml is to get things completely destroyed on your cloud.
including get rid of your VMs. I assume that you are using AWS?
No I do not deploy to cloud
I just have a set of Ubuntu hosts
You said earlier that I can use env=os for initcluster but now when I tried destroy provcluster I got an ansible FAIL: "shade is required for this module"
You said earlier that I can use env=os for initcluster but now when I tried destroy provcluster I got an ansible FAIL: "shade is required for this module" when destoying OpenStack VM
You said yesterday that I can use env=os for initcluster but now when I tried destroy provcluster I got an ansible FAIL: "shade is required for this module" when destoying OpenStack VM
You said yesterday that I can just use env=os for initcluster but now when I tried destroy provcluster I got an ansible FAIL: "shade is required for this module" when destoying OpenStack VM
@tongli Regarding issue https://jira.hyperledger.org/browse/CE-173 I also face same issue everytime when ran ansible cluster with aws instances. Reason I found is that icmp packets donot pass from flannel interface to docker interface thus causing to break connection between dockers. This happens due to ipatbles settings. If you edit iptables setting with `sud iptables -P FORWARD ACCEPT` it will fix the issue and all container then communicate to each other.
@tongli Regarding issue https://jira.hyperledger.org/browse/CE-173 I also face same issue everytime when ran ansible cluster with aws instances. Reason I found is that icmp packets donot pass from flannel interface to docker interface thus causing to break connection between dockers. This happens due to ipatbles settings. If you edit iptables setting with `sudo iptables -P FORWARD ACCEPT` it will fix the issue and all container then communicate to each other.
@tongli Regarding issue https://jira.hyperledger.org/browse/CE-173 I also face same issue everytime when ran ansible cluster with aws instances. Reason I found is that icmp packets donot pass from flannel interface to docker interface thus causing to break connection between dockers. This happens due to ipatbles settings. If you edit iptables setting with `sudo iptables -P FORWARD ACCEPT` it will fix the issue and all container then communicate to each other.
@SimonOberzan right, in that can you can not use provcluster.yml , that is for destroy VMs.
@MohammadObaid right, if you actually use aws provisioning modules , all these will be taken care of.
@tongli So how would I go about destroying everything, so I can deploy things on again?
when you manually setup your own VM, you may have missed some thing.
@SimonOberzan what are you using and how did you get your cluster up running?
if you are using AWS, you should simply use the steps that setup whole thing which including set up security rules on aws.
if you manually create VMs, then start from initcluster and setup fabric, things can get messy if you do not really know what you are doing.
I manually edited hosts file, than ran `ansible-playbook -i hosts -e "mode=apply env=os env_type=flanneld" initcluster.yml -K -vv` and than `ansible-playbook -i hosts -e "mode=apply env=bc1st deploy_type=compose" setupfabric.yml -K`.
@SimonOberzan right, so you may have missed steps setting up security rules on aws. which by default do not open many ports for things to work.
I do not use aws
@SimonOberzan ok, I thought you were using aws.
@tongli by aws provisioning modules you mean running ansible playbook `ansible-playbook -e "mode=apply" aws.yml` or I should run some other commands along with that playbook command
@SimonOberzan if you are using a clean ubuntu 16.04 image, then you should not have iptable issues.
unless your existing server already have some firewall settings?
@MohammadObaid right.
if you like to destroy on aws, doing this
```ansible-playbook -e "mode=destroy" aws.yml```
of course, in that case, you will have to change few things in vars/aws.yml file to make sure that few fields are matching your account stuff.
like what the doc suggested.
I was going to update CE-173, but you are chatting here about the same topic. Mohammad and I see the same issue.
@tongli Well I always ran same ansible playbook script and face same issue everytime . I have to manually ssh into servers and edit iptables setting. If I dont do that then containers couldnt communicate
exactly
@sdarwin are you also using ansible agent?
ansible-playbook -e "mode=apply" aws.yml
the standard playbook
that sounds good.
@sdarwin you are having same issue on iptables?
right. the solution appears to be "iptables -P FORWARD ACCEPT"
what aws ubuntu image are you using?
16.04
the image name.
@tongli Yeah the hosts are in an internal network
it is the image from the ansible code. Not modified.
oh. i c.
The Docker codebase, fairly recently, began to set iptables -P FORWARD DROP
and that has affected kubernetes, and other projects
Still I need to know how to "destroy" everything? Is it already supported for cases like mine or should I write a new env file?
@SimonOberzan you just need to run destroy for initcluster and setupfabric in reverse order.
it should be already there. I do not think you need to add anything assume that you know which env file and fabric layout file to use.
@tongli I use os.yml file even though I do not deploy to OpenStack
I recently moved a doc here. https://github.com/hyperledger/cello/blob/master/docs/ansible-agent-how-to.md
You told me to do so yesterday
I followed that doc
@SimonOberzan right, you will need to use that, that file gets used for provisioning and initcluster
Destroy for setupfabric works fine , but destroy for initcluster fails
please read that doc, it can provide you some insight.
I read it all
@SimonOberzan oh, great.
Its just that there are just vb/aws/os/azure envs and I dont use any
That is the destoy initcluster error:
```TASK [Destroy the OpenStack VM] ******************************************************************************************************************************************************************************************************
fatal: [fabric001]: FAILED! => {"changed": false, "failed": true, "msg": "shade is required for this module"}```
right. these files in vars directory are supposed to be like template or example configurations. different env should have similar files but same format and variable names.
@SimonOberzan that should only be needed when you deal with OpenStack cloud in provcluster
if you never invoke provcluster, then you should not need it.
if you do, then you need it.
I assume that you are not doing all this on OpenStack cloud, then you should not run this command
```ansible-playbook -i runhosts -e "mode=appy env=os" provcluster.yml```
or this
```ansible-playbook -i runhosts -e "mode=destroy env=os" provcluster.yml
apply not appy above.
when you have existing servers , you only need to do these two steps.
That fails at gathering facts
```ansible-playbook -i runhosts -e "mode=apply env=xx" initcluster.yml```
```ansible-playbook -i runhosts -e "mode=apply xxxx" setupfabric.yml
Still same error
@SimonOberzan I do not know why it could invoke these modules though since the initcluster simply uses ssh to install software and never calls any openstack modules.
At destroy I mean
right.
understand.
so can you install shade and see if the error goes away?
Still that one:
```TASK [Destroy the OpenStack VM] ******************************************************************************************************************************************************************************************************
fatal: [fabric001]: FAILED! => {"changed": false, "failed": true, "msg": "shade is required for this module"}```
put your exact command here
`ansible-playbook -e "mode=destroy env=os" provcluster.yml` That one?
sigh! I assume that you are not using openstack cloud, right?
No
then why are you running that command?
you should be running these two commands
I am asking you how to destroy everything and you provided me that command
```ansible-playbook -e "mode=destroy env=os" initcluster.yml```
provcluster is to provision vms against a cloud.
initcluster is to initialize the machines.
setupfabric is to stand up fabric network.
which clearly says in the doc.
https://github.com/hyperledger/cello/blob/master/docs/ansible-agent-how-to.md#provision-the-servers
That ran successfully so everything is destroyed now?
https://github.com/hyperledger/cello/blob/master/docs/ansible-agent-how-to.md#initialize-and-prepare-the-servers
it should.
Yeah you said that I probably didn't destroy everything and I told you I did run this
And than I asked you if I should also run destroy init :)
that should do the trick I think.
if it does not, patches will be very welcome.
Yeah but I tried that before I wrote you the message so the problem isn't there
Sure, I'm just asking you how to debug connectivity between hosts as there are connection timeouts between zookeepers on different hosts
if containers can not see each other like fails ping to each other, most likely the problems are flanneld overlay network gets messed up.
it could be few causes to that.
1. etcd data was not cleaned appropriately, since flanneld depends on etcd to work correctly.
2. dns container stopped running.
3. registrator containers which should be one on every node stopped running.
In the first message I have linked logs of etcd and flanneld services can you please look if something looks suspicious?
any of these issues can cause containers not to be able to communicate.
flanneld depends on iptables since it heavily manipulate iptables rules, so messing with iptables can cause issues very hard to figure out.
I saw that, and the msg says /coreos is not a subnet, that basically tells me that etcd database is messed up for some reason, you should not see that kind of error.
that is why I suggested to destroy and clean up things. also the issues with existing servers is that ansible agent has no idea what is on these servers.
That problem was solved by restarting flanneld.service, and once it worked even without restarting but the problem was the same.
I'll try to deploy the project to a clean set of VMs.
sure. when you found some problems, please try to figure out the problems and submit patches.
it will be way better than just me doing all these things.
I mean try to figure out solutions and submit patches. this is an open source project after all.
Has left the channel.
Has joined the channel.
@MohammadObaid I have submitted this patch to fix the ip forwarding issue. please take a look https://gerrit.hyperledger.org/r/#/c/15821/
@baohua @hightall can you look at this patch which is to fix the CE-173 issue. https://gerrit.hyperledger.org/r/#/c/15821
What is the 'blue point'? I found that at 'Cello Dashboard -> User Management -> Create User'. In source code, It seems to related with session. But I don't know exactly. Tell me anyone T.T
@tongli merged, thanks!
@miiiiiyoung oh, we haven't finished that part. will update soon
@tongli Yup I applied your patch in my project and it solved that isue .Thanks :)
Thank you for the fix.
and what if the server happens to be rebooted..? :-)
Just to keep in mind. Docker will launch. Docker will set the rule back to DROP.
@tongli @baohua @hightall Request for review https://gerrit.hyperledger.org/r/#/c/15807/
@sdarwin currently cello does not manage these containers after them started. Machine reboot is one area we need a bit help.
@sdarwin this also needs to be considered in two scenarios, one is the docker swarm like env (what you probably have), we use flanneld to setup overlay network (which is fine since it is a systemd service), and k8s env.
@sdarwin are you saying each docker service restart causes iptable rules to be changed?
@sdarwin I am not quite sure that is the case, can you confirm and see if there is a flag or something to change that behavior?
@tongli I'd definately submit a patch if I find a bug. Thank you for your time.
I have now deployed the project on a new set of clean ubuntu VMs. Initcluster now runs without any suspicious logs (http://prntscr.com/hhai9s). Setupfabric also runs without an error but there seems to be no connectivity betweenen containers of different hosts. I suspect that I simply didn't configure the network configs right, but I just can't make it work.
My VMs are in a VLAN `172.16.64.0/20`, so I set my netmasks to netmask `255.255.240.0`. The addresses of the hosts are `172.16.67.237`, `172.16.67.246`, `172.16.67.247`.
My os.yml: https://pastebin.com/kTZVXpaf . Please look at the container_network here to see if anything looks off. Also I have not modified service_ip_range or dns_service_ip, as I don't know what to set it to.
hmmm.
@SimonOberzan your VM IP segment duplicate with overlay network.
your VM or physical machine should not use IP in that segment.
172.16.0.0/16 gets used by the flanneld overlay network for containers.
What changes do you propose? I'm not very good at networking
your machines have to use something different or you change that segement to something else
https://github.com/hyperledger/cello/blob/master/src/agent/ansible/vars/os.yml#L29
that says container network.
your VM should not use any IP fall into that range.
@tongli @hightall @baohua @SriramaSharma https://gerrit.hyperledger.org/r/#/c/15827/ Request for review
@tongli You mean IPs of my host shouldn't start with 172.16.X.X?
correct.
how many machines do you have? how do they communicate with each other?
normally you should not need to change anything about the network that you connect your VMs.
@harsha the patch failed verification.
```15:05:09 make[2]: Entering directory '/w/workspace/cello-verify-x86_64/test'
15:05:09 docker-compose up --abort-on-container-exit
15:05:11 Creating network "dashboardtest_default" with the default driver
15:05:17 Building dashboard
15:05:18 Step 1/8 : FROM hyperledger/cello-baseimage
15:05:19 Service 'dashboard' failed to build: pull access denied for hyperledger/cello-baseimage, repository does not exist or may require 'docker login'
```
Looks like I need to make the target `make start` dependent on `docker` target
seems require the hyperledger/cello-baseimage
either we need to build that image locally, or need to put it to the docker hub to let ci pull it down
the are in a network 172.16.64.0/20, they have the IPs stated above and they can connect between each other directly with those IPs? I have 3 VMs. Sorry if I didn't answer your question but I'm not good at networking
@tongli they are in a network 172.16.64.0/20, they have the IPs stated above and they can connect between each other directly with those IPs? I have 3 VMs. Sorry if I didn't answer your question but I'm not good at networking
@harsha would suggest to build the image locally
Hi guys.
I'm new to Cello project and Ansible. Currently running a Virtual Box Ubuntu 17.04 and trying to deploy the ansible-agent-all-in-one ( https://github.com/hyperledger/cello/blob/master/docs/ansible-agent-all-in-one.md )
It's a fresh virual machine, jsut installed ssh and what the tutorial asks for but i keep on having an error:
fatal: [10.0.2.15]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Permission denied (publickey,password).\r\n", "unreachable": true}
Hi guys.
I'm new to Cello project and Ansible. Currently running a Virtual Box Ubuntu 17.04 and trying to deploy the ansible-agent-all-in-one ( https://github.com/hyperledger/cello/blob/master/docs/ansible-agent-all-in-one.md )
It's a fresh virual machine, jsut installed ssh and what the tutorial asks for but i keep on having an error:
fatal: [10.0.2.15]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Permission denied (publickey,password).\r\n", "unreachable": true}
I can ssh outside ansible
@smallant use -k with your ansible command? Or push a public key to the host.
You can do it with `ssh-copy-id -i ~/.ssh/id_rsa.pub blockchain@
You can do it with `ssh-copy-id -i ~/.ssh/id_rsa.pub
You can do it with `ssh-copy-id -i ~/.ssh
You can do it with `ssh-copy-id -i ~/.ssh/
@tongli , that issue was not from restarting the Docker service. However, a full machine reboot would trigger it. The very first time Docker runs.
@sdarwin I thought that the iptable command set to accept, just need to be done once, if machine reboots, docker services runs again reset that to drop as you indicated, then that is a big problem.
[ ](https://chat.hyperledger.org/channel/cello?msg=DGZ9jfj4ndwAmMM4f) @SimonOberzan thanks I'll try that :)
@tongli , the most elegant solution could be systemd drop-in units to customize Docker.
you mean each time after docker service starts, run that command?
that command will be part of docker systemd service?
ExecStartPost command
that can be done, actually cello create docker systemd service already. but I am still skeptical about that has to be done each time.
it doesn't have to be done each time.
only at boot
however, running it every time probably doesn't hurt anything.
[ ](https://chat.hyperledger.org/channel/cello?msg=KTxiYG3h3Ks28cZ6s) Also could I just change `172.16.0.0/16` in network_config to `172.17.0.0/16`?
@SimonOberzan sure, you can change the overlay to somewhere else which does not conflict with your VM IP space.
@SimonOberzan you would need to change the SubnetMin, SubnetMax and Network.
Ok, I'll try that, thank you
Has joined the channel.
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/15807/
@tongli @hightall @baohua https://gerrit.hyperledger.org/r/#/c/15827/ -- CI has finally passed now. Request for review.
Has joined the channel.
@baohua artifacts and crypto-config should already generated before using cello ?
yes, now we pre-include some generated one. certainly people can generate there own
Clipboard - 2017年12月1日上午11点19分
there is nothing
in the source code, it's at src/agent/docker/_compose_files/fabric-1.0/local/channel-artifacts
thanks,maybe something wrong in setup
@tongli It works now! I have used ```container_network: {
Network: "10.1.0.0/16",
SubnetLen: 24,
SubnetMin: "10.1.0.0",
SubnetMax: "10.1.255.0",
Backend: {
Type: "udp",
Port: 8285
}
}``` configs and it works but it somehow didn't work when I set it to `172.17.00` but I'll take it :)
Thank you for your help
I have made my first commit to an OS project and looks like I did a mess :sweat_smile: . I guess my first commit failed because I have also changed the .env file by mistake? Then I tried to push a fix but I guess I didn't do it right. What would be the right actions now, to push a fix? Also if I made any other mistakes (like I didn't include jira tag at the start of commit msg) please tell me so my future commits will be like they should.
@SimonOberzan happy for you.
@SimonOberzan you have to submit a patch set to gerrit for review. once maintainer approves your patch set, then your changes will be merged.
@SimonOberzan Not sure how you submitted your changes, I am sure it caused no harm at present.
On gerrit there is a subject column. Each subject has a history of commits. If I want to push a new fix can I push it in a history of an exsisting subject? Or does each subject only show commits made before pushing with git review?
@SimonOberzan FWIW if you are looking to submit a PR to a specific a project, would suggest you to go through
https://hyperledger-fabric.readthedocs.io/en/release/Gerrit/gerrit.html
Once you push your PR it will come up on top of the commit levels.
Thank you, I'll look into that.
Well I it's verified now :)
Well I it's verified now: https://gerrit.hyperledger.org/r/#/c/15865/ :)
Well, I it's verified now: https://gerrit.hyperledger.org/r/#/c/15865/ :)
Firewall Setup is must to be done on the worker node?
```Docker daemon setup
Docker images pulling
Firewall Setup```,here are three steps on worker node installation.
Yes, in case sometime the firewall reject the connection , need to open the acl
About ce-184
@harsha can u contact ry jones
to see how we can have the Docker hub images auto-built following the repo code commit? Thanks!
Has joined the channel.
I plan to install cello,but only the manager node and word node(docker),how to config and run the fabric env?
@baohua @hightall please review this patch to set default fabric version to 1.0.4 https://gerrit.hyperledger.org/r/#/c/15889/
Added comments, thanks!
@tongli @hightall as planed, we will release the v0.8.0 version around the end of Dec. pls help report whether there are big issues to address before the release. Thanks!
Has joined the channel.
Has joined the channel.
how can i use cello to monitor my fabric?
anyone can help me to answer my question?
Hi guys, in installing cello master node on an ubuntu 16.04 i faced the following issue in `make master-setup`
```
ERROR: Service 'user-dashboard' failed to build: The command '/bin/sh -c mv /usr/app/src/js /reactjs && cd /reactjs && npm install && npm run build && rm -rf node_modules && cd /reactjs/home && npm install && npm run build && rm -rf node_modules && cd /usr/app/src && npm install && npm run build && npm cache clean --force' returned a non-zero code: 1
Makefile:168: recipe for target 'start' failed
make: *** [start] Error 1
```
Hi guys, in installing cello master node on an ubuntu 16.04 i faced the following issue in `THEME=react make start`
```
ERROR: Service 'user-dashboard' failed to build: The command '/bin/sh -c mv /usr/app/src/js /reactjs && cd /reactjs && npm install && npm run build && rm -rf node_modules && cd /reactjs/home && npm install && npm run build && rm -rf node_modules && cd /usr/app/src && npm install && npm run build && npm cache clean --force' returned a non-zero code: 1
Makefile:168: recipe for target 'start' failed
make: *** [start] Error 1
```
it seems there's a simple script typo in the `mv` command, where may i fix this?
i just want to confirm that `THEME=vue make start` works, react does not.
@yipcma Does your cello-services eventually comes up when you select `THEME=react` ?
no, it just failed, vue works tho.
@harsha right now am facing an issue of inactive hosts and there's nothing i can do about it (delete host , stop chain) etc. what should do in this case?
Can you elaborate it please, maybe provide a snap/snippet of the same ?
Screen Shot 2017-12-04 at 1.09.12 PM.png
also the worker keeps restarting
Screen Shot 2017-12-04 at 1.24.48 PM.png
@yipcma Does it now allow you to delete from `Actions -- Delete` ?
@yipcma Does it allow you to delete from `Actions -- Delete` ?
nope it's greyed-out
and if i try clean it will pop fail bubble
i rebooted the machine and those dockers will still run at startup and keep restarting, host status still inactive
That's bcoz of the restart policy on the containers.
two things i'd like to know: 1. what does it mean by host inactive and what causes that.
2. how to set things right. (and wether the state/chain is persisted)
2. how to set things right. (and whether the state/chain is persisted)
FWIW, I am figuring out things too with cello ;) so bear with me..
hahaha ;) let's do it together
BTW, can you reset from `Actions --> Reset`
so what i just did: i stopped and removed them all.
nope it doens't work
now i rebooted, no more dockers auto start.
Now all set ?
no
they don't start now. they should no?
i'm going to do "make start" on the master and see if i can reproduce up to active host.
`make restart`
Or, on the client, `service docker stop` and check the node status on your master
now it's totally insane, same old inactive host and active running chain.
but there're no corresponding dockers running.
Can you kill/remove your running chain ?
nope
those dockers don't exist anymore, but the master cannot realize that
Did you issue `Action-->Reset` ?
Did you issue `Action-->Reset` ? which actually cleans-up stale docker-containers
reset is greyed out
you happen to know how i can reset the maste node? it remembers things that are long gone
Try `make restart` else `make redeploy`
tried `make restart` `make redeploy` none work, it still remembers...
now i got this ```Pulling mongo (hyperledger/cello-mongo:latest)...
ERROR: pull access denied for hyperledger/cello-mongo, repository does not exist or may require 'docker login'
Makefile:168: recipe for target 'start' failed
make: *** [start] Error 1```
What exactly are you trying to do ?
`make start`
yes
it didn't have that problem before, now this occurs
assuming you are under `cello/`
can you `rm -rf build`
and issue `make start`
thanks buddy very help, now let's see....
i tried `make clean` surprised that it doesn't clear /build
Target clean isn't clearing that.. Would request you to open a bug/jira item.
https://gerrit.hyperledger.org/r/#/c/15899/ - Request for review
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=KwEfCq3ntaebs6pWw) @Luke_Chen And it is compatible with the k8s v1.8?
Hi anyone got the restarting peer problem in creating a chain?
ok, the artificats are not properly copied, my artifact path is `/opt/cello` but the subfolder under `_compose_files` is `local`
ok, the artifacts are not properly copied, my artifact path is `/opt/cello` but the subfolder under `_compose_files` is `local`
just to update on the restarting problem, it is a folder copy and chown issue. I'm testing the master node and the worker node on the same computer. need to make sure `/opt/cello/mongo` is owned by root and `/opt/cello/fabric 1.0` is owned by USER:USER
Has joined the channel.
Hi All
need one help, I am setting up fabric network on aws using ansible cello
on multiple servers
and faced with the below issue
TASK [Start the flanneld service] ***************************************************************************************************************************
fatal: [172.31.63.87]: FAILED! => {"changed": false, "msg": "Unable to start service flanneld: Job for flanneld.service failed because a timeout was exceeded. See \"systemctl status flanneld.service\" and \"journalctl -xe\" for details.\n"}
fatal: [172.31.59.62]: FAILED! => {"changed": false, "msg": "Unable to start service flanneld: Job for flanneld.service failed because a timeout was exceeded. See \"systemctl status flanneld.service\" and \"journalctl -xe\" for details.\n"}
does any one knows what is this issue and how to fix it?
@ambatigaan yes, it is compatible
@yipcma yes, i guess we mentioned in the doc? If not, would welcome u to help commit a pr, thanks!
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/15899/
https://gerrit.hyperledger.org/r/#/c/15947/
https://gerrit.hyperledger.org/r/#/c/15807/
https://gerrit.hyperledger.org/r/#/c/15899/ -- Needs Review
https://gerrit.hyperledger.org/r/#/c/15947/ -- Needs one more +2
https://gerrit.hyperledger.org/r/#/c/15807/ -- Needs one more +2
https://gerrit.hyperledger.org/r/#/c/15899/ -- Needs one more +2
https://gerrit.hyperledger.org/r/#/c/15947/ -- Needs one more +2
https://gerrit.hyperledger.org/r/#/c/15807/ -- Needs one more +2
https://gerrit.hyperledger.org/r/#/c/15899/ -- Needs one more +2
https://gerrit.hyperledger.org/r/#/c/15947/ -- Needs one more +2
~https://gerrit.hyperledger.org/r/#/c/15807/ ~
https://gerrit.hyperledger.org/r/#/c/15963/ -- Needs Review
https://gerrit.hyperledger.org/r/#/c/15947/ -- Updated & Rebased, Needs review.
https://gerrit.hyperledger.org/r/#/c/15899/ -- Updated & Rebased, Needs review.
:)
Added my review, thanks
@tongli is presenting cello at lisbon hackfest! would like to hear more feedbacks there, good to instruct how we develop cello better for more user cases.
@baohua I have gone through this document https://docs.google.com/document/d/1eRNxxQ0P8yp4Wh__Vi6ddaN_vhN2RQHP-IruHNUwyhc/edit# and saw a section `terminate channel` feature . Is this feature available or just in theory for now ?
I havent seen any implementation example on offical docs or in node sdk
no, the channel cannot be terminated now
Hmm , means it is currently in development mode and we might see in future right ?
quick question on the peers generated from adding chains in cello. it seems that i cannot connect to them. when i look at `docker ps` the ports don't seem to be forwarded. thoughts on this or ways to mitigate?
also another query: i found 4 _sk files in the keystore under the artifacts, am curious why. in the fabric BYFN example there's only one.
@yipcma it should be only one _sk file that correspond to the public key, others could be removed.
@Luke_Chen guess i'll just rerun to cryptogen to get one then.
Has joined the channel.
Has joined the channel.
Has joined the channel.
v0.8.0 Release preparation: https://jira.hyperledger.org/browse/CE-188, @hightall @tongli
Has joined the channel.
Hi All
I am trying to create a fabric network using Ansible cello
in aws cloud
and running into issues
I ran the below command for initializing and got below errors the rest are successful:
Before I used private_ip and my process is stuck at Start the Flanneld service and throws the below first error, today I changed to Public_ip in the yml file and it still throws the first error but in went quickly to other steps and throwed the remaining errors as shown below, can you please help?
$ ansible-playbook -i run/runhosts -e "mode=apply env=aws env_type=flanneld" initcluster.yml
TASK [Start the flanneld service] ***************************************************************************************************************************
changed: [54.145.240.93]
fatal: [54.85.48.22]: FAILED! => {"changed": false, "msg": "Unable to start service flanneld: Job for flanneld.service failed because a timeout was exceeded. See \"systemctl status flanneld.service\" and \"journalctl -xe\" for details.\n"}
fatal: [35.168.7.149]: FAILED! => {"changed": false, "msg": "Unable to start service flanneld: Job for flanneld.service failed because a timeout was exceeded. See \"systemctl status flanneld.service\" and \"journalctl -xe\" for details.\n"}
TASK [Start registrator service] ****************************************************************************************************************************
fatal: [54.145.240.93]: FAILED! => {"changed": true, "cmd": ["docker", "run", "-d", "--name", "registrator.fabric003", "-v", "/var/run/docker.sock:/tmp/docker.sock", "email4tong/registrator:v10", "-internal", "-ttl", "30", "-ttl-refresh", "20", "-resync", "18", "skydns2://172.31.56.228:2379/fabricnet"], "delta": "0:00:15.019447", "end": "2017-12-08 17:18:04.950998", "msg": "non-zero return code", "rc": 125, "start": "2017-12-08 17:17:49.931551", "stderr": "Unable to find image 'email4tong/registrator:v10' locally\ndocker: Error response from daemon: Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers).\nSee 'docker run --help'.", "stderr_lines": ["Unable to find image 'email4tong/registrator:v10' locally", "docker: Error response from daemon: Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers).", "See 'docker run --help'."], "stdout": "", "stdout_lines": []}
PLAY RECAP **************************************************************************************************************************************************
35.168.7.149 : ok=34 changed=19 unreachable=0 failed=1
54.145.240.93 : ok=52 changed=27 unreachable=0 failed=1
54.85.48.22 : ok=36 changed=19 unreachable=0 failed=1
cloud : ok=4 changed=0 unreachable=0 failed=0
any help in this will be highly appreciated.
@sarifuddin you have already running ubuntu servers on aws?
@sarifuddin you have already running ubuntu servers on aws?
Have you tried with this command `ansible-playbook -e "mode=apply" aws.yml` ?
Have you tried with default command `ansible-playbook -e "mode=apply" aws.yml` ?
@sarifuddin I am in a meeting now, will look into your issue after.
I have configured the yml file for aws details
and then ran first the above single integrated command : ansible-playbook -e "mode=apply" aws.yml`
it ran into issue at same step
then I try to run each phase separately
@sarifuddin the problem that flanneled service can not start?
yes
did you create new VM? or are you reusing already started VMs?
the script created new VM's in AWS based on the aws.yml configuration file
I have customized according to my account
and then ran the provisioning command
and it completed successfully with 3 nodes up and running
so total 4 nodes: 3 nodes for the fabric and 1 ansible controller
i ran the command from Ansible controller host
ah! that is what I normally do.
which then provision the remaining 3 hosts
have 1 VM as ansible controller, so that you can use private_ip.
@sarifuddin so there was no error at all before you saw flanneld service errors?
yep, there was no error before flanneld
flanneld service is the first error
hmmm. flanneld rely on etcd service to be running correctly.
sometimes when you run ansible repeatedly especially in cloud env, it gets a bit confused if you do not config ssh correctly.
can you post your config file in ansible controller .ssh directory?
if you do not have one, please create one like this.
TASK [Reload services] **************************************************************************************************************************************
changed: [54.85.48.22]
changed: [54.145.240.93]
changed: [35.168.7.149]
TASK [Enable and start etcd services] ***********************************************************************************************************************
changed: [54.145.240.93]
changed: [35.168.7.149]
changed: [54.85.48.22]
PLAY [setup flanned on all nodes] ***************************************************************************************************************************
TASK [Gathering Facts] **************************************************************************************************************************************
ok: [54.85.48.22]
ok: [35.168.7.149]
ok: [54.145.240.93]
```Host *
StrictHostKeyChecking no
UserKnownHostsFile /dev/null
TCPKeepAlive yes
ServerAliveInterval 60```
what should be the name of this configuration file?
config
just config or config.sh
just config
it is not a shell file.
also remove known_hosts file , you should have one like that.
sure
after you have all that in place, please destroy everything, let's run again and see step by step what is going on.
ok
in aws.yml file what option should I give
node_ip: "public_ip",
last time i put public_ip
instead of private ip
in your configuration, you can use either.
since your ansible controller and new VMs are on the same network.
ok
you should use public_ip when your ansible controller and VMs are not on the same network. such as using your laptop as ansible controller.
ok
in aws.yml, if I set availability zone to a specific one, will it take that or it will take random
with in a region
i try setting a specific availability zone to match same as my ansible controller
but its picking different availability zone
set the availibility zone to empty
region needs to match your region, like I am in east coastal, I use N. Virginia
yaa, its matching
if you have run this before and VMs were created successfully, these settings should be fine.
Hey @tongli does ansible script work in kubernetes environment ?
@MohammadObaid it sets up k8s env, then deploy fabric onto it.
Oh . Just one more question . If one of my aws server goes down completely then I have to create flannel interface manually and add other flannel interface ip right ?
no, if one server goes down, your fabric network should continue to work.
are you talking about recovering the system that goes down?
or are you talking about adding a brand new node?
@sarifuddin how is it going?
server provisioning completed
i will now run the initialization
ansible-playbook -i run/runhosts -e "mode=apply env=aws env_type=flanneld" initcluster.yml
right now this is how its going
TASK [Download flannel package] *****************************************************************************************************************************
changed: [172.31.62.14]
changed: [172.31.51.53]
changed: [172.31.53.246]
TASK [Unpack flannel binaries] ******************************************************************************************************************************
changed: [172.31.53.246]
changed: [172.31.62.14]
changed: [172.31.51.53]
TASK [Configure flanneld services] **************************************************************************************************************************
changed: [172.31.62.14]
changed: [172.31.53.246]
changed: [172.31.51.53]
TASK [Change IP Forwarding] *********************************************************************************************************************************
changed: [172.31.53.246]
changed: [172.31.62.14]
changed: [172.31.51.53]
TASK [Reload services] **************************************************************************************************************************************
changed: [172.31.62.14]
changed: [172.31.53.246]
changed: [172.31.51.53]
TASK [Start the flanneld service] ***************************************************************************************************************************
its now trying to start flanneld service
staying there
getting error:
TASK [Start the flanneld service] ***************************************************************************************************************************
fatal: [172.31.62.14]: FAILED! => {"changed": false, "msg": "Unable to start service flanneld: Job for flanneld.service failed because a timeout was exceeded. See \"systemctl status flanneld.service\" and \"journalctl -xe\" for details.\n"}
fatal: [172.31.51.53]: FAILED! => {"changed": false, "msg": "Unable to start service flanneld: Job for flanneld.service failed because a timeout was exceeded. See \"systemctl status flanneld.service\" and \"journalctl -xe\" for details.\n"}
I ran the command $ systemctl status flanneld.service and it throws the below error:
ubuntu@ip-172-31-11-179:~$ systemctl status flanneld.service
● flanneld.service - Fabric Flanneld Service
Loaded: loaded (/lib/systemd/system/flanneld.service; enabled; vendor preset: enabled)
Active: activating (start) since Fri 2017-12-08 20:02:17 UTC; 1min 20s ago
Main PID: 4392 (flanneld)
Tasks: 9
Memory: 7.3M
CPU: 60ms
CGroup: /system.slice/flanneld.service
└─4392 /opt/fabric/bin/flanneld
Dec 08 20:03:28 ip-172-31-11-179 flanneld[4392]: E1208 20:03:28.439146 4392 network.go:102] failed to retrieve network config: 100: Key not found (/coreos
Dec 08 20:03:29 ip-172-31-11-179 flanneld[4392]: E1208 20:03:29.440059 4392 network.go:102] failed to retrieve network config: 100: Key not found (/coreos
Dec 08 20:03:30 ip-172-31-11-179 flanneld[4392]: E1208 20:03:30.441019 4392 network.go:102] failed to retrieve network config: 100: Key not found (/coreos
Dec 08 20:03:31 ip-172-31-11-179 flanneld[4392]: E1208 20:03:31.441935 4392 network.go:102] failed to retrieve network config: 100: Key not found (/coreos
Dec 08 20:03:32 ip-172-31-11-179 flanneld[4392]: E1208 20:03:32.442858 4392 network.go:102] failed to retrieve network config: 100: Key not found (/coreos
Dec 08 20:03:33 ip-172-31-11-179 flanneld[4392]: E1208 20:03:33.443688 4392 network.go:102] failed to retrieve network config: 100: Key not found (/coreos
Dec 08 20:03:34 ip-172-31-11-179 flanneld[4392]: E1208 20:03:34.444626 4392 network.go:102] failed to retrieve network config: 100: Key not found (/coreos
Dec 08 20:03:35 ip-172-31-11-179 flanneld[4392]: E1208 20:03:35.445552 4392 network.go:102] failed to retrieve network config: 100: Key not found (/coreos
Dec 08 20:03:36 ip-172-31-11-179 flanneld[4392]: E1208 20:03:36.446478 4392 network.go:102] failed to retrieve network config: 100: Key not found (/coreos
Dec 08 20:03:37 ip-172-31-11-179 flanneld[4392]: E1208 20:03:37.447408 4392 network.go:102] failed to retrieve network config: 100: Key not found (/coreos
lines 1-20/20 (END)
Can you please let me know if the below setting in aws.yml is correct or not
container_network: {
Network: "172.31.0.0/16",
SubnetLen: 20,
SubnetMin: "172.31.0.0",
SubnetMax: "172.31.15.255",
Backend: {
Type: "udp",
Port: 8285
}
i am using N Virginia Region
@tongli I am talking about recovery of that server.
Has joined the channel.
Has joined the channel.
@sarifuddin We can check few things, I am not sure if your aws VM actually use IP like 172.31.0.0/16.
if your AWS VMs are fall into that IP range, you will have to change your container_network IP segment.
I noticed that you have used 172.31.0.0/16 which is not the default from cello. You are free to change that but you will need to make sure that IP range does not conflict with your AWS VM private IP Address space.
I tried same thing so many times in my AWS account, I had no issues whatsoever setting things up.
welcome to review: https://gerrit.hyperledger.org/r/#/c/16089/
https://gerrit.hyperledger.org/r/16091
Thanks Tong, I have now changed the container_network IP segment to be different than my AWS instance ip and it worked now and able to standup all 3 nodes. Thanks for your help
Now my next task is to Deploy Business Network, for that I need to know the URL of my CA server. I can see there are multiple CA servers created, so how can I find out what is the URL i need to use, can you please guide?
Another question: Since I stopped my AWS instances, when I power on , how can I start the network again, I don't have to run the same procedures right, its already created, I just need to start my fabric, please guide?
https://gerrit.hyperledger.org/r/15899
https://gerrit.hyperledger.org/r/15947
welcome everyone to help review
@sarifuddin for CA url, you just need to find the CA container IP address.
All CA containers work at their default port.
Once you stop your AWS Instances, when you power them back up. these containers can not start by themselves, you will have to manually start them up.
unless there are ways to deal with container auto start like using --restart unless-stopped or always.
which we do not do right now. we may need to add that flag as a parameter.
should be easy to do.
https://gerrit.hyperledger.org/r/16101
this should only need to be done by changing these files https://github.com/hyperledger/cello/blob/master/src/agent/ansible/roles/deploy_compose/fabricsetup/templates/peer-compose.j2
all compose files in this directory https://github.com/hyperledger/cello/blob/master/src/agent/ansible/roles/deploy_compose/fabricsetup/templates/
@sarifuddin if you can submit a patch on the container restart policy, I will be happy to review and help.
having a flag for restart policy will be better, deployer can choose to have the restart policy or not.
then in the following, you can simply add a line
```container_name: {{ orderer.name }}
image: hyperledger/fabric-orderer:{{ thetag }}
network_mode: bridge```
```container_name: {{ orderer.name }}
image: hyperledger/fabric-orderer:{{ thetag }}
network_mode: bridge```
like the following
```container_name: {{ orderer.name }}
image: hyperledger/fabric-orderer:{{ thetag }}
restart: {{ fabric.restart}}```
fabric.restart should have been defined in bc1st.yml, bc2nd.yml and vb1st.yml file.
default should have been "no"
hope that helps.
https://gerrit.hyperledger.org/r/#/c/15899/ :- Needs one more +2
https://gerrit.hyperledger.org/r/#/c/15947/ :- Needs one more +2
Hey @tongli if ansible sets kbernetes environment then I should get cluster information using this command right `kubectl cluster-info` right? [ ](https://chat.hyperledger.org/channel/cello?msg=JXZvdeKj2wqzcatZD)
Has joined the channel.
@MohammadObaid sure.
you can access the dashboard at the fabric001:8080/iu
fabric001:8080/ui
replace the fabric001 with your first node IP address.
and all the containers have their service end point as well.
Has joined the channel.
make setup-worker
make: *** No rule to make target 'setup-worker'. Stop.
Im getting an error on the worker node when trying to pull the images
dont worry
worked it out
Hi Baohua, was on vacation until yesterday. About CE-184 responded to your question with my thoughts.
Request you to share your thoughts on the same.
ansible-playbook -i run/runhosts.tpl -e "mode=apply env=os env_type=flanneld" initcluster.yml
PLAY [Get start timestamp] *****************************************************
TASK [setup] *******************************************************************
ok: [cloud]
TASK [set_fact] ****************************************************************
ok: [cloud]
PLAY [fast initialize all the servers] *****************************************
TASK [setup] *******************************************************************
TASK [setup] *******************************************************************
TASK [setup] *******************************************************************
fatal: [35.189.54.223]: UNREACHABLE! => {"changed": false, "msg": "ERROR! SSH encountered an unknown error during the connection. We recommend you re-run the command using -vvvv, which will enable SSH debugging output to help diagnose the issue", "unreachable": true}
fatal: [35.201.16.177]: UNREACHABLE! => {"changed": false, "msg": "ERROR! SSH encountered an unknown error during the connection. We recommend you re-run the command using -vvvv, which will enable SSH debugging output to help diagnose the issue", "unreachable": true}
fatal: [35.201.16.185]: UNREACHABLE! => {"changed": false, "msg": "ERROR! SSH encountered an unknown error during the connection. We recommend you re-run the command using -vvvv, which will enable SSH debugging output to help diagnose the issue", "unreachable": true}
NO MORE HOSTS LEFT *************************************************************
PLAY [Setting up etcd services] ************************************************
PLAY [setup flanned on all nodes] **********************************************
PLAY [setup dns attribute for docker service] **********************************
PLAY [setup dns services and registrator] **************************************
PLAY [Reset connections] *******************************************************
PLAY [inform the installer] ****************************************************
TASK [setup] *******************************************************************
ok: [cloud]
TASK [debug] *******************************************************************
ok: [cloud] => {
"msg": "The work load started at 02:47:40, ended at 02:47:41"
}
PLAY RECAP *********************************************************************
35.189.54.223 : ok=0 changed=0 unreachable=1 failed=0
35.201.16.177 : ok=0 changed=0 unreachable=1 failed=0
35.201.16.185 : ok=0 changed=0 unreachable=1 failed=0
cloud : ok=4 changed=0 unreachable=0 failed=0
I cannot for the life of me work out why I cant use ansible to reach the other vms I have set up
Ive followed the read the docs pretty closely
@harsha thanks, added my comments
Has joined the channel.
Hi folks,
I still got the tox error in the cello directory when typing make check,
make: Circular build/docker/baseimage/.0.8.0-snapshot-c0cefd0 <- build/docker/baseimage/.0.8.0-snapshot-c0cefd0 dependency dropped.
make: Circular build/docker/mongo/.0.8.0-snapshot-c0cefd0 <- build/docker/mongo/.0.8.0-snapshot-c0cefd0 dependency dropped.
make: Circular build/docker/nginx/.0.8.0-snapshot-c0cefd0 <- build/docker/nginx/.0.8.0-snapshot-c0cefd0 dependency dropped.
tox
make: tox: Command not found
I am using Ubuntu 14.04 in AWS
Hi, folks.
I met a few problems when installing and using cello.
1. I still got the tox error when typing 'make check' in the cello directory after installation.
I used the cello committed on Mon Dec (c0cefd002b4943213acff684329ed115e583f438) and doing it on ubuntu 14.04 in AWS.
ubuntu@ip-xxx-xx-xx-x:~/cello$ make check
make: Circular build/docker/baseimage/.0.8.0-snapshot-c0cefd0 <- build/docker/baseimage/.0.8.0-snapshot-c0cefd0 dependency dropped.
make: Circular build/docker/mongo/.0.8.0-snapshot-c0cefd0 <- build/docker/mongo/.0.8.0-snapshot-c0cefd0 dependency dropped.
make: Circular build/docker/nginx/.0.8.0-snapshot-c0cefd0 <- build/docker/nginx/.0.8.0-snapshot-c0cefd0 dependency dropped.
tox
make: tox: Command not found
make: *** [check] Error 127
2. In addition, it seems that mongo doesn't start when to type 'make start'. Here is the docker log of mongo
ubuntu@ip-xxx-xx-xx-x:~/cello$ docker logs -f mongo
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
Could anyone give me comments?
Thanks in advance.
Hi, folks.
I met a few problems when installing and using cello.
1. I still got the tox error when typing 'make check' in the cello directory after installation.
I used the cello committed on Mon Dec (c0cefd002b4943213acff684329ed115e583f438) and doing it on ubuntu 14.04 in AWS.
ubuntu@ip-xxx-xx-xx-x:~/cello$ make check
...
make[1]: Entering directory '/home/ubuntu/cello'
make[2]: Entering directory '/home/ubuntu/cello/test'
docker-compose up --abort-on-container-exit
Creating network "dashboardtest_default" with the default driver
Creating dashboard-test
Creating mongo-test
Attaching to dashboard-test, mongo-test
mongo-test | /usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
mongo-test exited with code 1
Aborting on container exit...
Stopping dashboard-test ... done
Makefile:6: recipe for target 'test' failed
make[2]: *** [test] Error 1
make[2]: Leaving directory '/home/ubuntu/cello/test'
Makefile:124: recipe for target 'test-case' failed
make[1]: *** [test-case] Error 2
make[1]: Leaving directory '/home/ubuntu/cello'
Makefile:119: recipe for target 'check' failed
make: *** [check] Error 2
2. In addition, it seems that mongo doesn't start when to type 'make start'. Here is the docker log of mongo
ubuntu@ip-xxx-xx-xx-x:~/cello$ docker logs -f mongo
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
Could anyone give me comments?
Thanks in advance.
Hi, folks.
I met a few problems when installing and using cello.
1. I still got the tox error when typing 'make check' in the cello directory after installation.
I used the cello committed on Mon Dec (c0cefd002b4943213acff684329ed115e583f438) and doing it on ubuntu 16.04 in AWS.
ubuntu@ip-xxx-xx-xx-x:~/cello$ make check
...
make[1]: Entering directory '/home/ubuntu/cello'
make[2]: Entering directory '/home/ubuntu/cello/test'
docker-compose up --abort-on-container-exit
Creating network "dashboardtest_default" with the default driver
Creating dashboard-test
Creating mongo-test
Attaching to dashboard-test, mongo-test
mongo-test | /usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
mongo-test exited with code 1
Aborting on container exit...
Stopping dashboard-test ... done
Makefile:6: recipe for target 'test' failed
make[2]: *** [test] Error 1
make[2]: Leaving directory '/home/ubuntu/cello/test'
Makefile:124: recipe for target 'test-case' failed
make[1]: *** [test-case] Error 2
make[1]: Leaving directory '/home/ubuntu/cello'
Makefile:119: recipe for target 'check' failed
make: *** [check] Error 2
2. In addition, it seems that mongo doesn't start when to type 'make start'. Here is the docker log of mongo
ubuntu@ip-xxx-xx-xx-x:~/cello$ docker logs -f mongo
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
Could anyone give me comments?
Thanks in advance.
Hi, folks.
I met a few problems when installing a master node
1. When installing cello (make setup-master), I sometimes got this message. Is it okay?
debconf: delaying package configuration, since apt-utils is not installed
2. I still got the tox error when typing 'make check' in the cello directory after installation.
I used the cello committed on Mon Dec (c0cefd002b4943213acff684329ed115e583f438) and doing it on ubuntu 14.04 in AWS.
ubuntu@ip-xxx-xx-xx-x:~/cello$ make check
...
make[1]: Entering directory '/home/ubuntu/cello'
make[2]: Entering directory '/home/ubuntu/cello/test'
docker-compose up --abort-on-container-exit
Creating network "dashboardtest_default" with the default driver
Creating dashboard-test
Creating mongo-test
Attaching to dashboard-test, mongo-test
mongo-test | /usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
mongo-test exited with code 1
Aborting on container exit...
Stopping dashboard-test ... done
Makefile:6: recipe for target 'test' failed
make[2]: *** [test] Error 1
make[2]: Leaving directory '/home/ubuntu/cello/test'
Makefile:124: recipe for target 'test-case' failed
make[1]: *** [test-case] Error 2
make[1]: Leaving directory '/home/ubuntu/cello'
Makefile:119: recipe for target 'check' failed
make: *** [check] Error 2
3. In addition, it seems that mongo doesn't start when to type 'make start'. Here is the docker log of mongo
ubuntu@ip-xxx-xx-xx-x:~/cello$ docker logs -f mongo
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
Could anyone give me comments?
Thanks in advance.
@harsha @baohua Thanks,
I tried what you instructed, but the result was same.
To make it sure, I deleted the old instance in my AWS and created a new one with Ubuntu Server 16.04 LTS (HVM), SSD Volume Type
Then, I log-in as root and installed docker engine and compose.
The corresponding versions are shown below.
$docker -v
Docker version 1.12.6-cs13, build 0ee24d4
$docker-compose --version
docker-compose version 1.12.0, build b31ff33
Then, I cloned the latest cello source code containing the following log
commit c0cefd002b4943213acff684329ed115e583f438
Merge: 8719a9b 7b5f99b
Author: tong li
@KSLee Can you try issuing make start as root instead ?
@KSLee Can you try issuing make start as a root user instead ?
@harsha Thanks for the comment, Harsha. I also repeated the installation and run as root (including installation by ubuntu ID and run by root also), however, I got the same result.
Thanks!
I presume you have issued `make image-clean` and issued `make start` and then you are stuck with above issue ?
I presume you have issued `make image-clean` and issued `make start` and then you are stuck with above issue as root user ?
I presume you have issued `make image-clean` and issued `make start` as root user and then you are stuck with above issue ?
About point 1) it can be ignored.
@KSLee i just run `make check` on my local mac with the latest master branch, and it passed.
can u try with the latest code? `make clean check`
@baohua @harsha Thanks for the comments.
To make it sure, I deleted the old instance in my AWS and created a new one with Ubuntu Server 16.04 LTS (HVM), SSD Volume Type
Then, I log-in as root and installed docker engine and compose.
The corresponding versions are shown below.
$docker -v
Docker version 1.12.6-cs13, build 0ee24d4
$docker-compose --version
docker-compose version 1.12.0, build b31ff33
Then, I cloned the latest cello source code containing the following log
commit c0cefd002b4943213acff684329ed115e583f438
Merge: 8719a9b 7b5f99b
Author: tong li
i repeat again, but still not find the problem....let me think
can u run `docker images`?
@harsha when building images, there's msg like "make: Circular build/docker/nginx/.0.8.0-snapshot-c0cefd0 <- build/docker/nginx/.0.8.0-snapshot-c0cefd0 dependency dropped."
is this normal as `Circular build`?
@KSLee after starting, should have containers like
```
```CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
db110e0f6467 hyperledger/cello-restserver "python restserver.py" 2 minutes ago Up 2 minutes 80/tcp restserver
f63e2c63e3cd hyperledger/cello-nginx "/bin/bash /tmp/do..." 2 minutes ago Up 2 minutes 0.0.0.0:80->80/tcp, 0.0.0.0:8080->8080/tcp nginx
5dca464af0fd hyperledger/cello-mongo "docker-entrypoint..." 2 minutes ago Up 2 minutes 127.0.0.1:27017-27018->27017-27018/tcp mongo
b18900e3054e hyperledger/cello-dashboard "python dashboard.py" 2 minutes ago Up 2 minutes 8080/tcp dashboard
a2e31e9422f2 hyperledger/cello-watchdog "python watchdog.py" 2 minutes ago Up 2 minutes watchdog
```
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
db110e0f6467 hyperledger/cello-restserver "python restserver.py" 2 minutes ago Up 2 minutes 80/tcp restserver
f63e2c63e3cd hyperledger/cello-nginx "/bin/bash /tmp/do..." 2 minutes ago Up 2 minutes 0.0.0.0:80->80/tcp, 0.0.0.0:8080->8080/tcp nginx
5dca464af0fd hyperledger/cello-mongo "docker-entrypoint..." 2 minutes ago Up 2 minutes 127.0.0.1:27017-27018->27017-27018/tcp mongo
b18900e3054e hyperledger/cello-dashboard "python dashboard.py" 2 minutes ago Up 2 minutes 8080/tcp dashboard
a2e31e9422f2 hyperledger/cello-watchdog "python watchdog.py" 2 minutes ago Up 2 minutes watchdog
@baohua That's right, this is coming from Makefile and is informational.
ok
@baohua The followings are container information in my AWS.
As you see, mongo couldn't normally start because of permission error.
# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
cef2c63cef3a hyperledger/cello-mongo "docker-entrypoint.sh" 51 minutes ago Exited (1) 51 minutes ago mongo-test
4708c7b554b8 hyperledger/cello-dashboard "bash -c 'python /tes" 51 minutes ago Exited (137) 51 minutes ago dashboard-test
b0ecadce7e4e hyperledger/cello-user-dashboard "bash -c 'cd /usr/app" 52 minutes ago Exited (1) 52 minutes ago user-dashboard
a3749f3dff8b hyperledger/cello-dashboard "python dashboard.py" 52 minutes ago Up 52 minutes 8080/tcp dashboard
7b7988c5852c hyperledger/cello-nginx "/bin/bash /tmp/docke" 52 minutes ago Up 52 minutes 0.0.0.0:80->80/tcp, 0.0.0.0:8080->8080/tcp nginx
2a8d708fb08e hyperledger/cello-watchdog "python watchdog.py" 52 minutes ago Up 2 seconds watchdog
361ad5f5f430 hyperledger/cello-restserver "python restserver.py" 52 minutes ago Up 52 minutes 80/tcp restserver
dbc2d13fa0af hyperledger/cello-mongo "docker-entrypoint.sh" 52 minutes ago Restarting (1) 25 minutes ago 127.0.0.1:27017-27018->27017-27018/tcp mongo
# docker logs -f mongo
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
@baohua The followings are container information in my AWS.
As you see, mongo couldn't normally start because of permission error.
```# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
cef2c63cef3a hyperledger/cello-mongo "docker-entrypoint.sh" 51 minutes ago Exited (1) 51 minutes ago mongo-test
4708c7b554b8 hyperledger/cello-dashboard "bash -c 'python /tes" 51 minutes ago Exited (137) 51 minutes ago dashboard-test
b0ecadce7e4e hyperledger/cello-user-dashboard "bash -c 'cd /usr/app" 52 minutes ago Exited (1) 52 minutes ago user-dashboard
a3749f3dff8b hyperledger/cello-dashboard "python dashboard.py" 52 minutes ago Up 52 minutes 8080/tcp dashboard
7b7988c5852c hyperledger/cello-nginx "/bin/bash /tmp/docke" 52 minutes ago Up 52 minutes 0.0.0.0:80->80/tcp, 0.0.0.0:8080->8080/tcp nginx
2a8d708fb08e hyperledger/cello-watchdog "python watchdog.py" 52 minutes ago Up 2 seconds watchdog
361ad5f5f430 hyperledger/cello-restserver "python restserver.py" 52 minutes ago Up 52 minutes 80/tcp restserver
```dbc2d13fa0af hyperledger/cello-mongo "docker-entrypoint.sh" 52 minutes ago Restarting (1) 25 minutes ago 127.0.0.1:27017-27018->27017-27018/tcp mongo
# docker logs -f mongo
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
@baohua The followings are container information in my AWS.
As you see, mongo couldn't normally start because of permission error.
```# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
cef2c63cef3a hyperledger/cello-mongo "docker-entrypoint.sh" 51 minutes ago Exited (1) 51 minutes ago mongo-test
4708c7b554b8 hyperledger/cello-dashboard "bash -c 'python /tes" 51 minutes ago Exited (137) 51 minutes ago dashboard-test
b0ecadce7e4e hyperledger/cello-user-dashboard "bash -c 'cd /usr/app" 52 minutes ago Exited (1) 52 minutes ago user-dashboard
a3749f3dff8b hyperledger/cello-dashboard "python dashboard.py" 52 minutes ago Up 52 minutes 8080/tcp dashboard
7b7988c5852c hyperledger/cello-nginx "/bin/bash /tmp/docke" 52 minutes ago Up 52 minutes 0.0.0.0:80->80/tcp, 0.0.0.0:8080->8080/tcp nginx
2a8d708fb08e hyperledger/cello-watchdog "python watchdog.py" 52 minutes ago Up 2 seconds watchdog
361ad5f5f430 hyperledger/cello-restserver "python restserver.py" 52 minutes ago Up 52 minutes 80/tcp restserver
```dbc2d13fa0af hyperledger/cello-mongo "docker-entrypoint.sh" 52 minutes ago Restarting (1) 25 minutes ago 127.0.0.1:27017-27018->27017-27018/tcp mongo
```# docker logs -f mongo
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
```/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
@baohua The followings are container information in my AWS.
As you see, mongo couldn't normally start because of permission error.
```# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
cef2c63cef3a hyperledger/cello-mongo "docker-entrypoint.sh" 55 minutes ago Exited (1) 55 minutes ago mongo-test
4708c7b554b8 hyperledger/cello-dashboard "bash -c 'python /tes" 55 minutes ago Exited (137) 55 minutes ago dashboard-test
b0ecadce7e4e hyperledger/cello-user-dashboard "bash -c 'cd /usr/app" 56 minutes ago Exited (1) 56 minutes ago user-dashboard
a3749f3dff8b hyperledger/cello-dashboard "python dashboard.py" 56 minutes ago Up 56 minutes 8080/tcp dashboard
7b7988c5852c hyperledger/cello-nginx "/bin/bash /tmp/docke" 56 minutes ago Up 56 minutes 0.0.0.0:80->80/tcp, 0.0.0.0:8080->8080/tcp nginx
2a8d708fb08e hyperledger/cello-watchdog "python watchdog.py" 56 minutes ago Up 11 seconds watchdog
361ad5f5f430 hyperledger/cello-restserver "python restserver.py" 56 minutes ago Up 56 minutes 80/tcp restserver
```dbc2d13fa0af hyperledger/cello-mongo "docker-entrypoint.sh" 56 minutes ago Restarting (1) 2 minutes ago 127.0.0.1:27017-27018->27017-27018/tcp mongo
```# docker logs -f mongo
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
```/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
@baohua The followings are container information in my AWS.
As you see, mongo couldn't normally start because of permission error.
```# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
cef2c63cef3a hyperledger/cello-mongo "docker-entrypoint.sh" 55 minutes ago Exited (1) 55 minutes ago mongo-test
4708c7b554b8 hyperledger/cello-dashboard "bash -c 'python /tes" 55 minutes ago Exited (137) 55 minutes ago dashboard-test
b0ecadce7e4e hyperledger/cello-user-dashboard "bash -c 'cd /usr/app" 56 minutes ago Exited (1) 56 minutes ago user-dashboard
a3749f3dff8b hyperledger/cello-dashboard "python dashboard.py" 56 minutes ago Up 56 minutes 8080/tcp dashboard
7b7988c5852c hyperledger/cello-nginx "/bin/bash /tmp/docke" 56 minutes ago Up 56 minutes 0.0.0.0:80->80/tcp, 0.0.0.0:8080->8080/tcp nginx
2a8d708fb08e hyperledger/cello-watchdog "python watchdog.py" 56 minutes ago Up 11 seconds watchdog
361ad5f5f430 hyperledger/cello-restserver "python restserver.py" 56 minutes ago Up 56 minutes 80/tcp restserver
dbc2d13fa0af hyperledger/cello-mongo "docker-entrypoint.sh" 56 minutes ago Restarting (1) 2 minutes ago 127.0.0.1:27017-27018->27017-27018/tcp mongo```
```# docker logs -f mongo
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied
/usr/local/bin/docker-entrypoint.sh: line 27: /usr/local/bin/gosu: Permission denied```
can u run `docker images`
@baohua Sure,
```# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
hyperledger/cello-watchdog latest 3104df12505b About an hour ago 747.6 MB
hyperledger/cello-user-dashboard latest 14619e4bab6a About an hour ago 736.3 MB
hyperledger/cello-dashboard latest 2f84bbdbb6a0 About an hour ago 747.6 MB
hyperledger/cello-restserver latest e7ec4c44c975 About an hour ago 747.6 MB
hyperledger/cello-nginx 0.8.0-snapshot-c0cefd0 aaec9e404e5c About an hour ago 683.5 MB
hyperledger/cello-nginx latest aaec9e404e5c About an hour ago 683.5 MB
hyperledger/cello-mongo 0.8.0-snapshot-c0cefd0 1bb3d6767b24 About an hour ago 683.5 MB
hyperledger/cello-mongo latest 1bb3d6767b24 About an hour ago 683.5 MB
hyperledger/cello-baseimage 0.8.0-snapshot-c0cefd0 e66df7a130eb About an hour ago 683.4 MB
hyperledger/cello-baseimage latest e66df7a130eb About an hour ago 683.4 MB
ubuntu xenial 20c44cd7596f 3 weeks ago 122.8 MB```
Has joined the channel.
Trying to spin a blockchain network from cello dashboard, my peer containers goes into restart mode any ideas what this would mean. These are the statement with which the containers are filled with
`2017-12-12 11:07:33.409 UTC [main] main -> ERRO 001 Cannot run peer because error when setting up MSP from directory /etc/hyperledger/fabric/msp: err Could not load a valid signer certificate from directory /etc/hyperledger/fabric/msp/signcerts, err stat /etc/hyperledger/fabric/msp/signcerts: no such file or directory
`
Trying to spin a blockchain network from cello dashboard, my peer containers goes into restart mode any ideas what this would mean. These are the statement with which the containers are filled with
```2017-12-12 11:07:33.409 UTC [main] main -> ERRO 001 Cannot run peer because error when setting up MSP from directory /etc/hyperledger/fabric/msp: err Could not load a valid signer certificate from directory /etc/hyperledger/fabric/msp/signcerts, err stat /etc/hyperledger/fabric/msp/signcerts: no such file or directory
```
@baohua @hightall can you review this patch? thanks https://gerrit.hyperledger.org/r/#/c/15889/
@harsha have u run the msp materials copy from master node to worker node?
@baohua @hightall can you review this patch? thanks https://gerrit.hyperledger.org/r/#/c/15889
thanks.
reviewed, thanks
@KSLee strange, i checked, seems the info is correct, only one diff is that u're using docker 1.12, while i'm using 1.17. could u help post ur steps and error logs to the maillist, i will take a look later.
@baohua @hightall another one please. thanks https://gerrit.hyperledger.org/r/#/c/16169/
@baohua the base tag has been increased to 0.4.2.
so what I have in the patch is correct.
@baohua Thanks, baohua. I used docker 1.12 as the instruction in the Cello Docs indicates
I will try more (upgrade docker version or use other machine, etc) and if it doesn't work, it will post the steps in the email.
Thanks for your help!
Sure feel free to share ur progress. Will response ASAP.
[ ](https://chat.hyperledger.org/channel/cello?msg=ZtzkXAnFmokbWR8Qz) @baohua I presumed it should happen automatically, isn't it ?
Has joined the channel.
Hello All...I am new to hyperledger cello...I have one question, does cello dashboard works only on 8080 port
@mvaibhavshah Right. By default it comes up on 8080 port
@harsha : Thanks .....can we change it...as blockchain explorer also I think can show dashboard on 8080
You might want to have a look at https://github.com/hyperledger/cello/blob/master/docker-compose.yml#L32
@baohua @harsha I finally got to success in running cello in my AWS. I used ubuntu 16_04 and the newest docker CE and docker-compose whereas the instruction indicated to used 14_04, 1.10~1.13, 1.8~1.12.0, respectively. (Those versions don't work for me) In addition, I modified download_images.sh to download Fabric images.
Anyway, thanks folks!
@KSLee May I ask what you changed in download_images.sh ?
@harsha I just manually set ARCH as x86_64 as "ARCH=`uname -m | sed 's|i686|x86|' | sed 's|x86_64|x64|'" wasn't working for me in the script.
FYI, the result of `uname -m | sed 's|i686|x86|' | sed 's|x86_64|x64|'` showed x64 instead of x86_64, which works for my env.
This happens during `make setup-worker` ?
Has joined the channel.
any docs available to setup swarm in cello?
@tongli CE-194
https://gerrit.hyperledger.org/r/#/c/16091/
@KSLee so it's docker version issue? Sorry for the out-dated info, would u like to help fix the doc? i suggest we recommend to use docker 1.17CE now
@baohua @hightall new patch set to address the issue describe. please review. https://gerrit.hyperledger.org/r/#/c/15889/
@baohua @hightall Thanks so much!
will review, thanks
and welcome everyone to help review https://gerrit.hyperledger.org/r/16181
@baohua @hightall 谢谢兄弟们。
Has joined the channel.
It seems that, using the ansible agent (I use ansible alone without the cello-UI right now), it is not possible to deploy an orderer belonging to a certain organisation (say orga) onto a server when there is also a peer belonging to the same organisatiion defined
deployFail.PNG
Is there a way around this (or maybe I just did something wrong here), or am I limited to only deploying peers/orderes belonging to different organisations ?
network.PNG
Changing the orderers to belong to orgc and orgd results in everything working fine
Has joined the channel.
Has left the channel.
@baohua Yes. In my case, both docker version and linux versions were main issues and one script, image_download.sh.
I'd like to help to fix the doc, but I am not familiar with the process of updating documents.
Could you give me any info or instruction page about this?
@KSLee yes, here it is: http://cello.readthedocs.io/en/latest/CONTRIBUTING/
Has joined the channel.
Hello everyone! I am currently looking at Cello, and I saw Hardware requirements for both master and worker nodes - 8c16g100g. Is it possible to test on smaller systems like 1GB of RAM for pure testing purposes?
possible, but will be slower
ok thank you!
Has joined the channel.
@baohua https://gerrit.hyperledger.org/r/#/c/16203/ please take a look ~
great work luke!
after a quick look, why u need to delete those credentials files under msp?
@Luke_Chen and are u following the code at https://github.com/yeasy/docker-compose-files/blob/master/hyperledger_fabric/latest/docker-compose-2orgs-4peers-solo-be.yaml?
if so, it is not finished yet...
@baohua I did some midification base on that example , it can work right now.
what kind of modification, i will test locally
from my feeling, it shouldn't work right now :-)
https://gerrit.hyperledger.org/r/#/c/16203/4/src/agent/docker/_compose_files/fabric-1.0/local/fabric-solo-4.yaml
just by adding to ca
just by adding two cas
those CAs are not necessary, right?
They are necessary, explorer need to enroll certs
let me check
I think there is something wrong with worker node setup scripts. I tried to pull images, but failed, because of this line https://github.com/hyperledger/cello/blob/master/scripts/worker_node/download_images.sh#L16. I am running ubuntu 16.04, and ARCH variable is set to x64 and therefore it could not find docker images from dockerhub. If you see https://hub.docker.com/r/hyperledger/fabric-peer/tags/ it has tag x86_64-1.0.5 not x64-1.0.5
when i remove the sed call from download script, it works great
@harsha would u like to have a look at the download script?
@baohua On it !!
:thumbsup_tone1:
I have run make setup-worker, it pulled all the images, but none of containers are running. is there a script that I should run now?
@maris.c can u follow the steps at: http://cello.readthedocs.io/en/latest?
@baohua I am following this http://cello.readthedocs.io/en/latest/setup_worker_docker/
Yes then the next step is to run the cello service?
do you mean - make start ?
Has joined the channel.
this command just starts master node services
i think there should be a different one to start worker images
Hi guys. I have a question regarding provisioning the Worker nodes.
Can I somehow install all the needed packages on already existing on-premise machine (CentOS or Ubuntu), or I can only provision a new machine with VirtualBox?
I have deployed the Cello Master services on one Machine. I have a on-premise Kubernetes cluster (master and one node).
How to connect the Cello master services to the cluster?
I have deployed the Cello Master services on one Machine. I have a on-premise Kubernetes cluster (master and one node).
How to connect the Cello master services to the cluster?
Should the host be the Kubernetes Master?
Clipboard - December 14, 2017 12:32 PM
Has anybody who succesfully deployed using ansible been able to shutdown the network again via the "ansible-playbook -e "mode=destroy env=bc1st deploy_type=compose" setupfabric.yml" call from the examples?
destroy.PNG
[ ](https://chat.hyperledger.org/channel/cello?msg=m7JHtNbBdg5fkim45) @luminance That's right it should be combination of IPADDR:PORT port on which your K8 is running
[ ](https://chat.hyperledger.org/channel/cello?msg=FqCwC5GWpyrRdjm4K) @maris.c assuming you have added worker node via cello-dashboard services. You would now need to start a chain.
[ ](https://chat.hyperledger.org/channel/cello?msg=FqCwC5GWpyrRdjm4K) @maris.c assuming you have added worker node via cello-dashboard services. You would now need to start a chain from dashboard services
@harsha How can I find the default IP on which Kubernetes master is running?
(I am guessing that should be the port of the Controlling Access of the Kubernetes API.)
welcome to help review: https://gerrit.hyperledger.org/r/#/c/16205/
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/16207/ -- Needs review
hi. when . create new chain i get error message: Exception when compose start=404 Client Error: Not Found ("b'manifest for hyperledger/fabric-peer:latest not found'")
Could you provide Docker hub images with latest tag?
[ ](https://chat.hyperledger.org/channel/cello?msg=eNxAJznTAhRFjhZ6L) @harsha Thanks you response!
[ ](https://chat.hyperledger.org/channel/cello?msg=eNxAJznTAhRFjhZ6L) @harsha Thanks you for your response!
You sure you have ran `make setup-worker` on your worker node ?
yes
on your worker node can you show me `docker images | grep -i peer`
i will try to rebuild and then paste you output!
thanks
it's all good now! But I could not expose docker to master service. I needed to create /etc/systemd/system/docker.service.d/docker.conf file with https://pastebin.com/xdDs4Uvs and /etc/docker/daemon.json file with https://pastebin.com/N0vRyygm and then it worked fine!
but it my chaincodes it is showing Health FAIL so i guess something is not good yet
What can I do with the following error on the worker node when I try to `docker -H Worker_Node_IP:2375 info`?
```
DEBU[6742] Calling GET /v1.24/info
DEBU[6742] Unable to read peer creds: server socket is not a Unix socket
DEBU[6742] {Action=info, LoginUID unknown, PID unknown}
```
welcome to help review: https://gerrit.hyperledger.org/r/#/c/16205
https://jira.hyperledger.org/browse/CE-197
https://gerrit.hyperledger.org/r/16207
https://gerrit.hyperledger.org/r/16211
@baohua It seems some dir was missing in the former patch,and that was updated, sorry for the inconvenience
@baohua It seems some dirs were missing in the former patch,and that was updated, sorry for the inconvenience
@baohua It seems some dirs were missing in the former patch, they were added by the latest patch, sorry for the inconvenience
explorer can work base on my lastest patch
Clipboard - 2017年12月15日凌晨1点08分
but it won't work with v1.0.5 images
because businesschannel can not be created
Clipboard - 2017年12月15日凌晨1点12分
so in my current patch, explorer will only support v1.0.0, I will find out how to support the later verion it in the futur.e
@baohua https://gerrit.hyperledger.org/r/#/c/16203/
@Luke_Chen The BE has changed its struct, pls refer to this one: https://github.com/yeasy/docker-compose-files/blob/master/hyperledger_fabric/latest/docker-compose-2orgs-4peers-solo-be.yaml.
It can work with latest code and images to support blockchain-explorer.
Hello! Does anyone know what does health status fail mean? https://www.evernote.com/l/Af0CjsYNyepEpa5S4qc-vEvzrzX6HaSGBaw
Can you logon to worker node(s) and see the container status ..
@harsha I rebuilt my worker server and now I have error message Exception when compose start=404 Client Error: Not Found ("manifest for hyperledger/fabric-peer:1.0.5 not found").
`git pull; make setup-worker`
@baohua Thank you, that did the trick!
@harsha What do you mean by status exactly? here are my containers https://www.evernote.com/l/Af1rWeQqfvlJoY2cJq_hfIbXq-Tu_8KCHWI, seems that they are running ok!
@baohua Is this config right for orderer?
Clipboard - 2017年12月15日晚上7点29分
Clipboard - 2017年12月15日晚上7点30分
From CE-198 merge
[ ](https://chat.hyperledger.org/channel/cello?msg=KzXAWgrB4k9R7yoxQ) @maris.c You can check the status of your docker-container from 5 Field in the output
@harsha yes it is up and running
any hints where to looks next regarding this FAIL status?
Has joined the channel.
Firstly container status, usually it will be either in exit or restart state, you ought to look through the logs to understand where it is failing
Hi, I am new to Cello project, and just set it up in Docker host mode. I successfully up the server, but cannot add the host, which is the same host with the server node. I got the error log: dashboard | [2017-12-15 12:34:52,151] DEBUG [modules.host] [host.py:89 create()] - Create host: name=135, worker_api=10.244.48.135:2375, host_type=docker, capacity=5, log=local/, autofill=false, schedulable=false
dashboard | [2017-12-15 12:34:53,160] ERROR [agent.docker.docker_swarm] [docker_swarm.py:152 check_daemon()] - Exception in check_daemon Error while fetching server API version: HTTPConnectionPool(host='10.244.48.135', port=2375): Max retries exceeded with url: /version (Caused by NewConnectionError('
I am using the master branch code, and also found a problem on the compose file, need to add "user: "mongodb" to the mongo service, or the Mongo cannot start, it seems to be caused by MongoDB image issue: https://github.com/docker-library/mongo/issues/154
@harsha problem is with orderer service. https://www.evernote.com/l/Af1K0cUIvkJEAqCGQjB5I0tvhWDRmGeDu-8 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: Error while dialing dial tcp 172.18.0.8:7050: getsockopt: connection refused"; Reconnecting to {orderer.example.com:7050
@baohua I have changed the environment of orderer in my latest PR, explorer can work with v1.0.5 now.
https://gerrit.hyperledger.org/r/#/c/16229/
nice work, will review! @Luke_Chen
@baohua I have taken your advice and updated my patch set relativly, thanks !
@baohua I have taken your advice and updated my patch set relatively, thanks !
https://gerrit.hyperledger.org/r/#/c/16265/
https://gerrit.hyperledger.org/r/#/c/16267/
done, thanks
https://gerrit.hyperledger.org/r/#/c/16229/
The recent check-in wrt https://gerrit.hyperledger.org/r/#/c/16091/ and modifications to Dockerfile makes it tightly coupled with arch i.e x86
The intent of leveraging the tag `hyperledger/cello-baseimage` is to ensure it is arch independent
Has joined the channel.
good day. I'm a newbie in cello and i just want to ask if kubernetes is already supported? Based on the docs I can say it's supported but I can't find any docs on how to set it up. I'm don't know python either but i checked the codes and i can't find anything in the k8s agent.. i hope u guys can help. sorry for the newbie question
https://gerrit.hyperledger.org/r/#/c/16269/
@rhansharief thanks for ur questions, currently we have several work for k8s support, one is by @tongli to use ansible to setup k8s cluster, the other is by @grapebaba to support k8s directly. Hope tong and kai can give some comments. Beside, we have docs at http://cello.readthedocs.io/. Thanks!
thank you @baohua
https://gerrit.hyperledger.org/r/#/c/16275/
https://gerrit.hyperledger.org/r/#/c/16275/ -- Needs Review
https://gerrit.hyperledger.org/r/#/c/16275/ -- Needs one more +2
~ https://gerrit.hyperledger.org/r/#/c/16275/ ~
https://gerrit.hyperledger.org/r/#/c/16281/ -- Needs Review
https://gerrit.hyperledger.org/r/#/c/16281/ -- Needs one more +2
~ https://gerrit.hyperledger.org/r/#/c/16281/ ~
https://gerrit.hyperledger.org/r/#/c/16311/ -- Needs Review
https://gerrit.hyperledger.org/r/#/c/16311/ -- Needs +2
~ https://gerrit.hyperledger.org/r/#/c/16311/ ~
Has joined the channel.
Has joined the channel.
Has joined the channel.
is cello production ready?
@Varun2887 Yes.
https://gerrit.hyperledger.org/r/#/c/16381/ -- Needs Review
I install the cello on centos7,but stoped at ``+ apt-get update
Get:1 http://archive.ubuntu.com/ubuntu xenial InRelease [247 kB]
Get:2 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
Ign:3 http://repo.mongodb.com/apt/ubuntu xenial/mongodb-enterprise/3.4 InRelease
Get:4 http://repo.mongodb.com/apt/ubuntu xenial/mongodb-enterprise/3.4 Release [5406 B]
Get:5 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]
Get:6 http://security.ubuntu.com/ubuntu xenial-security/universe Sources [56.7 kB]
Get:7 http://archive.ubuntu.com/ubuntu xenial-backports InRelease [102 kB]
Get:8 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages [521 kB]
Get:9 http://archive.ubuntu.com/ubuntu xenial/universe Sources [9802 kB]
Get:10 http://repo.mongodb.com/apt/ubuntu xenial/mongodb-enterprise/3.4 Release.gpg [801 B]
Get:11 http://repo.mongodb.com/apt/ubuntu xenial/mongodb-enterprise/3.4/multiverse amd64 Packages [8490 B]
Get:12 http://security.ubuntu.com/ubuntu xenial-security/restricted amd64 Packages [12.9 kB]
Get:13 http://security.ubuntu.com/ubuntu xenial-security/universe amd64 Packages [239 kB]
Get:14 http://security.ubuntu.com/ubuntu xenial-security/multiverse amd64 Packages [3481 B]
Get:15 http://repo.mongodb.com/apt/ubuntu xenial/mongodb-enterprise/3.4/multiverse arm64 Packages [8472 B]
Get:16 http://repo.mongodb.com/apt/ubuntu xenial/mongodb-enterprise/3.4/multiverse ppc64el Packages [8489 B]
Get:17 http://repo.mongodb.com/apt/ubuntu xenial/mongodb-enterprise/3.4/multiverse s390x Packages [7283 B]
Get:18 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages [1558 kB]
Get:19 http://archive.ubuntu.com/ubuntu xenial/restricted amd64 Packages [14.1 kB]
Get:20 http://archive.ubuntu.com/ubuntu xenial/universe amd64 Packages [9827 kB]
```
It's OK after change the setup.sh
how to run the master?
you would need to issue `make setup-master`
Hi to all, currently checking Cello for the first time.
I'm also new to Ansible.
I have a virtualbox with a ubuntu server 17.04 and running the all in one tutorial.
It's a clean installation and I'm receiving this error:
fatal: [10.0.2.15]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Permission denied (publickey,password).\r\n", "unreachable": true}
I have ran this before trying to connect, like the tutorial says:
mkdir -p ~/.ssh && cd ~/.ssh && ssh-keygen -t rsa -f fd -P ""
cat ~/.ssh/fd.pub >> ~/.ssh/authorized_keys
whe i try to create a Host in the dashboard, i can´t:
when i try to create a Host in the dashboard, i can´t:
it shows: Can't create a host
@hendry19901990 Mind elaborating your problem a bit more please ?
https://gerrit.hyperledger.org/r/#/c/16435/ -- Needs Review
https://gerrit.hyperledger.org/r/#/c/16381 -- Needs Review
thank you
in the console log
i've got:
dashboard | [2017-12-22 19:58:57,299] ERROR [agent.docker.docker_swarm] [docker_swarm.py:152 check_daemon()] - Exception in check_daemon Error while fetching server API version: HTTPConnectionPool(host='192.168.0.1', port=2375): Max retries exceeded with url: /version (Caused by ConnectTimeoutError(
from ur master node can you kindly issue telnet
the latest cello version is unavailable.
I create a new host.
but the fabric network not startup.
@yoyokeen Please be more specific when you say `cello version is unavailable`
WRT to fabric network not starting-up can you please refer to the docker logs to see where it is failing.
master
```[3/4] Linking dependencies...
error An unexpected error occurred: "ENOENT: no such file or directory, lstat '/node_modules/.yarn-integrity'".
info If you think this is a bug, please open a bug report with the information provided in "/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/add for documentation about this command.
ERROR: Service 'user-dashboard' failed to build: The command '/bin/sh -c yarn add nodemon -g' returned a non-zero code: 1```
user-dashboard,when I run ' docker-compose build'
github,master
Would request you to follow this guide SUHAS
Would request you to follow this guide http://cello.readthedocs.io/en/latest/
thx @harsha
try it,again.
https://github.com/hyperledger/cello/tree/master/
I get the latest code from the url.
https://gerrit.hyperledger.org/r/#/c/16269/
Request for review :-
https://gerrit.hyperledger.org/r/#/c/16435
https://gerrit.hyperledger.org/r/#/c/16381
Request for review :-
https://gerrit.hyperledger.org/r/#/c/16435 - Needs one more +2
https://gerrit.hyperledger.org/r/#/c/16381
Hey @tongli and others can we use google cloud platform using ansible script?
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/16471/
what ip address i should use in daemon url
?
when i try to add a host
https://gerrit.hyperledger.org/r/#/c/16471/
Has joined the channel.
@harsha added some comments, would u like to have a look? @hightall
@harsha added some comments on https://gerrit.hyperledger.org/r/#/c/16381/.
Has left the channel.
Has joined the channel.
Has joined the channel.
hi all
i use cello to create to make blockchain
host added successfully and when add new chain to create blockchain
i found this error
`ERROR: for explorer Cannot start service explorer: oci runtime error:` `container_linux.go:247: starting container process caused` `"process_linux.go:359: container init caused \"rootfs_linux.go:54: mounting` `\\\"/opt/cello/fabric-1.0/local/explorer-artifacts/config.json\\\" to rootfs \\\"/var/lib/docker/overlay2`/c0942a0b749ad436d6f4480fb43623dbd44575fd17f0adfcdbea9390df2c4d8c/merged\\\" at \\\"/var/lib/docker/overlay2/c0942a0b749ad436d6f4480fb43623dbd44575fd17f0adfcdbea9390df2c4d8c/merged/blockchain-explorer/config.json\\\" caused \\\"not a directory\\\"\""
: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
can any one help me ?
( notice this log i get it from dashboard container )
run `make setup-worker` in cello dir of worker node
Hey all. Can I use google cloud platform with ansible instead of aws?
is there any reference about how to connect between cello and composer
Has joined the channel.
i have vm has cello and install chain in another vm
and now i have all peer and other containers in new vm but when i use composer run time i get this error
✖ Installing runtime for business network orange. This may take a minute...
Error: Error trying install composer runtime. Error: The Composer runtime failed to install on 1 or more peers
Command failed
i have vm has cello and install chain in another vm
and now i have all peer and other containers in new vm but when i use composer run time i get this error
✖ Installing runtime for business network orange. This may take a minute...
Error: Error trying install composer runtime. Error: The Composer runtime failed to install on 1 or more peers
Command failed
i have vm has cello and install chain in another vm
and now i have all peer and other containers in new vm but when i use composer run time i get this error
{
✖ Installing runtime for business network orange. This may take a minute...
Error: Error trying install composer runtime. Error: The Composer runtime failed to install on 1 or more peers
Command failed
}
Has joined the channel.
Hey @tongli I have tried to find kubernetes version and try to access dashboard on `fabric001` but couldnt find any version. Are you sure ansible uses kubernetes?
Has joined the channel.
Hello everyone, i want to deploy hyperledger fabric network using ansible but i am confused about server machines. Like can i deploy it on physical machines ? If so, then is there any tutorial or reference for that ? Because i have looked at this *http://cello.readthedocs.io/en/latest/setup_worker_ansible/* but there is no instruction regarding deployment on physical machines . Thanks in advance]\
Ansible should not care the target is vm or server. IMHO the way is similar.
@baohua Okay Thanks but one more thing, which *.yml* should i study or use from *vars* for my case or setup
@baohua Thank you for your response but one more thing which *.yml* should i study or use from *vars/* to use on my setup environment using 3 physical servers, i don't have aws or openstack cloud..
Hey guys, can I deploy a composer business network on top of **Cello** ?
@MohammadObaid most certainly. The first node in your cluster is used as the k8s cluster. if you choose to install fabric onto k8s, k8s gets installed and configured for you by cello. you should be able to see the dashboard off of the first node.
Has joined the channel.
@tongli could u help give some comments on @nazeem92 's question in multi-server deployments?
Has joined the channel.
@baohua @tongli when i want to login the dashboard,it tell Failed to load resource: the server responded with a status of 401 (UNAUTHORIZED)
the username/password is admin/pass
the username/password is admin/pass?
yes, it should be `admin/pass` if not modified
i used cello to install fabric network (2 org , 2 peers per each organization, 1 ca) then we followed the tutorial (https://hyperledger.github.io/composer/tutorials/deploy-to-fabric-multi-org.html) but we have only one ca instead of ca per org and when we run command
*** composer network start -c PeerAdmin@byfn-network-org1 -a tutorial-network/tutorial-network@0.0.1.bna -o endorsementPolicyFile=endorsement-policy.json -A alice -C alice/admin-pub.pem -A bob -C bob/admin-pub.pem ***
we get this error
✖ Starting business network definition. This may take a minute...
Error: Error trying to instantiate composer runtime. Error: Error: SERVICE_UNAVAILABLE
Command failed
docker pull hyperledger/cello-mongo can't pull @baohua @tongli ?
pls search in the docker hub, there requires some image tags.
https://gerrit.hyperledger.org/r/#/c/16381/ -- request for review
https://gerrit.hyperledger.org/r/#/c/16381/ -- Request for review, @baohua @hightall @tongli
~https://gerrit.hyperledger.org/r/#/c/16381/ ~ Done
+3
+2
https://gerrit.hyperledger.org/r/#/c/16435/ -- @tongli @hightall Request for review.
done!
https://gerrit.hyperledger.org/r/16613
done, thanks @hightall
Has joined the channel.
@hightall would u like to have a look at https://jira.hyperledger.org/browse/CE-215?
Has joined the channel.
Hi I am trying to setup cello project on 2 servers on azure one as master other as worker node looking at https://cello.readthedocs.io/en/latest/setup_worker_docker what should be the tcp://URL that needs to be provided here `DOCKER_OPTS="$DOCKER_OPTS -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --api-cors-header='*' --default-ulimit=nofile=8192:16384 --default-ulimit=nproc=8192:16384"` and also what should be the other paramerts
instead of 0.0.0.0:2375 I used vm address of my server
but I got this error
```
docker.service - Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
Active: inactive (dead) (Result: exit-code) since Mon 2018-01-08 11:48:02 UTC; 11s ago
Docs: https://docs.docker.com
Process: 107815 ExecStart=/usr/bin/dockerd -H fd:// $DOCKER_OPTS (code=exited, status=1/FAILURE)
Main PID: 107815 (code=exited, status=1/FAILURE)
Jan 08 11:48:02 vm2 systemd[1]: docker.service: Unit entered failed state.
Jan 08 11:48:02 vm2 dockerd[107815]: user namespaces
Jan 08 11:48:02 vm2 dockerd[107815]: -v, --version Print version information
Jan 08 11:48:02 vm2 dockerd[107815]: and quit
Jan 08 11:48:02 vm2 dockerd[107815]: Run 'dockerd COMMAND --help' for more information on a command.
Jan 08 11:48:02 vm2 systemd[1]: docker.service: Failed with result 'exit-code'.
Jan 08 11:48:02 vm2 systemd[1]: docker.service: Service hold-off time over, scheduling restart.
Jan 08 11:48:02 vm2 systemd[1]: Stopped Docker Application Container Engine.
Jan 08 11:48:02 vm2 systemd[1]: docker.service: Start request repeated too quickly.
Jan 08 11:48:02 vm2 systemd[1]: Failed to start Docker Application Container Engine.
```
I had edited docker.service file this way
```
DOCKER_OPTS="$DOCKER_OPTS -H tcp://hyperchain-info-vm3.southeastasia.cloudapp.azure.com:2375 -H unix:///var/run/docker.sock --api-cors-header='*' --default-ulimit=nofile=8192:16384 --default-ulimit=nproc=8192:16384"
EnvironmentFile=-/etc/default/docker
ExecStart=
ExecStart=/usr/bin/dockerd -H fd:// $DOCKER_OPTS```
Has joined the channel.
Hi, is node-9.x a pre-req for user-dashboard ?
@harsha seems so
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/16649/
Hi all, I downloaded cello from github, but while building it i'm getting the following error: updating the store state of sandbox failed: failed to update store for object type *libnetwork.sbState: json: cannot unmarshal object into Go value of type string
can anyone help me with this
https://gerrit.hyperledger.org/r/#/c/16725/
https://gerrit.hyperledger.org/r/#/c/16721/
@baohua @tongli when i run the command `make start` ,it will get the `https://nexus.hyperledger.org/content/repositories/releases/org/hyperledger/fabric/hyperledger-fabric/linux-amd64-1.0.5/hyperledger-fabric-linux-amd64-1.0.5.tar.gz`
if i download it in advance,where i can put it
I added a host and a chain from cello dashboard what that mean actually I cannt find any documentation regarding that
@DarshanBc cello acts as dashboard to manage lifecycle of your blockchain. in your case you have added a worker node on which you spawned a blockchain-network.
I have 3 servers on azure cloud I need to setup fabric on all those like orderer in one server endorsement container, peers, and ca of each org on the other 2 servers any idea how to do this
I am trying to address this problem via cello
am doing it in right way?
@harsha I have 3 servers on azure cloud I need to setup fabric on all those like orderer in one server endorsement container, peers, and ca of each org on the other 2 servers any idea how to do this
I am trying to address this problem via cello
am doing it in right way?
@DarshanBc Yes, cello is best fit for the scenario.
http://cello.readthedocs.io/en/latest/worker_ansible_howto/?highlight=azure
@harsha ok Currently I created a chain a host but I am not able to kill that process how would I do it?
This is a single node setup that you are referring to ?
yea'
@harsha i can't create the chain,where i can see the log
the page just tell failure
issue `make stop` on master node.
Clean up `/opt/cello` on both master and worker node.
issue make start on master node
and make setup-worker on worker node
@asaningmaxchain any logs/snap of the page would be helpful
@harsha i don't understand
Can you kindly share snap of the page where it fails ?
Selection_003.png
good morning, celloers, I finally took time made two youtube videos.
they are about cello ansible agent.
you can find them here. ```https://www.youtube.com/watch?v=ndLuUSK3XTs&t=12s```
and here ```https://www.youtube.com/watch?v=FV3rN_kwPnA&t=22s```
[ ](https://chat.hyperledger.org/channel/cello?msg=Y6cQcHwMdkP5Tj7E3) @tongli
@asaningmaxchain please work with bauhua and hightall on the aspect of this type of errors.
[ ](https://chat.hyperledger.org/channel/cello?msg=zt7swehgNLdaXQqzF) @baohua @hightall
Hi. I have set up my fabric network using Cello, but now I'm wondering if there is a way to use the Cello dashboard to control that network. Is that possible, or if not is it planned to be implemented?
Hi. I have set up my fabric network using Cello ansible agent, but now I'm wondering if there is a way to use the Cello dashboard to control that network. Is that possible, or if not is it planned to be implemented?
@asaningmaxchain can u help post the error log? `make log`
@SimonOberzan the ansible agent has not been integrated with the dashboard. @tongli might have more comments.
With git commit b14ec7fdc3111308a3126092e87db8ee32a73869, I see the following error on all arch's.
```make: Circular build/docker/baseimage/.s390x-0.8.0-snapshot-b14ec7f <- build/docker/baseimage/.s390x-0.8.0-snapshot-b14ec7f dependency dropped.
Building docker baseimage
if [ "$(docker images -q hyperledger/cello-baseimage 2> /dev/null)" == "" ]; then \
docker build -f build/docker/baseimage/Dockerfile \
-t hyperledger/cello-baseimage \
-t hyperledger/cello-baseimage:s390x-0.8.0-snapshot-b14ec7f \
. ; \
fi
/bin/sh: 1: [: unexpected operator
make: Circular build/docker/mongo/.s390x-0.8.0-snapshot-b14ec7f <- build/docker/mongo/.s390x-0.8.0-snapshot-b14ec7f dependency dropped.
Building docker mongo
if [ "$(docker images -q hyperledger/cello-mongo 2> /dev/null)" == "" ]; then \
docker build -f build/docker/mongo/Dockerfile \
-t hyperledger/cello-mongo \
-t hyperledger/cello-mongo:s390x-0.8.0-snapshot-b14ec7f \
. ; \
fi
/bin/sh: 1: [: unexpected operator
make: Circular build/docker/nginx/.s390x-0.8.0-snapshot-b14ec7f <- build/docker/nginx/.s390x-0.8.0-snapshot-b14ec7f dependency dropped.
Building docker nginx
if [ "$(docker images -q hyperledger/cello-nginx 2> /dev/null)" == "" ]; then \
docker build -f build/docker/nginx/Dockerfile \
-t hyperledger/cello-nginx \
-t hyperledger/cello-nginx:s390x-0.8.0-snapshot-b14ec7f \
. ; \
fi
/bin/sh: 1: [: unexpected operator
```
With git commit b14ec7fdc3111308a3126092e87db8ee32a73869, I see the following error on all arch's.
```make: Circular build/docker/baseimage/.s390x-0.8.0-snapshot-b14ec7f <- build/docker/baseimage/.s390x-0.8.0-snapshot-b14ec7f dependency dropped.
Building docker baseimage
if [ "$(docker images -q hyperledger/cello-baseimage 2> /dev/null)" == "" ]; then \
docker build -f build/docker/baseimage/Dockerfile \
-t hyperledger/cello-baseimage \
-t hyperledger/cello-baseimage:s390x-0.8.0-snapshot-b14ec7f \
. ; \
fi
/bin/sh: 1: [: unexpected operator
```
however when I switch my commit level to 8524352198b40cd61fabcaa6183619d3a615bc52, `make docker` works as expected
Will open a JIRA item to track
@baohua can you set the log in the specified folder
[ ](https://chat.hyperledger.org/channel/cello?msg=RaucRgviNjkw2XFCu) https://jira.hyperledger.org/browse/CE-222
watchdog | [2018-01-10 16:23:15,531] WARNING [modules.cluster] [cluster.py:374 delete()] - Error to run compose clean work
FileNotFoundError: [Errno 2] No such file or directory: './agent/docker/_compose_files/fabric-1.0/local/fabric-kafka-4.yaml'
@baohua
@harsha yes, and @rameshthoomu is helping, although we do not need run the building locally now, as in the master-setup command, we have pulled the images from dockerhub.
@asaningmaxchain would u like to create a bug in jira? thanks
@baohua I presume non-x86 docker images are yet to pushed ?
@baohua I presume non-x86 docker images are yet to be pushed ?
@harsha i was thinking talking to u on this haha
so have u tested if dockerhub can build images for them now?
so have u tested if dockerhub can build images automatically for them now?
otherwise, we have to push from local manually
dockerhub auto-build-trigger may not work on p/z . Hence I requested @rameshthoomu to create these jobs
https://jenkins.hyperledger.org/view/cello/job/cello-release-ppc64le/
https://jenkins.hyperledger.org/view/cello/job/cello-release-s390x/
Via which we can push the images
dockerhub auto-build-trigger may not work on p/z . Hence I requested @rameshthoomu to create these jobs
https://jenkins.hyperledger.org/view/cello/job/cello-release-ppc64le/
https://jenkins.hyperledger.org/view/cello/job/cello-release-s390x/
Via which we can push the images with relevant tags
great! let's take this workaround first.
Sure thing :)
maybe dockerhub can enhance its capability later
besides, would u like to enhance our doc to indicate how we make/build images now?
i saw u have added some image introduction there, it's great
Ok will add these jobs now and let you know the update @harsha
@baohua Sure thing, would open up a relevant a jira to discuss how to proceed ahead.
@rameshthoomu Thank you, I suspect `make docker` might break, however will wait to hear from you.
thanks, and let's keep improve this process. in future, we should encourage most users just pull images from dockerhub, and for dev, we can also provide method to build image locally.
this is quite important for the setup process
welcome to help review: https://gerrit.hyperledger.org/r/#/c/16745/
@hightall @tongli @harsha @lujiang
https://gerrit.hyperledger.org/r/#/c/16751/
Request for review :- https://gerrit.hyperledger.org/r/#/c/16761/
+2
hello.. I have a problem creating a chain.. after I create a chain, all docker images are up except for the blockchain-explorer.. it says
/blockchain-explorer/app/helper.js:164
if(channels[org][channelName] == undefined)
^
TypeError: Cannot read property 'businesschannel' of undefined
at Object.getChannelForOrg (/blockchain-explorer/app/helper.js:164:20)
at getChainInfo (/blockchain-explorer/app/query.js:91:23)
at Object.getChannelHeight (/blockchain-explorer/app/query.js:205:9)
at getMaxBlockNum (/blockchain-explorer/service/blockscanner.js:100:18)
at Object.syncBlock (/blockchain-explorer/service/blockscanner.js:40:9)
at Timeout._onTimeout (/blockchain-explorer/listener/blocklistener.js:50:22)
at ontimeout (timers.js:386:11)
at tryOnTimeout (timers.js:250:5)
at Timer.listOnTimeout (timers.js:214:5)
Has joined the channel.
Has joined the channel.
Hi guys, I'm trying to use cello to provision my network, but it doesn't work as expected. I've created a test Chain on one host and I've then tried to delete it, but I always get this error message `Failed! Failed to delete cluster `. I don't understand where I can find more info on why it didn't delete the chain
Hi guys, I'm trying to use cello to provision my network, but it doesn't work as expected. I've created a test Chain on one host and I've then tried to delete it, but I always get this error message `Failed! Failed to delete cluster `. I don't understand where I can find more info on why it didn't delete the chain. I also can't see any Platform Status, it keeps loading but nothing shows
Hi I am new to cello can any one explain how can we run the application on different host machines?
@rhansharief blockchain-explorer failed recently dueto code update, will fix it. could u help create a jira bug?
@fabcan can u find the error log by `make log`?
@Lavanya5896 certainly, cello is distributed, see cello.readthedocs.org
https://gerrit.hyperledger.org/r/#/c/16773/ @rhansharief welcome to help review the fix
@baohua among others `dashboard | [2018-01-11 14:53:06,741] WARNING [modules.cluster] [cluster.py:374 delete()] - Error to run compose clean work
dashboard | [2018-01-11 14:53:06,742] WARNING [resources.cluster_api] [cluster_api.py:290 cluster_delete()] - Failed to delete cluster ...
dashboard | 172.19.0.3 - - [11/Jan/2018 14:53:06] "DELETE /api/cluster HTTP/1.0" 400 -`
strange, there's not enough info. can u try our latest code, which should output details with error?
it's easy to do so: `git pull; make setup-master; make restart`
`git pull` gave me "already up to date", the sugested commands didn't give me different results, I've the same line of logs
@baohua `git pull` gave me "already up to date", the sugested commands didn't give me different results, I've the same line of logs
@baohua `git pull` gave me "already up to date", the sugested commands didn't give me different results, I've the same line of logs. I've spotted also a `FileNotFoundError: [Errno 2] No such file or directory: './agent/docker/_compose_files/fabric-1.0/local/fabric-kafka-4.yaml'` which was already signaled before
@baohua `git pull` gave me "already up to date", the sugested commands didn't give me different results, I've the same line of logs. I've spotted also a `FileNotFoundError: [Errno 2] No such file or directory: './agent/docker/_compose_files/fabric-1.0/local/fabric-kafka-4.yaml'` which was already signaled before from other users
oh, i know the problem, there's file missed. the solo one should work now.
btw, would u like to help report a bug in jira?
jira.hyperledger.org
@baohua by sifting only ERROR and WARNING (with `make logs | grep ERROR` or `make logs | grep WARN`) i get the following:
```
WARNING [resources.cluster_api] [cluster_api.py:290 cluster_delete()] - Failed to delete cluster 095...
WARNING [agent.docker.docker_swarm] [docker_swarm.py:479 compose_clean()] - Exception when cleaning project 095...
WARNING [modules.cluster] [cluster.py:374 delete()] - Error to run compose clean work
ERROR [agent.docker.docker_swarm] [docker_swarm.py:457 compose_clean()] - Error in stop compose project, will clean
```
hope it helps
yes, since the file is missing, it won't be able to clean the kafka mode chain now...
@baohua as you requested, https://jira.hyperledger.org/browse/CE-227
Has joined the channel.
On the same lines, trying to spin up a fabric-solo on ppc64le machine got the following :-
```dashboard | [2018-01-11 16:26:08,203] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:391 _compose_set_env()] - envs {'VM_ENDPOINT': 'tcp://129.40.81.29:2375', 'COMPOSE_PROJECT_NAME': '3b4c5c48e0e3445cbdb6490959640f60', 'DOCKER_HOST': 'tcp://129.40.81.29:2375', 'HLF_VERSION': '1.0.5', 'COMPOSE_PROJECT_PATH': '/opt/cello/fabric-1.0/local', 'CLUSTER_LOG_LEVEL': 'DEBUG', 'NETWORK_TYPES': 'fabric-1.0', 'PEER_NETWORKID': '3b4c5c48e0e3445cbdb6490959640f60', 'COMPOSE_FILE': 'fabric-solo-4.yaml', 'VM_DOCKER_HOSTCONFIG_NETWORKMODE': 'cello_net_solo', 'CLUSTER_NETWORK': 'cello_net_solo'}
dashboard | [2018-01-11 16:26:08,203] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:428 compose_up()] - template path ./agent/docker/_compose_files/fabric-1.0/local
dashboard | 5.7: Pulling from library/mysql
dashboard | [2018-01-11 16:26:08,543] WARNING [agent.docker.docker_swarm] [docker_swarm.py:432 compose_up()] - Exception when compose start=no matching manifest for linux/ppc64le in the manifest list entries
dashboard | [2018-01-11 16:26:08,543] WARNING [agent.docker.cluster] [cluster.py:58 create()] - failed to create cluster, with container={}
dashboard | [2018-01-11 16:26:08,543] WARNING [modules.cluster] [cluster.py:244 create()] - failed to start cluster=TEST, then delete
dashboard | [2018-01-11 16:26:08,543] DEBUG [modules.cluster] [cluster.py:315 delete()] - Delete cluster: id=3b4c5c48e0e3445cbdb6490959640f60, forced=True
dashboard | [2018-01-11 16:26:08,545] INFO [modules.cluster] [cluster.py:821 db_update_one()] - filter {'state': 'active', 'id': '3b4c5c48e0e3445cbdb6490959640f60'} operations {'user_id': '__SYSTEM__DELETING'}
```
On the same lines, trying to spin up a fabric-solo on ppc64le machine got the following :-
```dashboard | [2018-01-11 16:26:08,203] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:391 _compose_set_env()] - envs {'VM_ENDPOINT': 'tcp://129.40.81.29:2375', 'COMPOSE_PROJECT_NAME': '3b4c5c48e0e3445cbdb6490959640f60', 'DOCKER_HOST': 'tcp://129.40.81.29:2375', 'HLF_VERSION': '1.0.5', 'COMPOSE_PROJECT_PATH': '/opt/cello/fabric-1.0/local', 'CLUSTER_LOG_LEVEL': 'DEBUG', 'NETWORK_TYPES': 'fabric-1.0', 'PEER_NETWORKID': '3b4c5c48e0e3445cbdb6490959640f60', 'COMPOSE_FILE': 'fabric-solo-4.yaml', 'VM_DOCKER_HOSTCONFIG_NETWORKMODE': 'cello_net_solo', 'CLUSTER_NETWORK': 'cello_net_solo'}
dashboard | [2018-01-11 16:26:08,203] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:428 compose_up()] - template path ./agent/docker/_compose_files/fabric-1.0/local
dashboard | 5.7: Pulling from library/mysql
dashboard | [2018-01-11 16:26:08,543] WARNING [agent.docker.docker_swarm] [docker_swarm.py:432 compose_up()] - Exception when compose start=no matching manifest for linux/ppc64le in the manifest list entries
```
@baohua @harsha Do you have any plans to release `0.8.0` cello version..
FYI: release jobs in jenkins trigger based on the release tag event.. so, once you create release tag, Jenkins trigger `z and p` release jobs and publish images to hyperledger docker hub account
Has joined the channel.
yes, we are busy with bug fixing for the coming v0.8.0, it will come out soon with numbers of amazing features!
Enable the kafka mode support, welcome to review: https://gerrit.hyperledger.org/r/#/c/16789/ @Sufbin @tongli @hightall
Has joined the channel.
Fix explorer support: https://gerrit.hyperledger.org/r/#/c/16773/
Has joined the channel.
hi, all
I used cello ,yeasy/blockchain-explorer
Please open Internet explorer to access :http://explorer:8080/
/blockchain-explorer/app/helper.js:164
if(channels[org][channelName] == undefined)
^
TypeError: Cannot read property 'testblock' of undefined
at Object.getChannelForOrg (/blockchain-explorer/app/helper.js:164:20)
at getChainInfo (/blockchain-explorer/app/query.js:91:23)
at Object.getChannelHeight (/blockchain-explorer/app/query.js:205:9)
at getMaxBlockNum (/blockchain-explorer/service/blockscanner.js:100:18)
at Object.syncBlock (/blockchain-explorer/service/blockscanner.js:40:9)
at Timeout._onTimeout (/blockchain-explorer/listener/blocklistener.js:50:22)
at ontimeout (timers.js:386:11)
at tryOnTimeout (timers.js:250:5)
at Timer.listOnTimeout (timers.js:214:5)
In the docker , yeasy/blockchain-explorer
1515728025496.jpg
@qqbxclboy we are fixing this problem, please refer this link https://gerrit.hyperledger.org/r/#/c/16773
Hi @tongli I went through your youtube video but I already haver 3 azure servers running with ubuntu so how shall I go onto create an overlay network between them?
and then having configuration of fab001,002,003 as you have demonstrated
and then having configuration of fab001,002,003 as you have demonstrated on each servers
@baohua blockchain-explorer is now working on my end. thank you
I have 3 servers with hostname xyz.azure.com what I need to put in this runhost file its public IP?```cloud ansible_host=127.0.0.1 ansible_python_interpreter=python
169.45.102.186 private_ip=10.0.10.246 public_ip=169.45.102.186 inter_name=fabric001
169.45.102.187 private_ip=10.0.10.247 public_ip=169.45.102.187 inter_name=fabric002
169.45.102.188 private_ip=10.0.10.248 public_ip=169.45.102.188 inter_name=fabric003
[allnodes]
169.45.102.186
169.45.102.187
169.45.102.188
[etcdnodes]
169.45.102.186
169.45.102.187
169.45.102.188
[builders]
169.45.102.186
```
I have 3 servers with hostname xyz.azure.com what I need to put in this runhost file for its public IP?```cloud ansible_host=127.0.0.1 ansible_python_interpreter=python
169.45.102.186 private_ip=10.0.10.246 public_ip=169.45.102.186 inter_name=fabric001
169.45.102.187 private_ip=10.0.10.247 public_ip=169.45.102.187 inter_name=fabric002
169.45.102.188 private_ip=10.0.10.248 public_ip=169.45.102.188 inter_name=fabric003
[allnodes]
169.45.102.186
169.45.102.187
169.45.102.188
[etcdnodes]
169.45.102.186
169.45.102.187
169.45.102.188
[builders]
169.45.102.186
```
@harsha and @tongli can you help me?
@DarshanBc let me work with you on that around 9:30am est.
@DarshanBc if you have azure, you can use one command to setup everything and do not even have to deal with the runhost file.
@DarshanBc you only need to deal with the runhost file when you have physical machines or VMs created not by cello ansible agent.
@rhansharief good to hear that :) and welcome for more comments and fix!
https://gerrit.hyperledger.org/r/#/c/16815/
Has joined the channel.
https://jira.hyperledger.org/browse/CE-231
Request for review :- https://gerrit.hyperledger.org/r/#/c/16833/
Needs one more +2 https://gerrit.hyperledger.org/r/#/c/16833/
Has joined the channel.
Has joined the channel.
good day! just a clarification.. when i add a chain, i observed that all running peers from both orgs are all running from ports 7050-7053 with one ca running for both org.. this is correct right?
https://gerrit.hyperledger.org/r/#/c/16859/
@tongli baohua has answered me that the dashboard is not yet integrated with ansible agent. Can you please comment on the future of the integration? Is that planned and if so what is it's priority?
@tongli baohua has answered me that the dashboard is not yet integrated with the ansible agent. Can you please comment on the future of the integration? Is that planned and if so what is it's priority?
Hi All!! I'm encountering this error creating a chain `Cannot start service explorer: oci runtime error: container_linux.go:265: starting container process caused "process_linux.go:368: container init caused \"rootfs_linux.go:57: mounting \\\"/opt/cello/fabric-1.0/local/explorer-artifacts/config.json\\\" to rootfs \\\"/var/lib/docker/aufs/mnt/030a3f290302787881c036ba447716911ca660ee428caf6d40a227fa2a1c45e8\\\" at \\\"/var/lib/docker/aufs/mnt/030a3f290302787881c036ba447716911ca660ee428caf6d40a227fa2a1c45e8/blockchain-explorer/config.json\\\" caused \\\"not a directory\\\"\""
: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type``
Hi All!! I'm encountering this error creating a chain `Cannot start service explorer: oci runtime error: container_linux.go:265: starting container process caused "process_linux.go:368: container init caused \"rootfs_linux.go:57: mounting \\\"/opt/cello/fabric-1.0/local/explorer-artifacts/config.json\\\" to rootfs \\\"/var/lib/docker/aufs/mnt/030a3f290302787881c036ba447716911ca660ee428caf6d40a227fa2a1c45e8\\\" at \\\"/var/lib/docker/aufs/mnt/030a3f290302787881c036ba447716911ca660ee428caf6d40a227fa2a1c45e8/blockchain-explorer/config.json\\\" caused \\\"not a directory\\\"\""
: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type`
Hi All!! I'm encountering this error creating a chain `Cannot start service explorer: oci runtime error: container_linux.go:265: starting container process caused "process_linux.go:368: container init caused \"rootfs_linux.go:57: mounting \\\"/opt/cello/fabric-1.0/local/explorer-artifacts/config.json\\\" to rootfs \\\"/var/lib/docker/aufs/mnt/030a3f290302787881c036ba447716911ca660ee428caf6d40a227fa2a1c45e8\\\" at \\\"/var/lib/docker/aufs/mnt/030a3f290302787881c036ba447716911ca660ee428caf6d40a227fa2a1c45e8/blockchain-explorer/config.json\\\" caused \\\"not a directory\\\"\""
: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type`
Hi All!! I'm encountering this error creating a chain Cannot start service explorer: oci runtime error: container_linux.go:265: starting container process caused "process_linux.go:368: container init caused \"rootfs_linux.go:57: mounting \\\"/opt/cello/fabric-1.0/local/explorer-artifacts/config.json\\\" to rootfs \\\"/var/lib/docker/aufs/mnt/030a3f290302787881c036ba447716911ca660ee428caf6d40a227fa2a1c45e8\\\" at \\\"/var/lib/docker/aufs/mnt/030a3f290302787881c036ba447716911ca660ee428caf6d40a227fa2a1c45e8/blockchain-explorer/config.json\\\" caused \\\"not a directory\\\"\""
: `Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type`
@SimonOberzan We would like to integrate and need someone to help.
Has joined the channel.
@tongli I have to work on some other things now, but after I finish, I will try to get a permission from my management to work on this :)
sure
@ascatox , have u pull the latest code and run `make setup-worker` without any error msg?
https://gerrit.hyperledger.org/r/#/c/16859/
anybody wanna help fix this bug? https://jira.hyperledger.org/browse/CE-234
Hi @baohua
I do not see any responses in #blockchain-explorer wrt [CE-228]. However wanted to propose if we can build and integrate the docker-images via Cello ?
u're right. unless the blockchain-explorer team release the official images, we can only pick the 3rd ones temporarily.... have u discussed with those BE maintainers? i guess most projects release docker images now.
@tongli I have vms in azure which are not created by ansible agent [ ](https://chat.hyperledger.org/channel/cello?msg=Nxzb8s6miqMkZpFLB)
https://gerrit.hyperledger.org/r/#/c/16859/
Anyone wanna help this bug? https://jira.hyperledger.org/browse/CE-235
https://gerrit.hyperledger.org/r/#/c/16889/
[ ](https://chat.hyperledger.org/channel/cello?msg=ztDEWhnKPbcKsYohY) @baohua Yes I pull the latest version from without any kind of errors or problems
[ ](https://chat.hyperledger.org/channel/cello?msg=ztDEWhnKPbcKsYohY) @baohua Yes I pulled the latest version from without any kind of errors or problems
[ ](https://chat.hyperledger.org/channel/cello?msg=ztDEWhnKPbcKsYohY) @baohua The only problem encountered in the installation process, was in the *worker* side where I couldn't use the procedure to modify the docker start for Ubuntu 16.04 (my actual distro), but I had to use the "All distros" one.
Is it possible to add couchDB to the fabric chain created by Cello ???
@ascatox i didn't meet ur problem, but would u like to help report the bug details (including steps to repeat) in jira and post the link here? thanks!
@ascatox fabric doesn't support change database backend dynamically now...
[ ](https://chat.hyperledger.org/channel/cello?msg=k393yJzjPJuDxymvH) @baohua Thank you I solved my previous error installing everything in another VM. Thanks a lot for the support.
[ ](https://chat.hyperledger.org/channel/cello?msg=hFASHvEdHkgSCmvyM) @baohua I'm not interested to do this dynamically but in a configuration phase before starting everything
[ ](https://chat.hyperledger.org/channel/cello?msg=hFASHvEdHkgSCmvyM) @baohua I'm not interested to do this dynamically but in a configuration phase before starting everything.
oh, in that case, we do not have the feature. if u need it, pls feel free to create a jira task first.
certainly we welcome anyone to help contribute :)
[ ](https://chat.hyperledger.org/channel/cello?msg=TYYWkLJ6NqN2F6vfe) Hi cab anybody reply me for this query I'm stuck from past few days
https://gerrit.hyperledger.org/r/#/c/16899/
Has joined the channel.
@DarshanBc what problem did you see?
that runhosts file looks fine to me.
@tongli my query is I have URL for my servers not the public IP so how do I alter runhost file?
well, if you do not have public IPs, it will be hard to do this because you have to run the ansible script from a machine.
how do you access machines in your cloud if you do not have public IPs?
Has joined the channel.
@tongli I am doing ssh it via URL xxx.cloudapp.azure.com
https://gerrit.hyperledger.org/r/#/c/16931/
Has joined the channel.
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/16935/
Has joined the channel.
https://gerrit.hyperledger.org/r/16939
We will release v0.8.0-alpha this weekend, welcome to help report bugs
Has joined the channel.
Has joined the channel.
Do we have a document , how to deploy on multiple peers using kubernetes
i am looking for 4 peers 4 organizations.
@SunilMahamuni all you need to expose your K8 Port for your worker node from cello dashboard
FWIW, http://cello.readthedocs.io/en/latest/tutorial/
Has joined the channel.
Has joined the channel.
Has left the channel.
https://gerrit.hyperledger.org/r/#/c/16939/
+2, thanks
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/16935 @baohua
+1, pls update ur commit msg
thansk
https://gerrit.hyperledger.org/r/#/c/16935/
https://gerrit.hyperledger.org/r/#/c/16977/
https://gerrit.hyperledger.org/r/#/c/16979/ @baohua
done
congrats to huiying's first merged patchset! @huiyingGong
Has joined the channel.
Has joined the channel.
Has joined the channel.
hi everyone
I am having trouble in setting up the blockchain network with cello , i am using docker swarm and I dont know what is the host domain name? Should it be the ip of leader node?
@ctyu That's right.. it shd be IPADDR:PORT of the leader node ..
+1
https://gerrit.hyperledger.org/r/#/c/17019/ v0.8 release
https://gerrit.hyperledger.org/r/#/c/17023/
@harsha @baohua thz let me try try
Hi @tongli I tried creating my runhost file for existing Vms on azure and running *ansible playbook command* but I got this error ```PLAY [Get start timestamp] *****************************************************
TASK [setup] *******************************************************************
ok: [cloud]
TASK [set_fact] ****************************************************************
ok: [cloud]
PLAY [fast initialize all the servers] *****************************************
TASK [setup] *******************************************************************
TASK [setup] *******************************************************************
fatal: [xx.xx.xx.118]: UNREACHABLE! => {"changed": false, "msg": "ERROR! SSH encountered an unknown error during the connection. We recommend you re-run the command using -vvvv, which will enable SSH debugging output to help diagnose the issue", "unreachable": true}
fatal: [xx.xx.xx.209]: UNREACHABLE! => {"changed": false, "msg": "ERROR! SSH encountered an unknown error during the connection. We recommend you re-run the command using -vvvv, which will enable SSH debugging output to help diagnose the issue", "unreachable": true}
NO MORE HOSTS LEFT *************************************************************
PLAY [Setting up etcd services] ************************************************
PLAY [setup flanned on all nodes] **********************************************
PLAY [setup dns attribute for docker service] **********************************
PLAY [setup dns services and registrator] **************************************
PLAY [Reset connections] *******************************************************
PLAY [inform the installer] ****************************************************
TASK [setup] *******************************************************************
ok: [cloud]
TASK [debug] *******************************************************************
ok: [cloud] => {
"msg": "The work load started at 12:18:12, ended at 12:18:13"
}
PLAY RECAP *********************************************************************
xx.xx.xx.118 : ok=0 changed=0 unreachable=1 failed=0
xx.xx.xx.209 : ok=0 changed=0 unreachable=1 failed=0
cloud : ok=4 changed=0 unreachable=0 failed=0
```
I think the problem was with authentication I have priv keys to ssh to server there was no mention of them while running cello
@tongli when I used ssh debug Password authntication is set to no and user name is set to ubuntu I need to change authentication method as well as user name how do I do it?
@DarshanBc looks like you can not reach your host. the machine you are running ansible script has to be able to reach your fabric machines via ssh. You can try to manually ssh to these hosts and see if you can do that, if you can not do that manually, you won't be able to use ansible script.
@tongli I can ssh those servers manually through priv keys and pwd thing is I haven't mentioned either the path of priv keys or mentioned pwd before running the script so how do I include then while executing ansible-playbook command
so you can use your private key and pass phrase to access these machines, right?
Has joined the channel.
Yes
But how do I use it while running script
I mean ansible script
@DarshanBc use ssh-agent
https://gerrit.hyperledger.org/r/#/c/17019/ @hightall @tongli
@baohua, when are you planning to release 0.8.0?
are there any fixes we are waiting for?
let's give 1-2 weeks experience on alpha version, fixing user reported bug, after we think it's stable, we can consider the release. thoughts?
I like to integrate the ansible agent created cluster to dashboard, need a bit help.
i believe that won't be a easy work, so we may put it as after v0.8.0 release.
say the fabric network is running, how can we make the cluster(network) show up in the dashboard?
yes, it's useful
@hightall @huiyingGong would u like to give some comments on this feature?
we should have all the information about nodes, where they are, how to access them, where fabric components running,
@tongli exactly! feel free to create a jira new feature task first, let's track there.
what I am a bit concerned is that we are going to be based on k8s, k8s dashboard provides already a great deal of information, we are going to use helm to manage the cluster, will that be a lot of overlap?
yes, but k8s dashboard is focusing on those container info, he won't know the service type
that might be one place to differentiate
@tongli just rebased to latest master: https://gerrit.hyperledger.org/r/#/c/17019/.
k8s dashboard has service information as well.
which tells you what fabric network services you have currently, either peer or orderer or kafka etc.
at least that is how ansible agent sets up things in k9s env.
k8s
not that "service", i mean the app info, take fabric as example, k8s cannot know what is org, it can only know namespace. and it neither know blocks.
yeah detailed blockchain related info.
yes, as container level, k8s knows, it is a container cloud manager, but not a blockchain manager
like channels, orgs.
exactly
but we certainly can leverage k8s as one of our underly infrastructure. that's cello's initial scope, we take advantage of existing platform techniques, but focus on the DLT layer.
not sure if k8s dashboard can be customized to introduce new segement.
currently, we can support docker host/swarm/vmware center/ ansible (hosts, vm, cloud...)/
if it does, we can develop a dashboard plugin which deals with fabric specific features such as channels, orgs etc
yes. i'm not sure, but i believe from k8s's side, it will be a general container cloud manager.
as openstack to be a general vm cloud manager
@baohua I believe Makefile with version info. should be updated to publish the docker-images.
Alongwith `IS_RELEASE=true`which should be reset to false in next PR.
@harsha yes, let's follow that in the coming v0.8.0 major release. Btw, would u like to help enhance the doc on the release process?
for this alpha release, let's still follow the legacy way?
as we wanna quickly publish it for call for feedbacks and bug reports
how do u feel?
I've no problems with the approach, however the pressing point would be publishing the docker-images for non-x86 archs which aren't in the dockerhub yet, since we have made significant progress wrt v0.8.0-alpha we can do one-off publish, would be a good stepping point.
sure, thanks, let's finish all arch support in the 0.8.0 major release.
螢幕快照 2018-01-22 上午10.56.48.png
i am trying to setup with ansible, suppose 13.115.186.233 / 13.231.77.57 / 54.64.163.73 is the same fabric001/fabric002/fabric003
why cannot connect
haha soad
sosad
@here Hyperledger Cello v0.8.0-alpha is released, and thanks to every developer. Welcome to try and report issues: https://github.com/hyperledger/cello/releases/tag/v0.8.0-alpha.
@baohua echo that. thanks.
@ctyu ansible uses ssh to connect. you can set up ssh-agent , then you should be able to connect to these machines.
i already setup ssh-agent so that i can create fabric 001/fabric 002
@harsha would u like to add the dockerfiles for both v0.8.0-alpha and coming v0.8.0: https://jira.hyperledger.org/browse/CE-202, thanks!
@baohua on it now. :)
great
@cytu, it is easy to test this. if you set up ssh-agent already, you can manually test it, just do a ssh user@your_machine_ip,
if that works, you should be able to access these using the ansible script.
@tongli i can access with -i option ssh -i "key" user@your_machine_i
right in that case, when you start up your ssh-agent, you would use that key and passphrase.
not your default one.
that mean i need to set my key as default?
after v0.8.0, an important feature is to enable the hyperledger compose support: https://jira.hyperledger.org/browse/CE-245. Welcome anyone interested to help. @Yixing
@ctyu you do not have to.
@baohua * Enable user dashboard, can apply chain, query block/transaction,
upload/invoke/query chain code; seem it is a very exciting feature haha
just need to use it when you star tyour agent.
@ctyu exactly, it's great contribution from @hightall
welcome to help report issues and advice
@ctyu do this
```eval "$(ssh-agent -s)"
Agent pid 59566
```
ssh-add -K "whatever_your_key"
it will ask your passphrase, ssh will use the ssh-agent provided credentials to connect with remote servers.
let me try
haha
ssh-add -K "the pid?"
no. your key.
then what is the usage of eval "$(ssh-agent -s)"?
it start a agent.
thz
ssh-add -K "this should be the file path"?
haha
i am so stupid
say your key is named and place here ~/.ssh/mykey
then you do ssh-add -K ~/.ssh/mykey, it will ask you the passphrase.
after that you will be able to simply just do this
ok
``` ssh user@ip_address```
you will get logged in.
ansible agent just does that. only thing it can not do is it can not start your agent.
here is what I normally do
```eval `ssh-agent -s`
ssh-add ~/.ssh/fd```
fd is my key.
private key of course.
the public key should have been on your machine that you try to remotely access.
ok
螢幕快照 2018-01-22 上午11.20.47.png
just miss understand it will get the path from private_key
=m=
let me try again
seem working now!
thz @tongli
after setup i suppose can access my k8s in (domain:8080/api)??
i mean setup ansible?
Hi @tongli I established ssh but it is failing with this ```
I ran this command `ansible-playbook -vvv -i run/runhosts.tpl -e "mode=apply env=os env_type=flanneld ansible_ssh_user=hyperchain" initcluster.yml`
@tongli I ran this command ``ansible-playbook -vvv -i run/runhosts.tpl -e "mode=apply env=azure env_type=flanneld ansible_ssh_user=hyperchain" initcluster.yml
``` But I got this error ```fatal: [52.230.121.199]: FAILED! => {"failed": true, "msg": "ERROR! file or module does not exist: /home/rtcin/go/src/github.com/hyperledger/hyperchain/cello/src/agent/ansible/roles/env_flanneld/https:/storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz"}
```
@tongli I ran this command ```ansible-playbook -vvv -i run/runhosts.tpl -e "mode=apply env=azure env_type=flanneld ansible_ssh_user=hyperchain" initcluster.yml
``` But I got this error ```fatal: [52.230.121.199]: FAILED! => {"failed": true, "msg": "ERROR! file or module does not exist: /home/rtcin/go/src/github.com/hyperledger/hyperchain/cello/src/agent/ansible/roles/env_flanneld/https:/storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz"}
```
@tongli I ran this command ```ansible-playbook -vvv -i run/runhosts.tpl -e "mode=apply env=azure env_type=flanneld ansible_ssh_user=hyperchain" initcluster.yml
``` But I got this error ```fatal: [52.230.121.199]: FAILED! => {"failed": true, "msg": "ERROR! file or module does not exist: /home/rtcin/go/src/github.com/hyperledger/hyperchain/cello/src/agent/ansible/roles/env_flanneld/https:/storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz"}```
@tongli I ran this command ```ansible-playbook -vvv -i run/runhosts.tpl -e "mode=apply env=azure env_type=flanneld ansible_ssh_user=hyperchain" initcluster.yml
``` But I got this error ```fatal: [52.230.121.199]: FAILED! => {"failed": true, "msg": "ERROR! file or module does not exist: /home/rtcin/go/src/github.com/hyperledger/hyperchain/cello/src/agent/ansible/roles/env_flanneld/https:/storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz"} ```
@tongli I ran this command ```ansible-playbook -vvv -i run/runhosts.tpl -e "mode=apply env=azure env_type=flanneld ansible_ssh_user=hyperchain" initcluster.yml
``` But I got this error `fatal: [52.230.121.199]: FAILED! => {"failed": true, "msg": "ERROR! file or module does not exist: /home/rtcin/go/src/github.com/hyperledger/hyperchain/cello/src/agent/ansible/roles/env_flanneld/https:`/storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz"} ```
@tongli I ran this command ```ansible-playbook -vvv -i run/runhosts.tpl -e "mode=apply env=azure env_type=flanneld ansible_ssh_user=hyperchain" initcluster.yml
``` But I got this error `fatal: [52.230.121.199]: FAILED! => {"failed": true, "msg": "ERROR! file or module does not exist: /home/rtcin/go/src/github.com/hyperledger/hyperchain/cello/src/agent/ansible/roles/env_flanneld/https:``/storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz"} `
@tongli I ran this command ```ansible-playbook -vvv -i run/runhosts.tpl -e "mode=apply env=azure env_type=flanneld ansible_ssh_user=hyperchain" initcluster.yml
``` But I got this error `fatal: [52.230.121.199]: FAILED! => {"failed": true, "msg": "ERROR! file or module does not exist: /home/rtcin/go/src/github.com/hyperledger/hyperchain/cello/src/agent/ansible/roles/env_flanneld/https:/storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz"} `
@tongli I ran this command ```ansible-playbook -vvv -i run/runhosts.tpl -e "mode=apply env=azure env_type=flanneld ansible_ssh_user=hyperchain" initcluster.yml
``` But I got this error ` task path: /home/rtcin/go/src/github.com/hyperledger/hyperchain/cello/src/agent/ansible/roles/env_flanneld/fastinitnode/tasks/apply.yml:48....... fatal: [52.230.121.199]: FAILED! => {"failed": true, "msg": "ERROR! file or module does not exist: /home/rtcin/go/src/github.com/hyperledger/hyperchain/cello/src/agent/ansible/roles/env_flanneld/https:/storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz"} `
@tongli I ran this command ```ansible-playbook -vvv -i run/runhosts.tpl -e "mode=apply env=azure env_type=flanneld ansible_ssh_user=hyperchain" initcluster.yml
``` But I got this error ``` task path: /home/rtcin/go/src/github.com/hyperledger/hyperchain/cello/src/agent/ansible/roles/env_flanneld/fastinitnode/tasks/apply.yml:48....... fatal: [52.230.121.199]: FAILED! => {"failed": true, "msg": "ERROR! file or module does not exist: /home/rtcin/go/src/github.com/hyperledger/hyperchain/cello/src/agent/ansible/roles/env_flanneld/https:/storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz"} ```
@tongli I ran this command `ansible-playbook -vvv -i run/runhosts.tpl -e "mode=apply env=azure env_type=flanneld ansible_ssh_user=hyperchain" initcluster.yml
` But I got this error ``` task path: /home/rtcin/go/src/github.com/hyperledger/hyperchain/cello/src/agent/ansible/roles/env_flanneld/fastinitnode/tasks/apply.yml:48....... fatal: [52.230.121.199]: FAILED! => {"failed": true, "msg": "ERROR! file or module does not exist: /home/rtcin/go/src/github.com/hyperledger/hyperchain/cello/src/agent/ansible/roles/env_flanneld/https:/storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz"} ```
@tongli I ran this command `ansible-playbook -vvv -i run/runhosts.tpl -e "mode=apply env=azure env_type=flanneld ansible_ssh_user=hyperchain" initcluster.yml` But I got this error ``` task path: /home/rtcin/go/src/github.com/hyperledger/hyperchain/cello/src/agent/ansible/roles/env_flanneld/fastinitnode/tasks/apply.yml:48....... fatal: [52.230.121.199]: FAILED! => {"failed": true, "msg": "ERROR! file or module does not exist: /home/rtcin/go/src/github.com/hyperledger/hyperchain/cello/src/agent/ansible/roles/env_flanneld/https:/storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz"} ```
this file `golang/go1.8.3.linux-amd64.tar.gz` is present in `/cello/src/agent/ansible/roles/env_flanneld` directory
haha i successfully setup la thz ever y
haha i successfully setup la thz everyone
Has joined the channel.
Hi all
is it possible to configure cello in windows environment?
install virtual box
thank you @ctyu .
Has joined the channel.
@tongli this is the complete errror log
```TASK [Retrive golang binary and unpackage it] **********************************
task path: /home/rtcin/go/src/github.com/hyperledger/hyperchain/cello/src/agent/ansible/roles/env_flanneld/fastinitnode/tasks/apply.yml:48
Has joined the channel.
@harsha @baohua seems cello 0.8.0-alpha is not followed the release process.. I see cello release tag is 0.8.0-alpha, but I see VERSION=0.8.0 and IS_RELEASE = false in cello makefile https://github.com/hyperledger/cello/blob/master/Makefile#L35
When you create a release tag, you have to make sure IS_RELEASE=true and you have to create the release tag mentioned in the Makefile..
I see Docker tag has been pushed like this because of the IS_RELEASE = false https://hub.docker.com/r/hyperledger/cello-mongo/tags/
@rameshthoomu Agree with you.
@baohua We should update the flags accordingly as pointed out.
yes, @harsha let's update the doc and the dockerfiles for the releases. I look forward to seeing we can follow the new process for the v0.8.0 release.
@DarshanBc looks like that your server can not find where golang is. I had that issue when I was in Lisbon. For some reason the Europe countries have different location for downloading go, if you are in us, then this should not be an issue. You need to verify if your server can access that location.
Ok I'll see to it tomorrow and let you know
hi
just want to know if i have already use ansible to build a network , how can i plug the network to cello?
@baohua @rameshthoomu https://gerrit.hyperledger.org/r/17051 -- Updates the required flag,
FWIW, will be pushing another PR to reset the IS_RELEASE flag.
@baohua @rameshthoomu https://gerrit.hyperledger.org/r/17051 -- Updates the required flag,
FWIW, will be pushing another PR to reset the IS_RELEASE flag on master code
@harsha I have captured fabric release process here https://github.com/hyperledger/ci-management/blob/master/docs/Release_Process.md.. required some fine tuning to this but that's the overall process
@rameshthoomu Thank you !!
Has joined the channel.
hi all
is there any way that I could specify the nodes to deploy across multiple availability zones within the same region?
hi all
is there any way that I could specify the nodes to be deployed across multiple availability zones within the same region?
https://gerrit.hyperledger.org/r/#/c/17065/ -- Request for review.
@baohua Is there a process to update/refresh http://cello.readthedocs.io/ ?
yes, the doc will be updated periodically.
usually several days
@HL https://github.com/hyperledger/cello/blob/master/src/agent/ansible/roles/cloud_aws/provision/tasks/apply.yml
take a look
@ctyu thx!
@baohua Thank you. Looks like one-off update to http://cello.readthedocs.io did the charm
yeap, i didn't find way to auto-trigger that yet... but we regularly update...so, work around now
has anyone here successfully connected hyperledger composer to an active chain?
Has joined the channel.
Has joined the channel.
Request for review on the following :-
https://gerrit.hyperledger.org/r/17099 -- Release Process
https://gerrit.hyperledger.org/r/17051/ -- Update Makefile to comply with release tags
https://gerrit.hyperledger.org/r/17065/ -- Update Dockerfiles for v0.8.0-alpha
Request for review on the following :-
https://gerrit.hyperledger.org/r/17099 -- Release Process
https://gerrit.hyperledger.org/r/17051/ -- Update Makefile to comply with release tags
https://gerrit.hyperledger.org/r/17065/ -- Update Dockerfiles for v0.8.0-alpha -- Needs one more +2
Has joined the channel.
thanks @harsha
add some comments
Has joined the channel.
Hi when I run `ansible-playbook -vvv -i run/runhosts.tpl -e "mode=apply env=azure env_type=flanneld ansible_ssh_user=hyperchain" initcluster.yml`
I am getting this error
```
I think the problem is cello script is considering URL as file path which doesn't exist at all
In this `/home/rtcin/go/src/github.com/hyperledger/cello/src/agent/ansible/roles/env_flanneld/https:/storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz` its all URL after `/home/rtcin/go/src/github.com/hyperledger/cello/src/agent/ansible/roles/env_flanneld/`
https://gerrit.hyperledger.org/r/#/c/17065/ -- this needs Submit/Merge
done!
@tongli may help give comments on ur problem @DarshanBc
@baohua request for your review on https://gerrit.hyperledger.org/r/#/c/17051/
Request for review https://gerrit.hyperledger.org/r/#/c/17099/
Has joined the channel.
hi all
I'm trying to deploy a cello fabric on AWS following the cello wiki
it keep failing at this step :
just before i have 1 aws EC2 instance acting as installer
`
TASK [Gathering Facts] ************************************************************************************************************
fatal: [52.14.132.254]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Warning: Permanently added '52.14.132.254' (ECDSA) to the list of known hosts.\r\nPermission denied (publickey).\r\n", "unreachable": true}
fatal: [18.218.192.105]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Warning: Permanently added '18.218.192.105' (ECDSA) to the list of known hosts.\r\nPermission denied (publickey).\r\n", "unreachable": true}
fatal: [18.221.46.119]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Warning: Permanently added '18.221.46.119' (ECDSA) to the list of known hosts.\r\nPermission denied (publickey).\r\n", "unreachable": true} `
```
```
TASK [Gathering Facts] ************************************************************************************************************
fatal: [52.14.132.254]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Warning: Permanently added '52.14.132.254' (ECDSA) to the list of known hosts.\r\nPermission denied (publickey).\r\n", "unreachable": true}
fatal: [18.218.192.105]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Warning: Permanently added '18.218.192.105' (ECDSA) to the list of known hosts.\r\nPermission denied (publickey).\r\n", "unreachable": true}
fatal: [18.221.46.119]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Warning: Permanently added '18.221.46.119' (ECDSA) to the list of known hosts.\r\nPermission denied (publickey).\r\n", "unreachable": true}
```
I'm able to ssh using ubunut and fd key
```ssh -i ~/.ssh/fd ubuntu@18.221.46.119
Welcome to Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-1049-aws x86_64)
```
any idea why .. is it the ansible file ?
ok fixed it
thanks
ip:7050 ip:8080 ip:8081
ok now it is working but i can't acces either of REST api interface
like dashboard or any restapi from external ip:7050 or ip:8080 ip:8081
I opened all ports on AWS
how do i link between the dashboard the aws cluster
i see add host but they exist already
https://gerrit.hyperledger.org/r/#/c/17099/
Has joined the channel.
Hi @tongli this is the complete error ```TASK [Retrive golang binary and unpackage it] **********************************
task path: /home/rtcin/go/src/github.com/hyperledger/hyperchain/cello/src/agent/ansible/roles/env_flanneld/fastinitnode/tasks/apply.yml:48
<52.230.11.209> ESTABLISH SSH CONNECTION FOR USER: hyperchain
<52.230.11.209> SSH: EXEC ssh -C -q -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=hyperchain -o ConnectTimeout=10 -o ControlPath=/home/rtcin/.ansible/cp/ansible-ssh-%h-%p-%r -tt 52.230.11.209 '( umask 22 && mkdir -p "$( echo $HOME/.ansible/tmp/ansible-tmp-1516608420.49-127169744664202 )" && echo "$( echo $HOME/.ansible/tmp/ansible-tmp-1516608420.49-127169744664202 )" )'
<52.230.11.209> ESTABLISH SSH CONNECTION FOR USER: hyperchain
<52.230.11.209> SSH: EXEC ssh -C -q -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=hyperchain -o ConnectTimeout=10 -o ControlPath=/home/rtcin/.ansible/cp/ansible-ssh-%h-%p-%r -tt 52.230.11.209 '/bin/sh -c '"'"'sudo -H -S -n -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-mpgibfkxyljaxvrujpbtptppmjpaxtht; rc=flag; [ -r /opt ] || rc=2; [ -f /opt ] || rc=1; [ -d /opt ] && rc=3; python -V 2>/dev/null || rc=4; [ x"$rc" != "xflag" ] && echo "${rc} "/opt && exit 0; (python -c '"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'import hashlib; BLOCKSIZE = 65536; hasher = hashlib.sha1();
afile = open("'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'/opt'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'", "rb")
buf = afile.read(BLOCKSIZE)
while len(buf) > 0:
hasher.update(buf)
buf = afile.read(BLOCKSIZE)
afile.close()
print(hasher.hexdigest())'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"' 2>/dev/null) || (python -c '"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'import sha; BLOCKSIZE = 65536; hasher = sha.sha();
afile = open("'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'/opt'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'", "rb")
buf = afile.read(BLOCKSIZE)
while len(buf) > 0:
hasher.update(buf)
buf = afile.read(BLOCKSIZE)
afile.close()
print(hasher.hexdigest())'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"' 2>/dev/null) || (echo '"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'0 '"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'/opt)'"'"'"'"'"'"'"'"''"'"''
<52.230.11.209> PUT /home/rtcin/go/src/github.com/hyperledger/hyperchain/cello/src/agent/ansible/roles/env_flanneld/https:/storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz TO /home/hyperchain/.ansible/tmp/ansible-tmp-1516608420.49-127169744664202/source
fatal: [52.230.11.209]: FAILED! => {"failed": true, "msg": "ERROR! file or module does not exist: /home/rtcin/go/src/github.com/hyperledger/hyperchain/cello/src/agent/ansible/roles/env_flanneld/https:/storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz"}```
please take a look
https://gerrit.hyperledger.org/r/#/c/17169/
I also Tried with aws I got this error ```qgdc; LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 /usr/bin/python /home/ubuntu/.ansible/tmp/ansible-tmp-1516868541.83-22735414457251/setup; rm -rf "/home/ubuntu/.ansible/tmp/ansible-tmp-1516868541.83-22735414457251/" > /dev/null 2>&1'"'"'"'"'"'"'"'"''"'"''
fatal: [13.229.234.23]: FAILED! => {"changed": false, "failed": true, "invocation": {"module_name": "setup"}, "module_stderr": "", "module_stdout": "/bin/sh: 1: /usr/bin/python: not found\r\n", "msg": "MODULE FAILURE", "parsed": false}
fatal: [54.254.199.185]: FAILED! => {"changed": false, "failed": true, "invocation": {"module_name": "setup"}, "module_stderr": "", "module_stdout": "/bin/sh: 1: /usr/bin/python: not found\r\n", "msg": "MODULE FAILURE", "parsed": false}
```
@lujiang +2, thanks for the patchset!
anyone with insight about this
how do i link between the cello / dashboard the aws cluster or composer or even marble
i see add host but they exist already
Cello+Composer is a WIP. you need to work your way through to get it working for a marbles demo.
@harsha thank you so I'll try to walk through it did I miss it Cello dashboard should be integrated bu when i do deploy the AWS cluster there is no dashboard
how do you link both I installed dashboard on another instance
can you clarify when i do the awsansible cluster deployment it is only fabric (3 instances AWS) it doesn't deploy any of the dashboard, cmposer, explorer
but i can add these on another instance or one of the 3 instance it is not clear in the docs how you hook between these elements
Has left the channel.
I am trying to setup kubernetes cluster in aws with this command `ansible-playbook -e "mode=apply" awsk8s.yml`
I am getting this error
```
TASK [setup] *******************************************************************
ok: [cloud]
TASK [include] *****************************************************************
fatal: [cloud]: FAILED! => {"failed": true, "reason": "ERROR! no action detected in task\n\nThe error appears to have been in '/home/darsh/Desktop/cello/src/agent/ansible/roles/cloud_aws/prepare/tasks/apply.yml': line 36, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Gather vpc facts\n ^ here\n"}
PLAY RECAP *********************************************************************
cloud : ok=3 changed=0 unreachable=0 failed=1
```
this is my /var/aws.yml file
```
---
# AWS Keys will be use to provision EC2 instances on AWS Cloud
auth: {
auth_url: "",
# This should be your AWS Access Key ID
username: "AKIAJHXAEY3WIHZRZLYA",
# This should be your AWS Secret Access Key
# can be passed as part of cmd line when running the playbook
password: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
}
# These variable defines AWS cloud provision attributes
cluster: {
region_name: "ap-southeast-1", #TODO Dynamic fetch
availability_zone: "", #TODO Dynamic fetch based on region
security_group: "launch-wizard-1",
target_os: "ubuntu",
image_name: "ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-*",
image_id: "ami-325d2e4e",
flavor_name: "t2.medium", # "m2.medium" is big enough for Fabric
ssh_user: "ubuntu",
validate_certs: True,
private_net_name: "demonet",
public_key_file: "/home/darsh/.ssh/aws_keys.pub",
private_key_file: "/home/darsh/.ssh/aws_keys",
ssh_key_name: "fabric",
# This variable indicate what IP should be used, only valid values are
# private_ip or public_ip
node_ip: "public_ip",
container_network: {
Network: "172.16.0.0/16",
SubnetLen: 24,
SubnetMin: "172.16.0.0",
SubnetMax: "172.16.255.0",
Backend: {
Type: "udp",
Port: 8285
}
},
service_ip_range: "172.15.0.0/24",
dns_service_ip: "172.15.0.4",
# the section defines preallocated IP addresses for each node, if there is no
# preallocated IPs, leave it blank
node_ips: [ ],
# fabric network node names expect to be using a clear pattern, this defines
# the prefix for the node names.
name_prefix: "fabric",
domain: "fabricnet",
# stack_size determines how many virtual or physical machines we will have
# each machine will be named ${name_prefix}001 to ${name_prefix}${stack_size}
stack_size: 3,
etcdnodes: ["fabric001", "fabric002", "fabric003"],
builders: ["fabric001"],
flannel_repo: "https://github.com/coreos/flannel/releases/download/v0.7.1/flannel-v0.7.1-linux-amd64.tar.gz",
etcd_repo: "https://github.com/coreos/etcd/releases/download/v3.2.0/etcd-v3.2.0-linux-amd64.tar.gz",
k8s_repo: "https://storage.googleapis.com/kubernetes-release/release/v1.7.0/bin/linux/amd64/",
go_ver: "1.8.3",
# If volume want to be used, specify a size in GB, make volume size 0 if wish
# not to use volume from your cloud
volume_size: 8,
# cloud block device name presented on virtual machines.
block_device_name: "/dev/vdb"
}```
I am niether able to come up with the server in aws nor in azure with already existing servers or newly created servers if anyone is able to help me resolve errors here?
Has joined the channel.
Someone know if in the future versions of Cello, will be the possibility to install a fabric with CouchDB? Thanks for the answers!
Someone knows if in the future versions of Cello, will be the possibility to install a fabric with CouchDB? Thanks for the answers!
Someone knows if in the future versions of Cello will have the possibility to install a fabric with CouchDB? Thanks for the answers!
isn't aws cluster is already with couchDB
at least the one i deployed yesterday
[ ](https://chat.hyperledger.org/channel/cello?msg=SShdFmTAdiYuDQuDb) @gesaleh assuming you followed http://cello.readthedocs.io/en/latest/awsinstall/#deploy-hyperledger-cello-on-aws-ec2 and assuming you are latest master code of cello, you should have `only` cello and blockchain-explorer should be deployed
[ ](https://chat.hyperledger.org/channel/cello?msg=mjokXGYaGnvJnMvT6) @harsha I followed this http://cello.readthedocs.io/en/latest/setup_worker_ansible/
[ ](https://chat.hyperledger.org/channel/cello?msg=mjokXGYaGnvJnMvT6) @harsha
On AWS cloud::
1. make changes to vars/aws.yml according to your aws cloud
2. export AWS_SECRET_KEY="your secret key of your aws account"
3. To stand up the fabric network::
ansible-playbook -e "mode=apply" aws.yml
4. To tear down the fabric network::
ansible-playbook -e "mode=destroy" aws.yml
i assume i have only workers
There you go no cello dashboard will pop-up following that ;)
[ ](https://chat.hyperledger.org/channel/cello?msg=7zobdACmfxGtwxiNP) @ascatox Worth a feature enhancement please do rise a JIRA item.
[ ](https://chat.hyperledger.org/channel/cello?msg=7zobdACmfxGtwxiNP) @ascatox Worth a feature enhancement please do open a JIRA item.
@harsha this means i need a master unit right ? or i need to follow the other link ? and then how you link these works to master one
this is what i did I have an instance with
```docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
47a7ba5691fd hyperledger/cello-user-dashboard "bash -c 'cd /usr/ap…" 15 hours ago Up 15 hours 0.0.0.0:8081->8080/tcp user-dashboard
dda16a7e5d39 hyperledger/cello-watchdog "python watchdog.py" 15 hours ago Up 15 hours watchdog
3d6be220684a hyperledger/cello-mongo "docker-entrypoint.s…" 15 hours ago Up 15 hours 127.0.0.1:27017-27018->27017-27018/tcp mongo
f4099805c1e3 hyperledger/cello-mongo "docker-entrypoint.s…" 15 hours ago Up 15 hours 27017/tcp cello_dashboard_mongo_1
e20fffdae9f8 hyperledger/cello-dashboard "python dashboard.py" 15 hours ago Up 15 hours 8080/tcp dashboard
b9720eec3772 hyperledger/cello-restserver "python restserver.py" 15 hours ago Up 15 hours 80/tcp restserver
dbdb4087bdce hyperledger/cello-nginx "/bin/bash /tmp/dock…" 15 hours ago Up 15 hours 0.0.0.0:80->80/tcp, 0.0.0.0:8080->8080/tcp nginx
```
which means dashboard is there but when i give one of the addresses of the worker it doesnt work
Is this O/P from your worker node ?
From the system where you running `docker ps`, identify it's IP and from your browser hit, `
[ ](https://chat.hyperledger.org/channel/cello?msg=BBrNSsth6eWjDqjdn) @harsha i get ``` http://
8081 is open at aws
at 8080 it works
My bad 8080 it is :)
ok so i access 8080
but when i do add host it is not working it see nothing
Assuming you are adding worker node on which docker is running at 2375, you should be doing, `IPADDR:2375`
this is my issue none of the aws deployed ansible machine is running 2375
```docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2bfa547805f7 hyperledger/fabric-couchdb:x86_64-1.0.5 "tini -- /docker-e..." 18 hours ago Up 18 hours 4369/tcp, 5984/tcp, 9100/tcp couchdb-peer3rd-orga
469346f5d8dc hyperledger/fabric-peer:x86_64-1.0.5 "peer node start" 18 hours ago Up 18 hours peer3rd-orgb
5e9f6585c7ab hyperledger/fabric-couchdb:x86_64-1.0.5 "tini -- /docker-e..." 18 hours ago Up 18 hours 4369/tcp, 5984/tcp, 9100/tcp couchdb-peer3rd-orgb
4ba6894c2cd9 hyperledger/fabric-peer:x86_64-1.0.5 "peer node start" 18 hours ago Up 18 hours peer3rd-orga
789889df68cd hyperledger/fabric-kafka:x86_64-1.0.5 "/docker-entrypoin..." 18 hours ago Up 18 hours 9092-9093/tcp kafka3rd
b3c8809d29fb hyperledger/fabric-zookeeper:x86_64-1.0.5 "/docker-entrypoin..." 18 hours ago Up 18 hours 2181/tcp, 2888/tcp, 3888/tcp zookeeper3rd
8e5fb7a329a4 hyperledger/fabric-ca:x86_64-1.0.5 "fabric-ca-server ..." 18 hours ago Up 18 hours 7054/tcp ca1st-orgc
82e0b0567fad hyperledger/fabric-ca:x86_64-1.0.5 "fabric-ca-server ..." 18 hours ago Up 18 hours 7054/tcp ca1st-orgd
19b1e90a3c87 email4tong/registrator:v10 "/bin/registrator ..." 18 hours ago Up 18 hours registrator.fabric003
```
this is what i have on each instance created by ansible aws
``` docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
52600b585919 dev-peer1st-orga-firstchaincode-1.0-39aeac52e0b4ec021e4ea36fb80328cb4f49c84e86d511c4134039e543844dde "chaincode -peer.a..." 18 hours ago Up 18 hours dev-peer1st-orga-firstchaincode-1.0
f4a5ebe0a763 hyperledger/fabric-couchdb:x86_64-1.0.5 "tini -- /docker-e..." 18 hours ago Up 18 hours 4369/tcp, 5984/tcp, 9100/tcp couchdb-peer1st-orgb
4be5be71c973 hyperledger/fabric-couchdb:x86_64-1.0.5 "tini -- /docker-e..." 18 hours ago Up 18 hours 4369/tcp, 5984/tcp, 9100/tcp couchdb-peer1st-orga
6f4338465d71 hyperledger/fabric-peer:x86_64-1.0.5 "peer node start" 18 hours ago Up 18 hours peer1st-orgb
07bea6d2e4b1 hyperledger/fabric-peer:x86_64-1.0.5 "peer node start" 18 hours ago Up 18 hours peer1st-orga
2ca2a05009b9 hyperledger/fabric-orderer:x86_64-1.0.5 "orderer" 18 hours ago Up 18 hours 7050/tcp orderer1st-orgd
deed71b3ef6c hyperledger/fabric-orderer:x86_64-1.0.5 "orderer" 18 hours ago Up 18 hours 7050/tcp orderer1st-orgc
ef93ed9263cc hyperledger/fabric-kafka:x86_64-1.0.5 "/docker-entrypoin..." 18 hours ago Up 18 hours 9092-9093/tcp kafka1st
9a9ea016f15a hyperledger/fabric-zookeeper:x86_64-1.0.5 "/docker-entrypoin..." 18 hours ago Up 18 hours 2181/tcp, 2888/tcp, 3888/tcp zookeeper1st
57f357736ccb hyperledger/fabric-ca:x86_64-1.0.5 "fabric-ca-server ..." 18 hours ago Up 18 hours 7054/tcp ca1st-orga
ed40ff349033 email4tong/registrator:v10 "/bin/registrator ..." 18 hours ago Up 18 hours registrator.fabric001
84914394c4c9 skynetservices/skydns:2.5.3a "/skydns -machines..." 18 hours ago Up 18 hours skydns.fabric001
```
```CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
45e37fefc583 hyperledger/fabric-couchdb:x86_64-1.0.5 "tini -- /docker-e..." 18 hours ago Up 18 hours 4369/tcp, 5984/tcp, 9100/tcp couchdb-peer2nd-orgb
173f01529125 hyperledger/fabric-couchdb:x86_64-1.0.5 "tini -- /docker-e..." 18 hours ago Up 18 hours 4369/tcp, 5984/tcp, 9100/tcp couchdb-peer2nd-orga
015e1fb9d9e6 hyperledger/fabric-peer:x86_64-1.0.5 "peer node start" 18 hours ago Up 18 hours peer2nd-orgb
67ef0bd3741f hyperledger/fabric-peer:x86_64-1.0.5 "peer node start" 18 hours ago Up 18 hours peer2nd-orga
b61cde8f9b54 hyperledger/fabric-orderer:x86_64-1.0.5 "orderer" 18 hours ago Up 18 hours 7050/tcp orderer2nd-orgd
970ed70b0129 hyperledger/fabric-orderer:x86_64-1.0.5 "orderer" 18 hours ago Up 18 hours 7050/tcp orderer2nd-orgc
a7e0a574466c hyperledger/fabric-kafka:x86_64-1.0.5 "/docker-entrypoin..." 18 hours ago Up 18 hours 9092-9093/tcp kafka2nd
8a8fb382959d hyperledger/fabric-zookeeper:x86_64-1.0.5 "/docker-entrypoin..." 18 hours ago Up 18 hours 2181/tcp, 2888/tcp, 3888/tcp zookeeper2nd
4f1abd89f861 hyperledger/fabric-ca:x86_64-1.0.5 "fabric-ca-server ..." 18 hours ago Up 18 hours 7054/tcp ca1st-orgb
9430411e0cf7 email4tong/registrator:v10 "/bin/registrator ..." 18 hours ago Up 18 hours registrator.fabric002
```
as you can see the deployment worked but I might missed something it is really not clear how these get hooked to the cello dashboard or anything else like explorer or composer
No, not that, `docker demon` as such.
http://cello.readthedocs.io/en/latest/setup_worker_docker/#docker-daemon-setup
let me try to change one fabric
[ ](https://chat.hyperledger.org/channel/cello?msg=2KvefeBauToMrW69N) FYI, CE-250
@DarshanBc If you can allocate a block of time, then we can work together on this.
I am on US East coast. The information you provided is not enough for me to go on. need to know if you have successfully created VMs on AWS, if they can communicate, if the overlay network is ok. The message you posted there does not really tell me much.
@DarshanBc from your earlier message, it seems to me that you are having trouble to get golang packages. not sure why, but you can setup apache2 very easily to host golang package. you can download the golang package and placed on your apache2 server, then use the url in your configuration file, doing that is to pass the golang download issue. I do not know why you are having trouble to do that. I know that in Europe countries, golang download sites get redirected somehow.
any way, allocate a block of time which I can attend, then we can go step by step.
@hightall @baohua please review this, it is to make changes according to the recent fabric changes in capabilities. https://gerrit.hyperledger.org/r/#/c/17103/
@tongli capacity is introduced in fabric 1.1, correct?
@baohua that is right, 1.1 need the capabilities in the configtx file.
without it, chaincode wont instantiate.
the added code will detect if the code level is above 1.1 (version_compare), if it is, then it will add the capabilities, otherwise, as before.
it will work for all levels.
@harsha ok i restarted docker with 2375 but it doesn't see any of the cluster chain i can add the host
yes, and now 1.1-alpha has not been released, so i'm afraid we do not have the official code to test the feature?
we are using the head to test.
if you are using ansible agent, you can set a commit level to test every commit.
i hope there will be documentation to guide users for that, certainly can be added later.
we need a bit documentation to explain the yaml configuration file.
let me create a jira issue for that.
will add
+2!
@DarshanBc are you available now to get it started?
Hi @tongli as you suggested I will edit bc1st.yml and run/runhosts file for hostname and will try to bring up the pods thank you
more i read the docs more it is confusing : you provide ansible aws that deploy a clsuter of 3 (I assume this are workers) then when i check the docker is running normally with all instance but it is not running on 2375 port , if i change this to that port i can add the host at the master node but still it doesn't see the fabric chains , why it is not a discovery mode
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=2KvefeBauToMrW69N) @harsha Can you give me a direct link for Cello JIRA? Thanks in advance!!!
@gesaleh not sure what you are referring and what you are trying to do.
[ ](https://chat.hyperledger.org/channel/cello?msg=5LfoWRaNhx2HYjoLP) @tongli Hi let me explain :
I wanted to deploy a cluster on aws : i used ansible aws that deployed 3 instances with blockchain , then looking at cello i thought i'll have access to dashboard to see my instances and chain
this wasn't the case since docker with ansible aws doesn't listen on 2375 port
i thought these instances are workers and when i bring the master cello it should see the workers
@gesaleh ansible agent does not currently support cello dashboard.
@gesaleh it sets up the multiple node cluster.
ok it is clear
@tongli but i can do the same manually from cello dashboard right ?
and ansible cluster i can deploy on top explorer composer or marbles right
right. you can use command line tools to do these things.
but the organizations will be different.
using the ansible agent deploys the organizations and many peers and orderers based on vars/bc1st.yml file.
you may change that and redeploy.
ok so if i chnage that bc1st to have other org
and then deploy on top of it composer explorer or marbles
in fact i want to test creating chaincode on the fly if possible
like a new smartcontract needed i can add it to the chain
sure. you can do that against that cluster.
[ ](https://chat.hyperledger.org/channel/cello?msg=RQKwnHJohTBxsDQPy) @tongli [ ](https://chat.hyperledger.org/channel/cello?msg=RQKwnHJohTBxsDQPy) @tongli I'm getting this error (permission denied) while testing cello all in one server (http://hyperledger-cello.readthedocs.io/en/latest/ansible-agent-all-in-one/) ? even after setting ssh-agent. It is also bit confusing for me, I'm adding my own public key to my own authorized_keys list, why is that ? thank you
@Amir you can ssh to your own host manually and see if you can do that. if you can do that, then the script will be able to do that.
@Amir you can also post the msg here so that I can see if it is ssh issue or some other issues.
error messages
@Amir dinner time, you can post the error msg here, I can take a look later.
@tongli Thanks fot the reply, yes I can ssh to my self (I tried that now) but still the cello is not starting
OK, I will post the error msg, Thank you
Screenshot from 2018-01-26 15_08_02.png
@Amir what is the command you run to produce that error? I am pretty sure you did not change the something in that file.
the error was from this command 'ansible-playbook -i run/runhosts -e "mode=apply env=vb" initcluster.yml --skip-tags="resetconn" ' I followed the example from here (http://hyperledger-cello.readthedocs.io/en/latest/ansible-agent-all-in-one)
I also checked the "run/runhosts" file it contains my IP, which is correct I assume
@tongli I think the error was my user name, in /var/vb.yml it expects user name "ubuntu", I was trying with root. when I change the value ssh_user: "ubuntu", to root it seems working. Thank you for the help
Hello
I get this error when running setupfabric.yml for "bc2nd" env at Make targets task.
```TASK [Make targets] ******************************************************************
failed: [172.31.27.246] (item=cryptogen) => {"changed": true, "cmd": "make cryptogen >> /opt/gopath/bc2nd/build.log", "delta": "0:00:01.881681", "end": "2018-01-27 11:56:03.058493", "item": "cryptogen", "msg": "non-zero return code", "rc": 2, "start": "2018-01-27 11:56:01.176812", "stderr": "# runtime\n/opt/go/src/runtime/float.go:45: float64bits redeclared in this block\n\tprevious declaration at /opt/go/src/runtime/fastlog2.go:33\n/opt/go/src/runtime/mgclarge.go:142: mheap_.treapalloc undefined (type mheap has no field or method treapalloc)\n/opt/go/src/runtime/mgclarge.go:194: mheap_.treapalloc undefined (type mheap has no field or method treapalloc)\n/opt/go/src/runtime/mgclarge.go:251: undefined: scavengeTreapNode\n/opt/go/src/runtime/proflabel.go:34: getg().labels undefined (type *g has no field or method labels)\n/opt/go/src/runtime/proflabel.go:39: getg().labels undefined (type *g has no field or method labels)\n/opt/go/src/runtime/timestub.go:18: time_now redeclared in this block\n\tprevious declaration at /opt/go/src/runtime/stubs.go:245\n/opt/go/src/runtime/timestub.go:20: undefined: startNano\nmake: *** [build/bin/cryptogen] Error 2", "stderr_lines": ["# runtime", "/opt/go/src/runtime/float.go:45: float64bits redeclared in this block", "\tprevious declaration at /opt/go/src/runtime/fastlog2.go:33", "/opt/go/src/runtime/mgclarge.go:142: mheap_.treapalloc undefined (type mheap has no field or method treapalloc)", "/opt/go/src/runtime/mgclarge.go:194: mheap_.treapalloc undefined (type mheap has no field or method treapalloc)", "/opt/go/src/runtime/mgclarge.go:251: undefined: scavengeTreapNode", "/opt/go/src/runtime/proflabel.go:34: getg().labels undefined (type *g has no field or method labels)", "/opt/go/src/runtime/proflabel.go:39: getg().labels undefined (type *g has no field or method labels)", "/opt/go/src/runtime/timestub.go:18: time_now redeclared in this block", "\tprevious declaration at /opt/go/src/runtime/stubs.go:245", "/opt/go/src/runtime/timestub.go:20: undefined: startNano", "make: *** [build/bin/cryptogen] Error 2"], "stdout": "", "stdout_lines": []}
failed: [172.31.27.246] (item=configtxgen) => {"changed": true, "cmd": "make configtxgen >> /opt/gopath/bc2nd/build.log", "delta": "0:00:01.642085", "end": "2018-01-27 11:56:04.848553", "item": "configtxgen", "msg": "non-zero return code", "rc": 2, "start": "2018-01-27 11:56:03.206468", "stderr": "# runtime\n/opt/go/src/runtime/float.go:45: float64bits redeclared in this block\n\tprevious declaration at /opt/go/src/runtime/fastlog2.go:33\n/opt/go/src/runtime/mgclarge.go:142: mheap_.treapalloc undefined (type mheap has no field or method treapalloc)\n/opt/go/src/runtime/mgclarge.go:194: mheap_.treapalloc undefined (type mheap has no field or method treapalloc)\n/opt/go/src/runtime/mgclarge.go:251: undefined: scavengeTreapNode\n/opt/go/src/runtime/proflabel.go:34: getg().labels undefined (type *g has no field or method labels)\n/opt/go/src/runtime/proflabel.go:39: getg().labels undefined (type *g has no field or method labels)\n/opt/go/src/runtime/timestub.go:18: time_now redeclared in this block\n\tprevious declaration at /opt/go/src/runtime/stubs.go:245\n/opt/go/src/runtime/timestub.go:20: undefined: startNano\nmake: *** [build/bin/configtxgen] Error 2", "stderr_lines": ["# runtime", "/opt/go/src/runtime/float.go:45: float64bits redeclared in this block", "\tprevious declaration at /opt/go/src/runtime/fastlog2.go:33", "/opt/go/src/runtime/mgclarge.go:142: mheap_.treapalloc undefined (type mheap has no field or method treapalloc)", "/opt/go/src/runtime/mgclarge.go:194: mheap_.treapalloc undefined (type mheap has no field or method treapalloc)", "/opt/go/src/runtime/mgclarge.go:251: undefined: scavengeTreapNode", "/opt/go/src/runtime/proflabel.go:34: getg().labels undefined (type *g has no field or method labels)", "/opt/go/src/runtime/proflabel.go:39: getg().labels undefined (type *g has no field or method labels)", "/opt/go/src/runtime/timestub.go:18: time_now redeclared in this block", "\tprevious declaration at /opt/go/src/runtime/stubs.go:245", "/opt/go/src/runtime/timestub.go:20: undefined: startNano", "make: *** [build/bin/configtxgen] Error 2"], "stdout": "", "stdout_lines": []}
```
I am setting up the "all in one ansible" option
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=6aHxPzAKozQGE8mhy) Solved! I installed more than one version of Go.
I am unable to add an active chain
my host is a single docker host
Keeps failing
what's the steps and error log msg?
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=8AfjDZpHXfEHZyZvH) @tongli do you mean i need a new cluster each time of just a docker with the chaincode
[ ](https://chat.hyperledger.org/channel/cello?msg=uhWKQpZKqewT7fvWt) @baohua Unable to pull hyperledger/fabric-ca:1.0.5 from repository. Says: `manifest for hyperledger/fabric-ca:1.0.5`. The thing, i don't think 1.0.5 is a valid tag. Where do i fix this? It needs to be manifest for hyperledger/fabric-ca:
pls follow the docs to run `make setup-worker` first
did
Still didn't work
I then proceeded to setup a docker worker node. Still didn't work. Same error
It seems to omitted architecture in the tag name for the docker images
It seems to omit architecture in the tag name for the docker images
@gesaleh What I mean is this, once the cluster is up running, you can install whatever chaincode you like onto that cluster and start your transactions.
@hightall @baohua please take a look at this patch to update docker and go version, also moving up default fabric version to 1.1.0-preview. https://gerrit.hyperledger.org/r/#/c/17285/
@tongli would u like to keep the fabric 1.0.0 support, together with the coming 1.1.0 release? people are possible to use 1.0.0 release for quite a while until the 1.1.x become stable.
```[2018-01-29 01:06:13,107] DEBUG [resources.login] [utils.py:148 request_debug()] - path=/login, method=GET
[2018-01-29 01:06:13,108] DEBUG [resources.login] [utils.py:149 request_debug()] - request args:
[2018-01-29 01:06:13,109] DEBUG [resources.login] [utils.py:152 request_debug()] - request form:
[2018-01-29 01:06:13,109] DEBUG [resources.login] [utils.py:155 request_debug()] - request raw body data:
[2018-01-29 01:06:13,109] DEBUG [resources.login] [utils.py:156 request_debug()] - b''
[2018-01-29 01:06:13,110] DEBUG [resources.login] [utils.py:157 request_debug()] - None
172.18.0.4 - - [29/Jan/2018 01:06:13] "GET /login HTTP/1.0" 200 -
[2018-01-29 01:06:20,726] ERROR [modules.user.user] [user.py:99 get_by_username()] - get user exc User matching query does not exist.
[2018-01-29 01:06:20,728] INFO [modules.user.auth.login] [login.py:70 post()] - error 'NoneType' object has no attribute 'password'
172.18.0.4 - - [29/Jan/2018 01:06:20] "POST /api/auth/login HTTP/1.0" 401 -
[2018-01-29 01:06:23,078] ERROR [modules.user.user] [user.py:99 get_by_username()] - get user exc User matching query does not exist.
[2018-01-29 01:06:23,079] INFO [modules.user.auth.login] [login.py:70 post()] - error 'NoneType' object has no attribute 'password'
172.18.0.4 - - [29/Jan/2018 01:06:23] "POST /api/auth/login HTTP/1.0" 401 -
[2018-01-29 01:06:23,771] ERROR [modules.user.user] [user.py:99 get_by_username()] - get user exc User matching query does not exist.
[2018-01-29 01:06:23,779] INFO [modules.user.auth.login] [login.py:70 post()] - error 'NoneType' object has no attribute 'password'
172.18.0.4 - - [29/Jan/2018 01:06:23] "POST /api/auth/login HTTP/1.0" 401 -
[2018-01-29 01:07:57,608] ERROR [modules.user.user] [user.py:99 get_by_username()] - get user exc User matching query does not exist.
[2018-01-29 01:07:57,608] INFO [modules.user.auth.login] [login.py:70 post()] - error 'NoneType' object has no attribute 'password'
172.18.0.8 - - [29/Jan/2018 01:07:57] "POST /api/auth/login HTTP/1.1" 401 -```
what's the problem on 0.8,cant login dashboard
hi just want to know
if cello support eks
@baohua uses can change the version in the configuration file to install whatever version they prefer. This is just change the default. That is why the most of the changes are all in the files in vars directory.
@ctyu yes, we have used elk for log collection. cello is quite easy to extend with all those data analytics platforms
@tongli got, +2
@baohua haha do you free to give me a give tutorial on cello lol
i have successfully setup on awsk8s using anvisible
but i dont know how to configure the dns
and also i want to modify the source codeto support two zone
=m=
you can find some tutorials at cello.readthedocs.io. but there is no much on the elk side yet, we are adding more, welcome to help improve, too.
Has joined the channel.
i dont found any post that is related to configuring the DNS
do i need to setup route53
Has joined the channel.
Hi, after the setup of the docker worker, how can I make it start?
I managed to make it run on port 2375, but when visiting the dashboard it says "no authorization token found"
and the /login page doesn't show any login form, because all the css and js receive a 401
@tritone11 on your worker node, assuming u have cloned cello, issue make setup-worker
@baohua @hightall I have to fix a bug (I removed few extra lines from a configuration file), need to fix that. Please see the patch here. thanks https://gerrit.hyperledger.org/r/#/c/17301/
Hello, I managed to setup the master and worker nodes, and I can access the cello dashboard following this (http://hyperledger-cello.readthedocs.io/en/latest/installation_master/) but how can I deploy my chaincode (aka fabric smart contract) ? I tried to access the user dashboard from port 8081 as described here (http://cello.readthedocs.io/en/latest/user_dashboard/) but the page loads noting, not even error msg
Has joined the channel.
@tongli +2. And based on last week meetup, many people feel interested to use cello, we should also consider to enhance the documentation.
@Amir @hightall designed a user dashboard, you may wanna try. @hightall pls help see the issue, thanks!
@Amir oh, there's a little bug here, now you can use `THEME=vue make start` to start the service, or you make build-user-dashboard-js to build user-dashboard js
@hightall we should fix the problem or document the limitation.
@baohua Yes, And we need discuss about dashboard for admin, I think we need choose one theme for long term development, and drop the others, it's too heavy.
sure, what's ur suggestion?
beta release fixes can be collected at https://jira.hyperledger.org/browse/CE-256
v0.8.0-beta release fixes can be collected at https://jira.hyperledger.org/browse/CE-256
Has joined the channel.
@baohua Personal inclination, I like react :grin: maybe we can make a voting for theme selection.
[ ](https://chat.hyperledger.org/channel/cello?msg=MaZpoiupDnH3h2Dck) @baohua Thanks
[ ](https://chat.hyperledger.org/channel/cello?msg=RN8G57ERn8NgaXThf) @hightall Thank you I tried "make build-user-dashboard-js " and it is working
@baohua I would like to contribute to composer intergration, would you please assign the issue to me at jira? I can't access http://jira.hyperledger.org due to network problem.
@baohua I would like to contribute to composer intergration, would you please assign the issue to me in jira? I can't access http://jira.hyperledger.org due to network problem.
Hello, I have an issues with my running cello setup, I have a setup of a master node and two worker nodes. Whenever I create a chain the health column in the dashboard indicates "Fail" while the status indicates "running". what does the health indicates ? And I tried to see the containers created for this chain. If I used a solo consensus plugin all containers are UP (still the health shows fail) but when I use kafka the peer nodes are always on restarting mode. docker logs command for these containers shows the following error `error when setting up MSP from directory /etc/hyperledger/fabric/msp: err Could not load a valid signer certificate from directory /etc/hyperledger/fabric/msp/signcerts, err stat /etc/hyperledger/fabric/msp/signcerts: no such file or directory`
Has joined the channel.
@baohua Hi, I'm planning to use Cello to deploy Fabric on a Kubernetes cluster. Can you share the status of k8s support? The document at http://cello.readthedocs.io/en/latest/setup/#worker-node says "Currently we support Docker Host or Swarm Cluster as Worker Node. More types will be added soon." Does this mean it does not support k8s at this point?
@tkueda the support of k8s is via ansible now, and you may would like to read `docs/setup_worker_ansible.md`.
@baohua ok, many thanks. I will check it.
[ ](https://chat.hyperledger.org/channel/cello?msg=w9vmot34Ac7GEkALo) This Error was persisting because Ansibel driver was outdated Its version should be ansible>=2.3.0.0 error got resolved once I updated ansible driver thanks to @tongli
[ ](https://chat.hyperledger.org/channel/cello?msg=w9vmot34Ac7GEkALo) This Error was persisting because Ansible driver was outdated Its version should be ansible>=2.3.0.0 error got resolved once I updated ansible driver thanks to @tongli
Hi @tongli I am running this command `ansible-playbook -vvv -i run/runhosts -e "mode=apply env=azure env_type=flanneld ansible_ssh_user=hyperchain" initcluster.yml` But got this error ```TASK [Change IP Forwarding] **********************************************************************
changed: [52.230.121.199]
changed: [52.230.11.209]
TASK [Reload services] ***************************************************************************
changed: [52.230.121.199]
changed: [52.230.11.209]
TASK [Start the flanneld service] ****************************************************************
fatal: [52.230.121.199]: FAILED! => {"changed": false, "msg": "Unable to start service flanneld: Job for flanneld.service failed because a timeout was exceeded. See \"systemctl status flanneld.service\" and \"journalctl -xe\" for details.\n"}
fatal: [52.230.11.209]: FAILED! => {"changed": false, "msg": "Unable to start service flanneld: Job for flanneld.service failed because a timeout was exceeded. See \"systemctl status flanneld.service\" and \"journalctl -xe\" for details.\n"}```
you can log in to these machines and see what went wrong.
I suggest that you do this first.
```ansible-playbook -vvv -i run/runhosts -e "mode=destroy env=azure env_type=flanneld ansible_ssh_user=hyperchain" initcluster.yml```
notice the mode is now destroy.
which will actually do some cleanup.
I suspect that you have run this thing many times, you may have some bad data or files somewhere.
try run that command first,
then run your apply command.
before you run these commands, you can ssh to one of the machines and run this command to see what went wrong.
```sudo systemctl status flanneld```
that command should give you some hint what was wrong with the services.
@DarshanBc please see the msg above.
Has joined the channel.
Has joined the channel.
Hey @tongli if we have multiple computers and we have ip address of all these computers and all these computers can ping each other. In such cases we need to define those ip address in runhost file and then run that file using ansible right ?
yeah. make sure that you have python installed onto these machines since ansible rely on python for tasks.
and make sure these machines can use a ssh key for ssh communication.
you will need to do this once ssh public key is setup correctly on these machines.
```eval $(ssh-agent -s)```
ssh-add your-private-key file.
you will need to do this before you run initcluster.yml and setupfabric.yml playbooks.
@MohammadObaid see the msg above.
Hey @tongli What does fabric deployed by cello, comprises of ? I mean in terms of peers, orderer, kafka or ca etc
@saeedi cello ansible agent is a tool to stand up fabric network from zero to a working fabric network. It can provision VM from clouds such as AWS or OpenStack, then build, deploy fabric onto these nodes.
@saeedi you can choose deploy onto existing VM or physical servers as well.
but with existing servers regardless VM or physical machines, you will need to do a bit extra work before you run the ansible playbooks. these tasks include setup ssh-agent on the ansible controller which is two lines of code and inject your public ssh key onto the servers which you will use
@tongli Thanks :-)
let me know if you have any questions.
will be happy to take you through the process.
@tongli Yes sure. Thanks a lot. I will be starting with *aws* and then move to physical machines without use of cloud.
yeah. using aws is the easiest.
basically change few things in vars/aws.yml and vars/bc1st.yml file.
then fire it up by doing this
```ansible-playbook -e "mode=apply" aws.yml```
at the end you should have a working fabric cluster.
Oh okay . Thanks a lot.
@saeedi Also I have added basic explanation of some main files related to aws deployment incase you need it
:) https://medium.com/@mohammadobaid11/deploying-hyperledger-fabric-on-aws-using-ansible-c2b240eac83f
@MohammadObaid i was just starting to give that a read. Thanks a lot :-)
I have been following your work also and waiting for your next post regarding cello:smiley:
Yeah I am extremely sorry for delay. My startup is incubated in local accelerator so I was busy in that. I will complete in this coming weekend :)
@MohammadObaid good instructions.
currently we need to document to explain both vars/aws.yml and vars/bc1st.yml, I wonder if you can take that task?
the jira issue is here.
https://jira.hyperledger.org/browse/CE-253
@MohammadObaid looking forward to that. :-)
currently we need a document to detail what each field means in these two type of files
cloud/env configuration file such as vars/aws.yml vars/os.yml.
fabric network configuration file such as vars/bc1st.yml, vars/bc2nd.yml and vars/vb1st.yml.
each run needs two configuration files to specify how you want to layout your fabric network.
I had a doc before, for some reason, the doc was removed, but the content is now out dated. we need update anyway.
@MohammadObaid @saeedi any of you like to take that challenge?
@tongli I would love to, me and @MohammadObaid can work together on it. I am a newbie so i believe it will be a good learning experience as well.
great.
Yeah sure @tongli @saeedi
I will take a look in this coming weekend . Is that ok or their is urgency ?
in that case, can you login to jira to change the owner to you two?
it will be very nice to have your names on the contribution list.
I do have jira account for hyperledger but I am not sure whether I can submit task or not.
Yeah :D
https://github.com/hyperledger/cello/blob/master/docs/CONTRIBUTING.md
Great . I will give you an update very soon
the project is git clone https://gerrit.hyperledger.org/r/cello
holly cow! Red Hat bought CoreOS
@ i don't have a account on jira :sweat_smile:
@tongli i don't have a account on jira :sweat_smile:
you can register one.
it is free.
okay i will do it right now
@tongli It says that i need to contact JIRA administrators to request account
https://identity.linuxfoundation.org/
I thought that the linux foundation id getting created automatically when you register there.
@tongli Oh okay, i am looking into it
k
Yes you were right. My bad. I have logged into jira.hyperledger.org
@tongli I have one more general question, is it possible to combine *cello* and *composer*, i mean deploying *business network archive* on network deployed using cello ?
@saeedi I am not familiar with composer. cello standup a fabric network. if composer is to add apps on top of the fabric network, I do not see why it can not.
Has joined the channel.
Hello, in Cello dashboard under "/Active Chains", what does the health column indicates ? whenever I create a chain it shows "fail", how can I trace this error ? all docker containers used for that chain are up and running, thanks for the help in advance
Has joined the channel.
@baohua @hightall please review this patch https://gerrit.hyperledger.org/r/#/c/17395/
I restructured quite bit to make things a bit more efficient. and also allow cello go get the artifacts such as binaries and images from any docker hub regardless it is docker hub or a third party hub or a local hub. only needs to be configured in the configuration file.
Added code are mostly comments to make these configuration a bit clear.
Has joined the channel.
Has joined the channel.
add my comments, thanks
Has joined the channel.
@baohua the baseimage_tag is for downloading docker images for peer, orderer images, it is not the base image to build other images. It is a bit confusing but since it has been used since day one of the project, it will cause a lot of changes so I kept it. It is not the base image tag 0.4.5
Hi, I am pretty new to Cello. I am wondering if it supports Fabric 1.1.0 yet? Also, is this the best solution for setting a up multiple peer blockchain on multiple hosts?
Has joined the channel.
Clipboard - 1 февраля 2018 г., 18:43
[ ](https://chat.hyperledger.org/channel/cello?msg=nqiksHjkkyaoyc9qb) problem is solved , we using this
eval $(ssh-agent -s)
ssh-add ~/.ssh/fd
@Darumada looks like you can not ssh to the hosts. can you manually ssh to any of the hosts after you saw the errors?
if you already uses the ssh agent, the only thing I can think of maybe the hosts do not have the public key? If you are doing this on AWS, then these keys should have been injected automatically.
@baohua @hightall please take a look at the patch, I responded to baohua's comment. Thanks. https://gerrit.hyperledger.org/r/#/c/17395
how to add chaincode in ansible agent?
[ ](https://chat.hyperledger.org/channel/cello?msg=nfzJfjWNARR627nYK) @william123 Not yet on fabric 1.1.0 yet, it is not released :)
@tongli will u be able to attend the LA hackfest? https://docs.google.com/document/d/14yNuJsFEWnVsAclKb1QjPluRKgz3rIpaApRSppJj6to/edit
@baohua I will be there.
Already booked my flight.
great, would you like to give some update on cello there?
I would love to and plan to.
great, thanks, and you can find some entire info at github repo and wiki, too.
great, thanks, and you can find some entire project info at github repo and wiki, too.
feel free to ping me if you need more
+2 on https://gerrit.hyperledger.org/r/#/c/17395/.
@baohua thanks, are you not coming?
I already place my name for the cello update.
pity, no plan for me, will have the spring festival then :)
Besides, i think we can have some hacking topics with cello, like bug fix or feature development. it's hackfest, hope most attendee will enjoy the hacking
Hello, I am using cello-ansible, however I couldn't find the script that generates the crypto material, specifically the channel.tx artifacts in the repository. Can you guys point me to script that handles the generation of artifactsin the project? Thanks
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=Bmaw9qe9QLKy2Lpyj) @baohua Makes sense. The documentation says 1.0. Does that include 1.05 for instance?
@FiratSertgoz the ansible script has a role called setupcerts , you can take a look there.
in the role, cryptogen is called to generate all certificates and keys for all parties.
@william123 if you want to have a taste of 1.1 now, you can use latest ansible agent to build everything off of the master, then ansible agent can help you stand up a fabric network.
It won’t be 1.1 but it will be using the latest fabric code.
Hello, I have a three servers running, one master and two worker nodes for cello
I did the setup using the make file (not the ansible or other agents) from cello as shown here (http://hyperledger-cello.readthedocs.io/en/latest/installation_master/),
everything fine I can access the dashboard (both cello dashboard and user dashboard)
but I can’t create a chain in cello dashboard, when I create the chain the health status shows fail
can someone help me please, I'm stacked with this for few days now
I tried to find the source of the error, from the log output (make logs) I saw that `[cluster.py:802 refresh_health()] - check MY--WORKE-IP:8150 result 111` which is not returning _0_ making the status fail
but from the worker node, using `netstat` I can see that port 8150 is open
@tongli can you please help me ?
[ ](https://chat.hyperledger.org/channel/cello?msg=iDX78sNrMamQ59KoA) @william123 exactly, it's 1.0.5 now
Hi guys. I have some question, can you help me?
With Cello user dashboard, we can Install and Instantiate smart contract (.go file) to chain. Can we do the same with bna file that generated by Hyperledger Composer? How can we do that? Thanks.
Composer generator only create .go file for cto and some system struct, don't create code for transaction
Hi @tongli I am gettin these 2 errors with etcd service and flanneld service ```hyperchain@hyperchainvm001:~$ systemctl status flanneld
● flanneld.service
Loaded: not-found (Reason: No such file or directory)
Active: inactive (dead)
hyperchain@hyperchainvm001:~$ systemctl status etcd
● etcd.service - Fabric ETCD V3 Service
Loaded: loaded (/lib/systemd/system/etcd.service; enabled; vendor preset: enabled)
Active: activating (start) since Fri 2018-02-02 10:37:11 UTC; 1min 4s ago
Main PID: 11196 (etcd)
Tasks: 6
Memory: 7.4M
CPU: 380ms
CGroup: /system.slice/etcd.service
└─11196 /opt/fabric/bin/etcd
Feb 02 10:38:14 hyperchainvm001 etcd[11196]: e817a72b8dba4c48 became candidate at term 154323
Feb 02 10:38:14 hyperchainvm001 etcd[11196]: e817a72b8dba4c48 received MsgVoteResp from e817a72b8dba4c48 at term 154323
Feb 02 10:38:14 hyperchainvm001 etcd[11196]: e817a72b8dba4c48 [logterm: 1, index: 3] sent MsgVote request to ab9a73145290816b at term 154323
Feb 02 10:38:14 hyperchainvm001 etcd[11196]: e817a72b8dba4c48 [logterm: 1, index: 3] sent MsgVote request to c18d9c210f61122e at term 154323
Feb 02 10:38:14 hyperchainvm001 etcd[11196]: publish error: etcdserver: request timed out
Feb 02 10:38:15 hyperchainvm001 etcd[11196]: e817a72b8dba4c48 is starting a new election at term 154323
Feb 02 10:38:15 hyperchainvm001 etcd[11196]: e817a72b8dba4c48 became candidate at term 154324
Feb 02 10:38:15 hyperchainvm001 etcd[11196]: e817a72b8dba4c48 received MsgVoteResp from e817a72b8dba4c48 at term 154324
Feb 02 10:38:15 hyperchainvm001 etcd[11196]: e817a72b8dba4c48 [logterm: 1, index: 3] sent MsgVote request to ab9a73145290816b at term 154324
Feb 02 10:38:15 hyperchainvm001 etcd[11196]: e817a72b8dba4c48 [logterm: 1, index: 3] sent MsgVote request to c18d9c210f61122e at term 154324```
Hi @tongli I am getting these 2 errors from the log of Vm with etcd service and flanneld service ```hyperchain@hyperchainvm001:~$ systemctl status flanneld
● flanneld.service
Loaded: not-found (Reason: No such file or directory)
Active: inactive (dead)
hyperchain@hyperchainvm001:~$ systemctl status etcd
● etcd.service - Fabric ETCD V3 Service
Loaded: loaded (/lib/systemd/system/etcd.service; enabled; vendor preset: enabled)
Active: activating (start) since Fri 2018-02-02 10:37:11 UTC; 1min 4s ago
Main PID: 11196 (etcd)
Tasks: 6
Memory: 7.4M
CPU: 380ms
CGroup: /system.slice/etcd.service
└─11196 /opt/fabric/bin/etcd
Feb 02 10:38:14 hyperchainvm001 etcd[11196]: e817a72b8dba4c48 became candidate at term 154323
Feb 02 10:38:14 hyperchainvm001 etcd[11196]: e817a72b8dba4c48 received MsgVoteResp from e817a72b8dba4c48 at term 154323
Feb 02 10:38:14 hyperchainvm001 etcd[11196]: e817a72b8dba4c48 [logterm: 1, index: 3] sent MsgVote request to ab9a73145290816b at term 154323
Feb 02 10:38:14 hyperchainvm001 etcd[11196]: e817a72b8dba4c48 [logterm: 1, index: 3] sent MsgVote request to c18d9c210f61122e at term 154323
Feb 02 10:38:14 hyperchainvm001 etcd[11196]: publish error: etcdserver: request timed out
Feb 02 10:38:15 hyperchainvm001 etcd[11196]: e817a72b8dba4c48 is starting a new election at term 154323
Feb 02 10:38:15 hyperchainvm001 etcd[11196]: e817a72b8dba4c48 became candidate at term 154324
Feb 02 10:38:15 hyperchainvm001 etcd[11196]: e817a72b8dba4c48 received MsgVoteResp from e817a72b8dba4c48 at term 154324
Feb 02 10:38:15 hyperchainvm001 etcd[11196]: e817a72b8dba4c48 [logterm: 1, index: 3] sent MsgVote request to ab9a73145290816b at term 154324
Feb 02 10:38:15 hyperchainvm001 etcd[11196]: e817a72b8dba4c48 [logterm: 1, index: 3] sent MsgVote request to c18d9c210f61122e at term 154324```
If I am trying to run ```ansible-playbook -i run/runhosts -e "mode=destroy env=azure env_type=flanneld ansible_ssh_user=hyperchain" initcluster.yml
```
I am getting this error ```TASK [Remove registrator service] **********************************************
fatal: [52.230.121.199]: FAILED! => {"changed": true, "cmd": ["docker", "rm", "-f", "registrator.hyperchainvm003"], "delta": "0:00:00.031081", "end": "2018-02-02 10:46:45.659680", "msg": "non-zero return code", "rc": 1, "start": "2018-02-02 10:46:45.628599", "stderr": "Error: No such container: registrator.hyperchainvm003", "stderr_lines": ["Error: No such container: registrator.hyperchainvm003"], "stdout": "", "stdout_lines": []}
fatal: [52.230.11.209]: FAILED! => {"changed": true, "cmd": ["docker", "rm", "-f", "registrator.hyperchainvm002"], "delta": "0:00:00.021778", "end": "2018-02-02 10:46:45.602873", "msg": "non-zero return code", "rc": 1, "start": "2018-02-02 10:46:45.581095", "stderr": "Error: No such container: registrator.hyperchainvm002", "stderr_lines": ["Error: No such container: registrator.hyperchainvm002"], "stdout": "", "stdout_lines": []}
fatal: [52.163.112.118]: FAILED! => {"changed": true, "cmd": ["docker", "rm", "-f", "registrator.hyperchainvm001"], "delta": "0:00:00.415782", "end": "2018-02-02 10:46:46.063424", "msg": "non-zero return code", "rc": 1, "start": "2018-02-02 10:46:45.647642", "stderr": "Error: No such container: registrator.hyperchainvm001", "stderr_lines": ["Error: No such container: registrator.hyperchainvm001"], "stdout": "", "stdout_lines": []}
```
hi guys how to add hyperledger explorer with ansible ?
@Darumada Not sure what your end goal is. However you can clone and run HL-Explorer or clone cello and install it. as HL_Explorer is integrated
Has joined the channel.
@DarshanBc I think you have some misconfigurations.
@DarshanBc it will be nice to look at your configuration files to figure things out.
https://gerrit.hyperledger.org/r/#/c/17509/
Clipboard - 4 февраля 2018 г., 14:43
Снимок экрана от 2018-02-04 14-41-17.png
Снимок экрана от 2018-02-04 14-42-17.png
Снимок экрана от 2018-02-04 14-42-33.png
Снимок экрана от 2018-02-04 14-43-53.png
Снимок экрана от 2018-02-04 14-44-08.png
Снимок экрана от 2018-02-04 14-44-21.png
i use ansible agent and aws cloud , node and npm installed
[ ](https://chat.hyperledger.org/channel/cello?msg=a4BnkWQquqk68pMmB) In case someone have the same problem, I resolved this issue as follows
After setting up everything correctly whenever I create a chain from the operator-dashboard the health status was always on “fail”. It seems that the container providing explorer service was not responding.
I resolved the issue by removing `sleep 30;` command from (src/agent/docker/_compose_files/fabric-1.0/local/fabric-solo-4.yaml) under the section of explorer.
cello1.png
# we should never process in-processing chains unless deleting one
if chain_user_id.startswith(SYS_USER): ----> IT WILL TRUE
if chain_user_id.startswith(SYS_DELETER): # in system processing, TBD
for i in range(retries):
time.sleep(period)
if cluster_handler.get_by_id(chain_id).get("user_id") != \
chain_user_id:
return
logger.info("Delete in-deleting chain {}/{}".format(
chain_name, chain_id))
cluster_handler.delete(chain_id)
return
it makes the health is FAIL.
when I comment line 48~58, the health becomes OK.
the code snippet comes from https://github.com/hyperledger/cello/blob/master/src/watchdog.py line 47~58
the code snippet comes from https://github.com/hyperledger/cello/blob/master/src/watchdog.py line 48~58
I use docke as worker node.
yes, we are changing the health check logs in https://gerrit.hyperledger.org/r/#/c/17511/ (existing one seems not very stable), pls wait for a while @Norberthu
https://gerrit.hyperledger.org/r/#/c/17511/
Hey @tongli please review this patch set regarding CE-253 https://gerrit.hyperledger.org/r/17513 . I think I made some mistake related to formatting styles. Let me know if I need to add something in this. I will then submit another patch
@Darumada How you get this kubernetes dashboard after deploying fabric using ansible? In my case it dosent show anything
[ ](https://chat.hyperledger.org/channel/cello?msg=J6wGybu7M9LTCgCHd) @Norberthu does this happen if you try to use kafka consensus plugin when you create the chain ?
Hi guys , how to deploy nodejs chaincode with ansible agent?
@MohammadObaid add some comments on https://gerrit.hyperledger.org/r/#/c/17513/, thanks for your first pr!
[ ](https://chat.hyperledger.org/channel/cello?msg=H3Hb95fiEL2RRutEw) @Amir solo, no kafka
[ ](https://chat.hyperledger.org/channel/cello?msg=mnmQBkfGEqaJkNMzM) @baohua thanks
[ ](https://chat.hyperledger.org/channel/cello?msg=NB9FqjRBGgjXd3GfY) @baohua OK, I successfully run vbk8s.yml on my VirtualBox env. How can I open the cell dashboard? I can access k8s dashboard but I cannot find the cello dashboard...
in k8s mode, we do not connect the dashboard yet. you can use the k8s one for a workaround.
https://gerrit.hyperledger.org/r/#/c/17511/
@tongli @hightall
I see. Do you have any plans to support k8s from the cello dashboard? (I'm just asking to know if I can contribute to this project
[ ](https://chat.hyperledger.org/channel/cello?msg=ADXZkYtGa7Kyhw3BM) @Norberthu emm I'm getting the error on kafka too, for the solo one I resolved the issue as I described here https://chat.hyperledger.org/channel/cello?msg=9CCmbAWkpgJm6XJLi
@tkueda exactly, welcome for you help. feel free to pick up this work: https://jira.hyperledger.org/browse/CE-168
@tkueda exactly, welcome for you help! feel free to pick up this work: https://jira.hyperledger.org/browse/CE-168
and we can first split it into several smaller subtasks, which are easy to track and fix.
CE-168
[ ](https://chat.hyperledger.org/channel/cello?msg=xyQumKjXmZQ8SvWwD) @Amir I has tried it as you described , but had no effect.
[ ](https://chat.hyperledger.org/channel/cello?msg=AJ2geLaXrrXndsyzT) @Norberthu can you post the log, specifically the output generated while you create the chain (for my case I used the Makefile to setup the master and worker nodes as shown here (http://hyperledger-cello.readthedocs.io/en/latest/installation_master/). I can see the logs by running `make logs` from the master node). If you can post the log it may help to see why the health is failing, Also as Baohua pointed there is ongoing work to change health check logs hopefully the update may fix such issues
who deploy nodejs chaincode ? can u help me?
Has joined the channel.
Does anybody know if IBM Blockchain uses `cello` to standup fabric network? If not, how do they do it?
We need the dockerfiles for the coming v0.8.0-beta release, who would like to volunteer? https://jira.hyperledger.org/browse/CE-260. Thanks!
Let me take that.. :)
Request for Review
Request for Review - https://gerrit.hyperledger.org/r/17569
https://gerrit.hyperledger.org/r/17573
https://gerrit.hyperledger.org/r/17575
https://gerrit.hyperledger.org/r/17577
https://gerrit.hyperledger.org/r/17579
+2, thanks!
https://gerrit.hyperledger.org/r/#/c/17569/ -- Should this be abandoned ?
aha, harsha you can help change it to fix the images dockerfiles for the latest version. we missed some image dockerfiles there.
the final target is to run all services using docker image
you may notice that we have more cello images at https://hub.docker.com/r/hyperledger/?.
Agreed.
@harsha changed according to your comment, thanks! https://gerrit.hyperledger.org/r/17577
@baohua Any CE for v0.8.0 ?
@baohua Any JIRA numbe for v0.8.0-beta release ?
@harsha for https://gerrit.hyperledger.org/r/#/c/17575/ , do you mean we can use git tag as docker image tag? there is no change for baseimage and user-dashboard, just need to create docker tag?
@baohua Would be manually creating the tag in docker-hub space, and link it to https://github.com/hyperledger/cello/blob/master/dockerhub/baseimage/Dockerfile . So we may not need to create baseimage for each and every release.
yes and @baohua Would be manually creating the tag in docker-hub space, and link it to https://github.com/hyperledger/cello/blob/master/dockerhub/baseimage/Dockerfile . So we may not need to create baseimage for each and every release.
Has joined the channel.
hello! i believe this is the right place to ask this question... I am currently working on a project with Hyperledger Composer, which runs on top of Fabric. I should be able to orchestrate a blockchain with Cello, right? If so, is there any tutorial out there??? I am very lost here...
[ ](https://chat.hyperledger.org/channel/cello?msg=8MumXwbmjAR4Fup6c) @pacoard you can start from this doc http://cello.readthedocs.io/en/latest/
@baohua @hightall as I have promised, I reworked this document heavily to document the ansible agent. https://gerrit.hyperledger.org/r/#/c/17607/
I reworked the most of the content so that the document will be easy to follow. Please review. Thanks.
great, documentation is very important, thanks! @tongli
@pacoard if composer can connect to an existing chain, then it will be easy. cello boots up fabric chains, and then expose the services port for usage.
@baohua @hightall please take a look at the patch when you have few minutes. https://gerrit.hyperledger.org/r/#/c/17607
will do today, it's a bit long to read through....
it is a long document and trying to be very thorough.
cello-operator-dashboard | Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.
cello-operator-dashboard |
cello-operator-dashboard | To deploy your application across the swarm, use `docker stack deploy`.
[ All containers will be scheduled on the current node.] --> Is that ture ?
https://gerrit.hyperledger.org/r/17579
https://gerrit.hyperledger.org/r/17577
@lujiang +1'ed https://gerrit.hyperledger.org/r/#/c/17579/
Has anyone deploy Fabric in the cloud, and in a consortium manner?
Has anyone deploy Fabric in the cloud, and in a consortium style configuration? If yes, can you direct message me?
Has anyone deploy Fabric in the cloud, and in a consortium style (i.e.peers, orderers, ca in different data centres or different cloud subscription) configuration? If yes, can you direct message me?
@paul.sitoh Not sure about others, however this shd be a good entry for you to start off with http://cello.readthedocs.io/en/latest/awsinstall/#deploy-hyperledger-cello-on-aws-ec2
@baohua @hightall Please review the second patch set. [CE-253] Ansible agent needs a document
Ansible agent needs document to detail the steps
how to use the ansible agent. This patch set
reworked an existing document and reformat the
document so that the steps can be easily followed
Change-Id: I10b4c5efa6678eba4422a675baaac300bdceb544
Signed-off-by: tongliofcary
https://gerrit.hyperledger.org/r/#/c/17607/
when I run 'git review' with the error. how to fix them?
```remote:
remote: Processing changes: refs: 1
remote: Processing changes: refs: 1, done
To ssh://gerrit.hyperledger.org:29418/cello
! [remote rejected] HEAD -> refs/publish/master/CE-263 (not Signed-off-by author/committer/uploader in commit message footer)
error: failed to push some refs to 'ssh://yoyokeen@gerrit.hyperledger.org:29418/cello'```
```zhangyinxiadeMacBook-Pro:cello zhangyinxia$ git commit --amend
[CE-263 9bb2f17] Create dockerfiles for operator-dashboard
Date: Thu Feb 8 09:22:16 2018 +0800
6 files changed, 135 insertions(+)
create mode 100644 dockerhub/operator-dashboard/Dockerfile
create mode 100644 dockerhub/operator-dashboard/requirements.txt
create mode 100644 dockerhub/v0.8.0-alpha/operator-dashboard/Dockerfile
create mode 100644 dockerhub/v0.8.0-alpha/operator-dashboard/requirements.txt
create mode 100644 dockerhub/v0.8.0-beta/operator-dashboard/Dockerfile
create mode 100644 dockerhub/v0.8.0-beta/operator-dashboard/requirements.txt
zhangyinxiadeMacBook-Pro:cello zhangyinxia$ git log
commit 9bb2f17147d463e694192e393dacfa497081ec65 (HEAD -> CE-263)
Author: yoyokeen <602840159@qq.com>
Date: Thu Feb 8 09:22:16 2018 +0800
Create dockerfiles for operator-dashboard
Change-Id: I2f4957083e53f74d0c7d5885c2a760bb3ef6aa7b```
that's mean I need to switch another branch?
@yoyokeen can u help paste the steps to reproduce the problem? e.g., following http://cello.readthedocs.io/en/latest/CONTRIBUTING/, 1. git clone 2. blabla.... 3. Problem happens!
git add not with the 'Signed-off-by', so cant run 'git review'
how to fix it?
git commit lost '-s'
that's the problem
would suggest you just clone a fresh one, and manually put the changed files inside
try it now
https://gerrit.hyperledger.org/r/#/c/17607/
I find the new file already existing in the mater, that havent been review.
what's the next step?
what's ur step now?
let's u to review
sorry i cannot follow you .... can u help follow http://cello.readthedocs.io/en/latest/CONTRIBUTING and tell which step you meet the problem? e.g., post the command you have done
Hello, I was trying to create a channel from cello user dashboard, and I'm getting this error from the logs
`cello-user-dashboard | 02:29:26 0|index | [2018-02-08 02:29:26.626] [DEBUG] Create-Channel - Successfully acquired admin user for the organization "org1"
cello-user-dashboard | 02:29:26 0|index | [2018-02-08 02:29:26.648] [DEBUG] Helper - [crypto_ecdsa_aes]: ecdsa signature: Signature {
cello-user-dashboard | 02:29:26 0|index | r:
` cello-user-dashboard | 02:29:26 0|index | [2018-02-08 02:29:26.626] [DEBUG] Create-Channel - Successfully acquired admin user for the organization "org1"
cello-user-dashboard | 02:29:26 0|index | [2018-02-08 02:29:26.648] [DEBUG] Helper - [crypto_ecdsa_aes]: ecdsa signature: Signature {
cello-user-dashboard | 02:29:26 0|index | r:
``` cello-user-dashboard | 02:29:26 0|index | [2018-02-08 02:29:26.626] [DEBUG] Create-Channel - Successfully acquired admin user for the organization "org1"
cello-user-dashboard | 02:29:26 0|index | [2018-02-08 02:29:26.648] [DEBUG] Helper - [crypto_ecdsa_aes]: ecdsa signature: Signature {
cello-user-dashboard | 02:29:26 0|index | r:
any help ?
@hightall pls help have a look
https://gerrit.hyperledger.org/r/#/c/17661/
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/17667/
@lc_wxk add some comments: https://gerrit.hyperledger.org/r/#/c/17667/. Thanks!
@baohua ok
@yoyokeen add some comments on https://gerrit.hyperledger.org/r/#/c/17661/, thanks!
@baohua, I already droped the local branch, so if I modify it I need to do once more again.
:joy:
https://gerrit.hyperledger.org/r/#/c/17669/
@harsha https://gerrit.hyperledger.org/r/#/c/17577/ this is for alpha use, so I use FROM hyperledger/cello-baseimage:x86_64-0.8.0-alpha in dockerfile
Congrats to @yoyokeen for her 1st merged patchset: https://gerrit.hyperledger.org/r/#/c/17669/!
@lujiang I presume we do not need for alpha any more. ^^ @baohua
@lujiang I presume we do not anymore PR alpha any more. ^^ @baohua
Has joined the channel.
Hey @tongli before runnning ansible scrirpt we need to generate key pair from aws and use their name in aws configuation file right?
no. actually you can simply generate ssh key pair from your ansible controller, ansible will upload and create, then inject into vm automatically
@MohammadObaid I just landed this doc, please take a look.
https://github.com/hyperledger/cello/blob/master/docs/worker_ansible_howto.md
it has great details.
especially look at this https://github.com/hyperledger/cello/blob/master/docs/worker_ansible_howto.md#setup-ansible-controller
and this if you care about ssh keys
https://github.com/hyperledger/cello/blob/master/docs/worker_ansible_howto.md#generate-a-ssh-key-pair
```When you work with have a cloud such as OpenStack, AWS, the ssh public key you generated here will be automatically injected into the severs in Provision the servers step.```
Yes I have read that. Great documentation :). but if ansible generated that key pair then how would we connect to that instance because to connect to that instance you need `pem` file !!! . It hasn't been a problem for me but for for few days I have facing such situation in which ansible script run smoothly butunable to connect to any ubuntu instances and getting error `Permission denied (publickey).`
Yes @tongli I have read that. Great documentation :). but if ansible generated that key pair then how would we connect to that instance because to connect to that instance you need `pem` file !!! . It hasn't been a problem for me but for for few days I have facing such situation in which ansible script run smoothly butunable to connect to any ubuntu instances and getting error `Permission denied (publickey).`
For ansible I am running this command `ansible-playbook -e mode="apply" aws.yml`
Has joined the channel.
Can I get a tiny code review for https://gerrit.hyperledger.org/r/#/c/17697/
Fixes a few small things with the ansible playbook.
@MohammadObaid the public key is uploaded to aww before nothing happens, then the key gets injected to each vm.
@MohammadObaid that is how it will work.
I was testing the ansible agent on three separate physical machines (1 for ansible controller and the other two for fabric network). I setup the ssh keys and created manually the `run/runhosts` file and started the `setupfabric.yml` playbook as `ansible-playbook -i run/runhosts -e "mode=apply env=vb1st" setupfabric.yml` but the play book stacks on `TASK [Pull certificates from the builder machine]` just hangs there no error or warning
any help ?
tried to use `bc1st.yml` configuration file it is the same just hangs at `TASK [Pull certificates from the builder machine]`
the task seems straight forward, it is trying to copy `certs.tgz` file from the builder and I checked the files exists. But I don't know why it pauses here
@bleggett +2!
[ ](https://chat.hyperledger.org/channel/cello?msg=vshxdCgBdjxyzJCLz) @hightall any help please ?
@harsha I agree with you to "I presume we do not anymore PR alpha any more", so that we can abandon what has done this time for engine/operator dashboard/watchdog in alpha folder, do you agree?
and one more concern about your comment on "we may not need dockerhub/engine/Dockerfile & dockerhub/engine/requirements.txt", I might think they are used for latest tag
and the dockerfile used for latest tag image, I think we don't need to specify the baseimage tag, see dockerhub/mongo/Dockerfile, we need to remove "x86_64-0.8.0-alpha" in line 1 : FROM hyperledger/cello-baseimage:x86_64-0.8.0-alpha
@baohua need your comment
Sounds reasonable to me. besides, do we need some separate `latest` dir under `dockerhub` to make it more clear?
Has joined the channel.
I can't find user-dashboard ui on 8081, can anyone helps me
https://gerrit.hyperledger.org/r/#/c/17723/
https://gerrit.hyperledger.org/r/#/c/17723
@lujiang We should always publish the built docker-images on release branch. The master branch can be built locally on the machine. The reason `latest tag` in dockerhub namespace should rather be pointing towards newly cut branch as opposed to master branch, given the default nature of `docker github automated build` it always publish new docker images unless turned off for every potential merge into master code, which results in unnecessary publishing of docker images. As `cello-baseimage:RELEASE_TAG` is built first and components like mongo,nginx leverage `cello-baseimage:RELEASE_TAG` and build on top of this.
[ ](https://chat.hyperledger.org/channel/cello?msg=2q9GiqzQNtN8HzHA8) @Brandon try running ´make build-user-dashboard-js´ from cello/
Is there an issue with the new images ? the engine, watchdog and dashboard containers are always on restarting mode and can't access the dashboard
Has joined the channel.
Why I can access dashboard on 8080, can't access user dashboard on 8081
[ ](https://chat.hyperledger.org/channel/cello?msg=JcQxh9p7xBYi4KbMb) @zsmartc did you tried to run `make build-user-dashboard-js` from cello/ folder ?
@harsha the reason we use docker image in dockerhub is to save the effort of building images locally by users. So the latest tag can safely point to the latest code at master branch, to allow users to quickly pull down and start to use cello. does this make sense?
https://gerrit.hyperledger.org/r/#/c/17761/
@baohua , et al, I do not disagree with the approach, however I feel we should not let experimental or WIP features get exposed and published in dockerhub, rather let users build/compile them locally unless those features are thoroughly tested and published in release-based approach to dockerhub. Also latest tag only points to x86 images, and since manifest aren't in place this would pull x86 images onto non-x86 platform. However i wouldn't mind publishing other way around too (ie latest tag pointing to latest code on master branch)
hello, in the user dashboard, what do you exactly upload as a smart contract?
Clipboard - 10 de febrero de 2018 12:21
in my case, i am using hyperledger composer, which generates a .bna file that I really doubt could work here
@harsha @lujiang `non-x86` cannot be supported by dockerhub today, hence we need to use other workarounds. For x86 one, let's use dockerhub to help users avoid local building (developers might wanna build locally, but as far as i know, other users just wanna a open-box-to-user solution).
@harsha @lujiang `non-x86` cannot be supported by dockerhub today, hence we need to use other workarounds. For `x86` one, let's use dockerhub to help users avoid local building (developers might wanna build locally, but as far as i know, other users just wanna a open-box-to-user solution).
@tongli would u like to give comments on this? https://gerrit.hyperledger.org/r/#/c/17697/
@MohammadObaid any update? https://gerrit.hyperledger.org/r/#/c/17513/
WeChat Image_20180211104733.png
how to config the chain size or organizations in fabric network
I can only choose 4 peers
and is it possible to create several channels on cello platform?
Are the images(build-login,build-home,build-dashboard and cello-user-dashboard) in docker-compose-build-js.yaml changed frequenctly?Is it possible to provide these images in dockerhub directly?
@baohua Remove all trailing spaces in 3rd patch set
Sure, and after the pr passed ci, feel free to post the link here to invite people to help review
Alright :)
pr means jenkins test ?
@Brandon Cello only support predefined cluster so far, which mean you can't customize organizations.
But I think features relative to organizations customization will come in the nearly future
[ ](https://chat.hyperledger.org/channel/cello?msg=8mb39zkipfPn574gn) @Luke_Chen thanks a lot. by the way , how can i change the predefined cluster?
@Brandon here, https://github.com/hyperledger/cello/blob/master/src/agent/docker/_compose_files/fabric-1.0/fabric-solo-4.yaml
it's a docker-compose start up file.
@Brandon Oh I forgot, of course it is possible to create several channels by default, cello will create a `bussinesschannel` which contains all peers for you
@Brandon Oh I forgot, of course it is possible to create several channels , by default, cello will create a `bussinesschannel` which contains all peers for you
for orther channels, you must create them manually.
Has joined the channel.
Mongo can start in first time, but after I use make stop to stop all contains and make start againt, then mongo can't start up. Does anyone encounter the same problem, and know the reason?
[ ](https://chat.hyperledger.org/channel/cello?msg=zr9ypCTRe52PA8oQX) @zsmartc ya I got this issue with engine, watchdog and dashboard services. make restart didn't help but redeploying each service separately solved my issue. `make redeploy service=SERVICE_NAME`
@baohua @hightall please take a look at this patch for updating to k8s 9.x and allow using the latest k8s dashboard. thanks https://gerrit.hyperledger.org/r/#/c/17003/
Congrats to @bleggett for his first accepted pactchset to Cello: https://gerrit.hyperledger.org/r/#/c/17697/. Look forward to seeing more contributions!
@tongli pls help fix the ci check failure for the patchset
@baohua I have the same ci check failure: ImportError: No module named 'flask_testing'
what's the step to repeat the problem? seems we have the module already.
local check can reproduce
also jenkens: https://jenkins.hyperledger.org/job/cello-verify-x86_64/575/console
can you help dig out/fix the problem? working on other part now.
https://gerrit.hyperledger.org/r/#/c/17783/
investigating
@hightall about your https://gerrit.hyperledger.org/r/#/c/17761/ patch I have a question, we hard code the FABRIC_CA_SERVER_CA_KEYFILE on fabric1.0-solo.yaml(https://github.com/hyperledger/cello/blob/master/src/agent/docker/_compose_files/fabric-1.0/fabric-solo-4.yaml) so far, if we allow crypto-config directory generating dynamically, the CAs will totally break away the chain of trust, because the CA won't find the hard code private key file and will generate a brand new pair of pub/prv keys.
@hightall about your https://gerrit.hyperledger.org/r/#/c/17761/ patch I have a question
we hard code the FABRIC_CA_SERVER_CA_KEYFILE on fabric1.0-solo.yaml(https://github.com/hyperledger/cello/blob/master/src/agent/docker/_compose_files/fabric-1.0/fabric-solo-4.yaml) so far, if we allow crypto-config directory generating dynamically, the CAs will totally break away the chain of trust, because the CA won't find the hard code private key file and will generate a brand new pair of pub/prv keys.
[ ](https://chat.hyperledger.org/channel/cello?msg=iQ2JmkuRYGsGLwX2u) @Amir This method didn't work. The docker log indicated the following error "python: can't open file 'dashboard.py': [Errno 2] No such file or directory". Anyone know the reason?
@Luke_Chen Yes, If we dynamic generate crypto-config, then when compose start docker, the CA_KEYFILE will use env from operator dashboard, that will be dynamic variable, and the user-dashboard will also do that, and develop can download crypto-config files for development
[ ](https://chat.hyperledger.org/channel/cello?msg=nQmbcEHXZz2X4Qep8) @zsmartc I'm now having the same issue and `make redeploy service=SERVICE_NAME` is not resolving it. I'm trying to do diagnosis it, I will let you know if I have something and please let me know if you resolve it too. any one having the same issue ?
@hightall Thanks, I explained my concern because I am working on hyperledger/composer intergration, which is strongly relative to fabric-ca.
@Amir have you clean up the /opt/cello dir on your worker node?
@Amir maybe try to clean up /opt/cello on your master node first.
find RCA for check failure hyperledger/cello-user-dashboard
The cello-baseimage build in dockerhub is not successful until 30 minutes ago
now it ok for check
[ ](https://chat.hyperledger.org/channel/cello?msg=PjiEP3r2gA9DG9Cst) @Luke_Chen OK thanks, I will try it now
@Luke_Chen Thanks it is working now
@zsmartc the solution from @Luke_Chen ( clean the /opt/cello/ folder ) resolved my issue
https://gerrit.hyperledger.org/r/#/c/17781/
[ ](https://chat.hyperledger.org/channel/cello?msg=CjheoAdHKsCtNAex8) @Amir I still can't start up watchdog, operator-dashboard and engine. Error in docker logs was "python: can't open file 'watchdog.py': [Errno 2] No such file or directory".
@lujiang @baohua looks like this missed Dockerfile for nginx under 0.8.0-nginx https://gerrit.hyperledger.org/r/#/c/17781/
looks like it's already in https://github.com/hyperledger/cello/tree/master/dockerhub/v0.8.0-beta/nginx
oh, @lujiang did u remove the dockerfile for nginx?
@baohua baohua, what would you like me to update cello on at LA?
i think maybe u can help share the latest release/feature/progress, and help encourage more developers to use and join develop. As a provision and operational management tool, we wanna help community adopt and manage blockchain techniques in more efficient way.
any good idea from ur side?
https://gerrit.hyperledger.org/r/#/c/17783/
@baohua same goal here. Thinking if we can integrate cello dashboard (chain management UI) with k8s dashboard.
if we can do that, it will be really exciting.
yeap, l remember there's a jira issue related to it
we hope can have it in v0.9
yeah. if you can give me a list of things I would be really happy to get in front of the people giving the updates.
we got 20 minutes I think.
Seems we need enhance our frontend dev team, lots of work to do with the dashboard recently.
aha, a quick sync talk
@tongli as maintainer, one special responsibility is to help more people to join to contribute and adopt.
right agreed.
I very much like to have ansible agent as part of the docker image, so that people can simply start up the cello image, then they can start running the playbooks.
one of the issues that people have is to setup ansible env. 90% of the times people stumbled on that.
yes, we should hide those efforts to make it easy for people to use cello.
if the cello image includes the ansible env, then people do not have to do that part, they can simply use pre-canned configurations or update the configuration,they will only have to deal with the configuration files, then it is a go.
big advantage doing that.
you mean those ansible play-books at Master node?
yes, I mean that the cello image you guys have been working on, we can have ansible and dependency installed into,
the entire `src` is included in the latest baseimage now
which is also the base for most cello service images
right. src is included but with no ansible runtime, right?
hmmm. we are building multiple cello images?
I thought there is just one for everything.
oh, you wanna install the ansible env into the engine image? I think it should be ok.
yeap, there are images for different services
we hope to make the future deployment to be easy to extend to distributed env.
we hope to allow the future deployment easy to extend to distributed env.
I am a bit concerned on making so many images, then people will have to choose which one to use etc.
that normally creates a lot of confusions.
but if that is the direction cello community likes to go, I can live with.
oh, that is because we got several service components now. keep a single image for each component make it easy to deploy actually, and more `micro-service`.
just like fabric, have several images for peer, orderer, ca....
that is probably true, but requires people knows a bit more.
yes, i also update the doc to introduce those images
the initial goal of using docker images is to allow people avoid provision efforts of using cello
right, that I agree and I think it is a good idea, I was thinking the same thing.
yeap, since it's just introduced. pls help test the provision and documentation, to make sure that even beginners can successfully boot up cello to use easily.
@baohua good deal. back to the update items. please send me few lines so that I can emphasize when I give the updates.
or simply paste them here so that others can also have some knowledge.
you can find the highlights at https://github.com/hyperledger/cello/releases/tag/v0.8.0-alpha
And after that, mostly we are working on enabling the docker images
and bug fixes
k. thanks for that.
https://gerrit.hyperledger.org/r/17795
+2
@baohua @hightall please see this, the issue was the mongo test was failing, and fixed by other patch set. I just did the rebase. https://gerrit.hyperledger.org/r/#/c/17003/
Has joined the channel.
Hi!
I follow this manual https://medium.com/@mohammadobaid11/deploying-hyperledger-fabric-on-aws-using-ansible-c2b240eac83f
and I'm getting errors:
`fatal: [34.229.115.77]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Warning: Permanently added '34.229.115.77' (ECDSA) to the list of known hosts.\r\nPermission denied (publickey).\r\n", "unreachable": true}
`
Can anybody help?
Hi!
I follow this manual https://medium.com/@mohammadobaid11/deploying-hyperledger-fabric-on-aws-using-ansible-c2b240eac83f
and I'm getting errors:
```fatal: [34.229.115.77]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Warning: Permanently added '34.229.115.77' (ECDSA) to the list of known hosts.\r\nPermission denied (publickey).\r\n", "unreachable": true}
```
Can anybody help?
@krabradosty what task?
@krabradosty what task is actually failing?
Has joined the channel.
@bleggett Gathering Facts
By the way, I can connect to new ec2 instances with private key I pointed at the aws.yml
@tongli +2!
Has joined the channel.
Hello guys.
Running Cello for the first time, running into an issue also mentioned here: https://jira.hyperledger.org/browse/FAB-1341
Creating a host fails from the dashboard.
Logs showing a connection refused on my host ip
any ideas?
@baohua thanks a lot.
@tongli when a patchset passed CI, should post gerrit link to the jira issue and mark it as `Review`; after the pr is merged. should also update the jira status to `Done`.
[ ](https://chat.hyperledger.org/channel/cello?msg=9BivchFN2ixNGx8YW) @Amir I encountered the same problem. So did you solve this problem?
[ ](https://chat.hyperledger.org/channel/cello?msg=u3o9mmitNgzNtwGMF) @zsmartc I still can't create a channel from the user dashboard. But I use a work around by editing the default `businesschannel` and add/install my own chaincode on top
if you want to follow that way look at the scripts under `scripts/` folder
https://gerrit.hyperledger.org/r/#/c/17833/
[ ](https://chat.hyperledger.org/channel/cello?msg=5qcqLZt3aKDYZMvdF) @Amir Thank you. I will look at the scripts
@baohua, these things should have been done automatically. that is why patch set has these jira issue in the patch set comment, we need to make that happen.
@here, happy New Year to all who celebrate Chinese New Year, cello has had a great year, we are moving to the right direction. Wish you all the best in the coming year and wish Cello to grow like these cryptocurrencies. 祝各位新年快乐,平安,健康!
Hello, guys. I have managed to deploy a fabric network with ansible-k8s-aws configuration. The next step for me is to somehow connect my local NodeSDK client to the network. However, I could not find a way to expose the ports of kubernetes. Is there a way to do this in the ansible configs or would it be wiser for me to just ditch k8s and go docker-host?
Thanks
@FiratSertgoz when you use k8s to setup fabric network, you already have all the peers and orderers setup as k8s services.
these ports are already exposed. you should be able to see these services on your k8s dashboard
Has joined the channel.
Hello all! I'm a new in Cello . Can I apply it to existing hyperledger network ?
And maybe you can recommend some tutorial for begging?
@vladyslavmunin https://github.com/hyperledger/cello/blob/master/docs/setup_worker_ansible_allinone.md
I would start there.
Has joined the channel.
Has joined the channel.
the last push, which makes the production mode true i.e `DEV=False`, uses `docker-compose.yml` and it doesn't allow access to operator-dashboard. when I use `docker-compose-dev.yml` by changing `DEV=True` I can access the operator dashboard. Is there any additional thing needed to use production mode ?
btw the container providing the dash board service is up and running
Folks!
Where can I find information about how to deploy fabric with CA server in cluster mode? I'm using ansible.
@krabradosty if you are using ansible agent, and the default configuration file, you probably already have ca servers running. :-)
@tongli Yes, but not in cluster mode
are you setting up fabric network using src/agent/ansible playbooks?
if you are doing that, then ca should have been setup for you.
if not, then I do not know how to add ca.
by default, it gets you a 3 nodes fabric cluster.
I thought it's a three CA of different organizations. Not a cluster.
Can I configure CA to use not default DB as described here: http://hyperledger-fabric-ca.readthedocs.io/en/latest/users-guide.html#configuring-the-database ?
Q about the ansible playbook: The baseline Ubuntu 16.04 image that we default to doesn't include python2 by default anymore, only python3. I've tested the playbook and it works fine with python3 as long as you install `pip3` via apt, and tell ansible to use Ubuntu's `python3` binary
Should I A) Update the docs to mention this and leave it as-is B) Update the scripts to expect python3 C) Explicitly install python2 via apt?
Q about the ansible playbook: The baseline Ubuntu 16.04 image that we default to doesn't include python2 by default anymore, only python3. I've tested the playbook and it works fine with python3 as long as you install `pip3` via apt, and tell ansible to use Ubuntu's `python3` binary
Would you guys prefer a PR that: 1. Update the docs to mention this and leave it as-is 2. Update the scripts to expect python3 3. Explicitly install python2 via apt?
Q about the ansible playbook: The baseline Ubuntu 16.04 image that we default to doesn't include python2 by default anymore, only python3. I've tested the playbook and it works fine with python3 as long as you install `pip3` via apt, and tell ansible to use Ubuntu's `python3` binary
Would you guys prefer a PR that: 1. Update the docs to mention this and leave it as-is 2. Update the scripts to expect python3 or 3. Explicitly install python2 via apt?
Q about the ansible playbook: The baseline Ubuntu 16.04 image that we default to for fabric nodes doesn't include python2 by default anymore, only python3. I've tested the playbook and it works fine with python3 as long as you install `pip3` via apt, and tell ansible to use Ubuntu's `python3` binary
Would you guys prefer a PR that: 1. Update the docs to mention this and leave it as-is 2. Update the scripts to expect python3 or 3. Explicitly install python2 via apt?
Q about the ansible playbook: The baseline Ubuntu 16.04 image that we default to for fabric nodes doesn't include python2 by default anymore, only python3. I've tested the playbook and it works fine with python3 as long as you install `pip3` via apt, and tell ansible to use Ubuntu's `python3` binary
Would you guys prefer a PR that: 1. Update the docs to mention this and leave it as-is 2. Update the scripts to expect python3 via `ansible_python_interpreter=/usr/bin/python3` or 3. Explicitly install python2 via apt?
Q about the ansible playbook: The baseline Ubuntu 16.04 image that we default to for fabric nodes doesn't include python2 by default anymore, only python3. I've tested the playbook and it works fine with python3 as long as you install `pip3` via apt, and tell ansible to use Ubuntu's `python3` binary
Would you guys prefer a PR that: 1. Update the docs to mention this and leave it as-is 2. Update the scripts to expect python3 via `ansible_python_interpreter=/usr/bin/python3` or 3. Explicitly install python2 via apt during node provisioning?
Q about the ansible playbook: The baseline Ubuntu 16.04 image that we default to for fabric nodes doesn't include python2 by default anymore, only python3. I've tested the playbook and it works fine with python3 as long as you install `pip3` via apt, and tell ansible to use Ubuntu's `python3` binary
Would you guys prefer a PR that: 1. Updates the docs to mention this and leave it as-is 2. Update the scripts to expect python3 via `ansible_python_interpreter=/usr/bin/python3` or 3. Explicitly install python2 via apt during node provisioning?
This doesn't really affect the ansible host at all.
https://gerrit.hyperledger.org/r/17833
Happy Spring Festival to everyone!
@tongli Well, when i check my kubernetes services on dashboard, I only see internal endpoints set, the external endpoints don't exist. Also when i cUrl the public ip address of the server with the port of the peer lets say i cant access.
Hi team.. how does cello achieve multi-tenancy is it on chaincode level or on peer level ?
I mean on a same set of peers we start diff channels or different peers altogether when we want to deploy a new chaincode
@AshishMishra 1 i guess this multi-tenancy is depending on the fabric features, and cello doesn't add any limitation to that.
@baohua , I 'm not looking for the limitation but want to understand how cello is achieving multi tenancy. It can either do at chaincode level, where I 've diff chaincodes and channels for diff tenants but being served by the same peer or else I can spin new peers altogether so I 'm achieving multi-tenancy on container level.
Not really, cello now provides two roles. 1) `operator`: that can manage the networks, and 2) `users`: that use a single fabric network to run chaincodes. Within the same network, there's no further tenancy roles.
Not really, cello now support two roles now. 1) `operator`: that can manage the networks, and 2) `users`: that use a single fabric network to run chaincodes. Within the same network, there's no further tenancy roles.
Thanks @baohua . If I understood correctly so multi-tenancy is done on the chaincode or channel level only. So, when I deploy a new chaincode, I also have to fill chaincode size, which means on how many peers it will install the chaincode?
if you wanna install a cc to a peer, need to explicitly specify the peer address in SDK.
Hi @baohua I figured out that all the certs are packed inside allcerts.tgz for setup inbc1st.yaml and its being pulled from builder machine how can I custoimize file in it?
Hi @baohua I figured out that all the certs are packed inside allcerts.tgz for setup in bc1st.yaml and its being pulled from builder machine how can I custoimize file in it?
yeap, now we only support fixed credentials. will extend to dynamic generation soon.
ok can I generate those in my machine locally and put it in builder machine
as allcertz.tgz
yes, that's the possible way i can see now.
ok
@baohua I find that there are no orderer certs in keyfiles directory how does it works ?
i guess there is: https://github.com/hyperledger/cello/tree/master/src/agent/docker/_compose_files/fabric-1.0/crypto-config/ordererOrganizations/example.com
I feel this is nowhere related to example setup of orga,orgb,orgc,orgd that is there in bc1st.yaml I am talking about allcerts.tgz in builder machine
only these are the directories present `firstchannel.tx genesis.block orga orga.anchor.tx orgb orgb.anchor.tx orgc orgd
`
only these are the directories present `firstchannel.tx genesis.block orga orga.anchor.tx orgb orgb.anchor.tx orgc orgd`
only these are the files and directories present `firstchannel.tx genesis.block orga orga.anchor.tx orgb orgb.anchor.tx orgc orgd`
@baohua can you please address this above query
Hi, I will restructure my question above. I want to be able to access my fabric network which is running on k8s cluster from OUTSIDE the cluster. However, since we are using ClusterIP in the setup, I believe that it is not possible to do this. Can you help me with this situation? Thanks you
Has joined the channel.
Hey, we have deployed cello and added host successfully.
How do we instantiate Chaincode ?
@DarshanBc aha, seems your question is about fabric: what these credentials be used, i would like to suggest you take a look at the doc here: http://hyperledger-fabric.readthedocs.io/en/release/build_network.html
@baohua I have used hyperledger-Fabric in normal single host environment but setup of orga,orgb,orgc,orgd is unlike Fabric I am not able to find this folder /crypto-config/ordererOrganizations/example.com in allcerts.tgz that is present in builder machine apart from that all the files are similar to files under cryptoconfig/ in fabric
which `allcerts.tgz`?
`/opt/gopath/bc1st/fabric` in builder machine
I think its not using certs to bringup orderer
is there any particular reason why its not used ?
I see that this parameter is not included ``` - ORDERER_GENERAL_TLS_PRIVATEKEY=/etc/hyperledger/crypto/orderer/tls/server.key
- ORDERER_GENERAL_TLS_CERTIFICATE=/etc/hyperledger/crypto/orderer/tls/server.crt
- ORDERER_GENERAL_TLS_ROOTCAS=[/etc/hyperledger/crypto/orderer/tls/ca.crt, /etc/hyperledger/crypto/peerOrg1/tls/ca.crt, /etc/hyperledger/crypto/peerOrg2/tls/ca.crt]```
if it needs to be included how can this be added since all the peers are distributed in network ```- ORDERER_GENERAL_TLS_ROOTCAS=[/etc/hyperledger/crypto/orderer/tls/ca.crt, /etc/hyperledger/crypto/peerOrg1/tls/ca.crt, /etc/hyperledger/crypto/peerOrg2/tls/ca.crt]```
Small PR: https://gerrit.hyperledger.org/r/#/c/17957/ I plan to tweak docs around this as well, but figured doc changes should be in a separate PR.
@bleggett added some comment, please take a look. thanks.
Any documentation on how the network gets setup using cello. I am using the azure and bc1st profiles. I need this information to figure out how to configure the network_config.json file in the node sdk
@tongli after setting up bc1st profile how to create a channel install chaincode and instantiate it?
there is an channel and chaincode already installed
look at the setupfabric.yml file and see how it was done.
look at this file cello/src/agent/ansible/roles/deploy_compose/fabricsetup/tasks/apply.yml
at the very end, it boots up a fabric-tool container and run a script.
which can be found in the dochannel.j2 file.
How do we connect the new chain created on Baas to the network created from Cello ?
Clipboard - February 16, 2018 12:35 PM
Hello again, I have found a way to expose the K8s services outside the cluster by using Nodeports and I got it working. However, even after I placed all the required certificates ( I am using tls) I am still getting SSL errors when I am trying to connect to the peers from my NodeSDK application. I am guessing it has something to do with the hostnames of the peers. When working with AWS should I be pointing the hostnames as the clusterIPs or the publicIPs given by EC2. Thanks
@tongli and @baohua but still is is not instantiated how to do it [ ](https://chat.hyperledger.org/channel/cello?msg=zdNXKpAxvLtrZvAAA) @tongli
@tongli and @baohua but still is is not instantiated how to do it [ ](https://chat.hyperledger.org/channel/cello?msg=zdNXKpAxvLtrZvAAA)
Has left the channel.
What kind of errors are you getting?
interesting article on cello: https://www.linkedin.com/pulse/blockchain-meets-devops-hyperledger-project-cello-alan-dickman/?trackingId=RdEHE%2BNdyKL0lipPNLDUrg%3D%3D
interesting article on cello: https://www.linkedin.com/pulse/blockchain-meets-devops-hyperledger-project-cello-alan-dickman
Has joined the channel.
Hello, I'm deploying using `Makefile` as shown here (http://hyperledger-cello.readthedocs.io/en/latest/installation_master/). the deployment works fine and all services are running but the operator dashboard is not accessible, even if the docker providing that service is up and running. any idea why ?
the default deployment uses `docker-compose.yml` from cello/ folder. when I change to development mode to use `docker-compose-dev.yml` I can access the operator dashboard. Is there a known issue on the with the operator dashboard while using production mode ? Thank you
the default deployment uses `docker-compose.yml` from cello/ folder. when I change to development mode to use `docker-compose-dev.yml` I can access the operator dashboard. Is there a known issue on the operator dashboard while using production mode ? Thank you
When is the next stable released planed ?
@tongli I've resolved the issues, it was indeed about the hostnames of the peers.
@FiratSertgoz glad to hear that.
i have some ec2 instances, one with the cello master running, and three more in a docker swarm (1 master, 2 workers).
i've tried to use the cello dashboard to create a chain on the docker swarm, but after i choose to create the chain, nothing happens
i created the "swarm" host with type "SWARM", capacity 10, and swarm_master_ip:2375. is that the right setup?
Also, where can I find logs to see what's going on? I looked at "docker logs container-id" for the containers on the cello master, but nothing jumped out at me.
you can run `make logs` at master node to check logs
@baohua can cello be used in product environment now?
Where can I find information on the networking setup by cello. How are the different servers able to connect using the private IP?
Do all the servers have to be in the same subnet?
Any example where servers are setup in different networks?
Has joined the channel.
I've set up the master node, but when I try to connect to localhost:8080, I get a message from Firefox saying the connection was reset. There isn't anything interesting in the log for the operator dashboard container. All it contains is:
* Running on http://0.0.0.0:8080/ (Press CTRL+C to quit)
* Restarting with stat
* Debugger is active!
* Debugger PIN: 190-633-246
I can docker exec into the container and can see that Python is running: python dashboard.py
Any suggestions on how to figure out what's wrong?
@toddinpal issue `make log` to understand better where you are failing.
Hi team, how is the ansible cello project related to cello project? I 'm keen on using them both but documentation is not helping me. My primary goal is to setup a fabric cluster using ansible cello on k8. It's working on AWS but how do I use the same on local physical servers? It doesn't work if I provide it a readymade host file.
Has joined the channel.
@hightall r u around?
@hightall would like to get a bit more information on the blockchain explorer function, do you have few screenshots that I can use?
Hey guys, I`m new in this cello journey
Nice to mee you all
I`ve working on the first steps, and i have receive this error, i believe this is related to ssh
[192.168.0.6]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: ssh: connect to host 192.168.0.6 port 22: Connection refused\r\n", "unreachable": true}
Does anyone have pass throw this problems ? Thank you very much
@Menniti are you using the ansible agent?
@tongli yes you are correct
you can post a bit more info here regarding your env, I will be able to help.
also a lot of docs available to help you out
especially this one https://github.com/hyperledger/cello/blob/master/docs/worker_ansible_howto.md
@tongli sorry, which kind information should I provide?
I`ve follow this instructions
https://github.com/hyperledger/cello/blob/master/docs/setup_worker_ansible_allinone.md
what are you trying to do with how many nodes etc.
ok, that is everything on one node.
is your server at 192.168.0.6?
can you ssh manually to it?
No, i don`t even try it
this should be some docker?
looking into a docker there is no containers there
@tongli here is some screenshot on Blockchain explorer
Clipboard - February 21, 2018 8:50 PM
And you can start up one in 3 steps:
```bash
1. git clone https://github.com/yeasy/docker-compose-files.git
2. cd hyperledger_fabric/v1.0.5
3. HLF_MODE=be make start
```
@Menniti if you have followed the steps in that doc, please tell me where you stuck.
@baohua that is based on that the fabric network is up running?
yes
BE is a query tool that allows u query some basic info for a running fabric network using the sdk.
it cannot allow to operate the network, neither with the chaincode.
@tongli
The step is: Stand up the fabric network
comands
ansible-playbook -i run/runhosts -e "mode=apply env=vb" initcluster.yml --skip-tags="resetconn"
ansible-playbook -i run/runhosts -e "mode=apply env=bc2nd" setupfabric.yml
read there again, i realize maybe i need to config the ssh
reconfig or adjust the path
@Menniti yeah, make sure that you can connect to the ssh manually, simply do command like this.
```ssh youruser@192.168.0.6```
if that is failing, you know that you did not even have ssh going.
that will not make you going far.
to diagnose, can use `ssh -v HOST_IP`, which will output more debug info.
to diagnose, can use `ssh -v USER@HOST_IP`, which will output more debug info.
@baohua if I stand up fabric network using ansible agent, how can I start using explorer?
@tongli thank you very much, it is not working.. I will figure out and try to solve that
Thanks for your time man!
@harsha There is nothing interesting in the logs. It seems if I go to localhost:80 I get the operator dashboard, but the tutorial says to use localhost:8080. What am I missing?
@toddinpal I thought we get cello dashboard on 8081 per http://cello.readthedocs.io/en/latest/dashboard_user/#dashboard-for-user
And you can verify by looking at port forwarding on `docker ps -a`
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3bf7ca9fe38a hyperledger/cello-user-dashboard "bash -c 'cd /usr/..." 11 hours ago Up 11 hours 0.0.0.0:8081->8080/tcp cello-user-dashboard
00478ae0b339 hyperledger/cello-watchdog "python watchdog.py" 11 hours ago Up 11 hours cello-watchdog
6386dd8a9bd8 hyperledger/cello-engine "python restserver.py" 11 hours ago Up 11 hours 80/tcp cello-engine
d2bacec9fbf7 hyperledger/cello-mongo "docker-entrypoint..." 11 hours ago Up 11 hours 27017/tcp cello_dashboard_mongo_1
00fed24b9bf3 hyperledger/cello-nginx "/bin/bash /tmp/do..." 11 hours ago Up 11 hours 0.0.0.0:80->80/tcp, 0.0.0.0:8080->8080/tcp cello-nginx
9a7dabc55850 hyperledger/cello-mongo "docker-entrypoint..." 11 hours ago Up 11 hours 127.0.0.1:27017-27018->27017-27018/tcp cello-mongo
2462260bdbd7 hyperledger/cello-operator-dashboard "python dashboard.py" 11 hours ago Up 11 hours 8080/tcp cello-operator-dashboard
[ ](https://chat.hyperledger.org/channel/cello?msg=ZxiLnkw4jrj9GGcoC) @toddinpal did you deploy using `Makefile`, like `make setup-master` in the master node ?
I got the same issue while doing the same, and this seems an issue for me while using production mode. which uses by `default docker-compose.yml` to build the network. When I use `docker-compose-dev.yml` it is working fine
I got the same issue while deploying this way, and this seems an issue for me while using production mode. which uses by default `docker-compose.yml` to build the network. When I use `docker-compose-dev.yml` it is working fine
I got the same issue while deploying this way, and this seems an issue for me while using production mode. which uses by `default docker-compose.yml` to build the network. When I use `docker-compose-dev.yml` it is working fine
and to use `docker-compose-dev.yml` change `DEV` variable `True` in `cello/.makerc/operator-dashboard` file
@Amir Yes, I used the makefile, make setup-master and then make start
OK, try this, change this line `DEV?=False` inside the file `cello/.makerc/operator-dashboard` to `DEV?=True` and do `make restart` in the master node
ok, thanks, I will
you are welcome. let me know if the result (if it works or not)
you are welcome. let me know if the result (it is works or not)
can someone explain me why the production mode don't give access to the operator dashboard (nor error in the logs) ?
Has joined the channel.
@tongli i don't think it's possible now.
[ ](https://chat.hyperledger.org/channel/cello?msg=TDqDC9vDXCHYMPdaR) @baohua what about other cello services like the operator dashboard and user-dashboard ? I deployed on 3 physical servers using ansible and flanneld as an overlay network. I can see the containers for the fabric network. but where are the containers for cello services ?
@toddinpal are u using the latest image? i found there's a bug in the nginx config, sorry for the inconvenience.
@toddinpal are u using the latest image? i found there's a bug in the nginx config, it is fixed now, pls pull latest code.
Is there anyone who is willing to demo Cello either via Skype or any other online means?
Hi folks!
It is not clear to me from the documentation, can I bind master node with worker nodes that I deploy using ansible?
@paul.sitoh @hightall is preparing some video tutorial, hope that can help you later.
Just need some primer to understand the basics for now. Hopefully from there we can pick up more knowledge by ourselves. At the moment, trying to guess what it does from source code is very painful
@paul.sitoh sorry to hear that. have you had a look at http://cello.readthedocs.io/?
I read it but I wasn't really sure what cello was supposed to achieve especially in relation to Fabric. For example, how does cello help in terms of 'consortium' based configuration.
Or is cello trying to solve another problem altogether?
Or is it simply a container orchestrator with no reference to fabric?
@paul.sitoh This is my understanding of cello, anyone correct me if I’m wrong
cello manages the infrastructure for blockchain network. i.e it provides the same view for blockchain network whatever the underlying system/server is.
Assume you have a heterogeneous infrastructure (mix of physical and virtual servers/containers) and you want to build a blockchain network on top of these machines. You can manually configure each of them for your blockchain network or use cello and have the same view to create your blockchain network. Plus additional features as described in the doc (hopefully more in the future)
May be the best analogy will be if you take openstack for virtual machines cello is the same for blockchain networks (currently focused on Fabric)
[ ](https://chat.hyperledger.org/channel/cello?msg=kznJYumHxtwPd8NsT) @baohua Thank you for the fix, it is working for me now
Tiny fix for doc bug injected by me: https://gerrit.hyperledger.org/r/#/c/18091/
@bleggett aha, i guess the original target is for python 3.x version. But i'm fine with the change, let's see @tongli 's comments.
[ ](https://chat.hyperledger.org/channel/cello?msg=yEYbpHeYcpBuW6cny) @Amir Hi I can see the analogy to OpenStack. What I am not clear is how Cello can deal with the concept of Consortium based network in the context of Fabric. For example, in Fabric there is this concept of a participating organisation instantiating, for want of term, a Fabric instance (Peers, Orderers, CA) and as the lead organisation then invites another organisation to instantiate their own Fabric instance and then join the lead one to form a business network.
[ ](https://chat.hyperledger.org/channel/cello?msg=yEYbpHeYcpBuW6cny) @Amir Hi I can see the analogy to OpenStack. What I am not clear is how Cello can deal with the concept of Consortium based network in the context of Fabric. For example, in Fabric there is this concept of a participating organisation instantiating, for want of term, a Fabric instance (Peers, Orderers, CA) and the lead organisation then invites another to instantiate their own Fabric instance and join the lead one to form a business network.
[ ](https://chat.hyperledger.org/channel/cello?msg=yEYbpHeYcpBuW6cny) @Amir Hi I can see the analogy to OpenStack. What I am not clear is how Cello can deal with the concept of Consortium based network in the context of Fabric. For example, in Fabric there is this concept of a participating organisation instantiating, for want of term, a Fabric instance (Peers, Orderers, CA) and as the lead organisation then invites another to instantiate their own Fabric instance and join the lead one to form a business network.
Also how cello can help with this http://hyperledger-fabric.readthedocs.io/en/latest/channel_update.html
I appreciate the I am Cello is probably and attempt to cater for other projects such as Sawtooth and not necessarily bound to Fabric.
I appreciate the I am Cello is probably and attempt to cater for other projects such as Sawtooth and not necessarily bound to Fabric. So it can't do the kind of configuration I mentioned.
I appreciate the I am Cello is probably and attempt to cater for other projects such as Sawtooth and not necessarily bound to Fabric. So it can't do the kind of configuration I mentioned but I don't have enough information to be certain so appreciate some advice.
@baohua As part of a previous change I had redone the docs to recommend python3 instead of python2, but I backed that out. Forgot to remove these two references
@baohua As part of a previous change I had redone the docs to recommend python3 instead of python2, but I backed that out. Forgot to remove these two references when I did that.
@baohua As part of a previous change I had redone the docs to recommend python3 instead of python2, but I backed that specific change out. Forgot to remove these two references when I did that.
hey guys,
How can i access the user dashboard ?
I`ve setup the
```
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6637ee97c431 hyperledger/cello-mongo "docker-entrypoint..." 21 hours ago Up About an hour 127.0.0.1:27017-27018->27017-27018/tcp cello-mongo
8302dda61c21 hyperledger/cello-watchdog "python watchdog.py" 21 hours ago Up About an hour cello-watchdog
95753a56a7f1 hyperledger/cello-engine "python restserver.py" 21 hours ago Up About an hour 80/tcp cello-engine
8c7c3c76c2b7 hyperledger/cello-operator-dashboard "python dashboard.py" 21 hours ago Up About an hour 8080/tcp cello-operator-dashboard
307535cd3b4b hyperledger/cello-nginx "/bin/bash /tmp/do..." 21 hours ago Up About an hour 0.0.0.0:80->80/tcp, 0.0.0.0:8080->8080/tcp cello-nginx
dcd652f3998d hyperledger/cello-mongo "docker-entrypoint..." 21 hours ago Up About an hour 27017/tcp cello_dashboard_mongo_1
```
But I have no idea how should i procedd to continue the things here
@bleggett exactly. and i have py2 and py3 locally, where i have to use pip3 to indicate the py3 env.
@Menniti could u try the latest code? e.g., `git pull` then `make start`. after that, can access `localhost:8080`
sure
make start
ops
t
Oh my god, worked
ahh you guys have fixed that right?
@baohua, thank you very much man
:)
@baohua, I`ve trying to create a chain... but it says the name that i`m choose is not valid, should i follow any patter?
alphabeta and number should be OK
fabric has some limitation on channel name, you can check its documentation
ah ok, i will take a look there
@baohua, I`ve looking into it, but I do not found any limitations for chaincodenames... I`m trying to put simple names, like menniti123 or abc123 or Abc123
none of the work
none of that worked*
@baohua I`ve discoverd the error, I need to restart the entire service, once I had previusly dockers containers running
Now, I`m trying to add a host here, and it gives me error in order to create the host.
As per documentation says, i should replace Daemon URL with docker, honestly i have no idea how what IP should a set there (obviusly I`ve searched in google before come here to ask help)
do you mean cannot add a docker host?
if you do not know what is the daemon url for a docker host, maybe can have a look at docker's documentation.
if you do not know what is the daemon url for a docker host, maybe can have a look at docker's documentation, e.g., https://docs.docker.com/engine/docker-overview/.
https://gerrit.hyperledger.org/r/18249
Are there instructions on how to use Cello with Swarm?
@baohua Nice!! thank you for sharing, i will study a little more about docker.
Thankss
@toddinpal it supposes to be the similar way with docker, but i've not got chance to verify with the latest version yet. welcome for any feedbacks.
@Menniti sure. and blockchain is a very broad areas including several techniques. cello even has more like those container techniques, cloud computing and orchestration.
Has joined the channel.
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=s6G3acXZA4TBoMznH) @paul.sitoh as far as my understanding the main functionality of cello is one step lower than the fabric layer (i.e infrastructure level). The only thing cello can provide for the fabric layer (for the functionalities of HLF blockchain ) is a nice user interface to facilitate the things you listed.
Has joined the channel.
Has joined the channel.
hey everyone :wave:
whenever I run the awsk8s.yml file for the ansible cello agent I get to a point where I get this message UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Permission denied (publickey).\r\n", "unreachable": true}
whenever I run the awsk8s.yml file for the ansible cello agent I get to a point where I get this message
`UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Permission denied (publickey).\r\n", "unreachable": true}`
when I check the ec2 instance involved I can see that it has the key from my local machine
does it have to have an ec2 key instead ?
[ ](https://chat.hyperledger.org/channel/cello?msg=rHC4y4vZckCwudEem) @Dilip 25 can you connect to the servers using ssh (just ssh without using ansible) ?
@Amir thanks Amir, but i'm having an issue deploying the fabric network
```TASK [Reset ssh connection to make group change take effects] ******************
changed: [XX.XXX.XXX.XX]
changed: [XX.XXX.XXX.XX]
changed: [XX.XXX.XXX.XX]
PLAY [build fabric artifacts] **************************************************
TASK [Gathering Facts] *********************************************************
fatal: [XX.XXX.XXX.XX]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Permission denied (publickey).\r\n", "unreachable": true}
to retry, use: --limit @/home/dukuo/cello/src/agent/ansible/awsk8s.retry
PLAY RECAP *********************************************************************
XX.XXX.XXX.XX : ok=67 changed=39 unreachable=0 failed=0
XX.XXX.XXX.XX : ok=94 changed=51 unreachable=1 failed=0
XX.XXX.XXX.XX : ok=67 changed=39 unreachable=0 failed=0
cloud : ok=26 changed=11 unreachable=0 failed=0
fabric001 : ok=6 changed=1 unreachable=0 failed=0
fabric002 : ok=6 changed=1 unreachable=0 failed=0
fabric003 : ok=6 changed=1 unreachable=0 failed=0 ```
so i'm unable to access k8s's dashboard
any ideas?
any ideas? (thanks so far)
@tongli i'm following your video tutorial btw
[ ](https://chat.hyperledger.org/channel/cello?msg=8ivihJRwx9dZ4Hdd8) @Dilip 25 yes the fabric deployment requires an ssh connection. If you are unable to connect using ssh, the deployment will fail
see that's the thing @Amir I can login via ssh to the ec2 instances created by the deployment
it's just that the deployment fails for an unknown key error
Has joined the channel.
@Dilip 25 let me know if you have any issues, I can certainly help with.
Hey @tongli ... indeed I'm having the same issue above
when buildinf fabric artifacts it seems that the builder instance does not have a public key
when building fabric artifacts it seems that the builder instance does not have a public key
but when I check `~/.ssh/authorized_keys` after ssh'ing to the instance it says that it has my public key created in the VM controller instance from my laptop
so maybe it's a key between instances ?
at first I thought it might have been related to the fact that I was using a different username than `ubuntu` on my laptop's VM, but after changing it back to `ubuntu` stil didn't work as expected
@Dilip 25 what env. do you have and how did you run the playbooks?
@Dilip 25 a bit more info can really help me to understand what you were trying to do and help you.
@tongli I did nothing more than installing ansible and hardcoded the AWS password in the aws.yml file for testing purposes
could you provide me with a link to install it properly @tongli? because in your video you already have ansible installed
@Dilip 25 this doc explains a lot and have step by step instructions on how to set up ansible. https://github.com/hyperledger/cello/blob/master/docs/worker_ansible_howto.md
please take a look at the doc and ask specific questions such as how many nodes you like to create and what specific errors that you see when you run what commands.
thank you @tongli ! I just started a fresh VM and was using this as a reference http://hyperledger-cello.readthedocs.io/en/latest/ansible-agent-all-in-one/ but followed the same steps as the link you gave me so far, so now i'll switch to the github version
@Dilip 25 if you are following that all in one, will be nice to know where you stuck and error msg.
[ ](https://chat.hyperledger.org/channel/cello?msg=izS3jyrhctxT5LpXC) @tonli
that was the error I was getting following the first one
now i'm stuck here https://github.com/hyperledger/cello/blob/master/docs/worker_ansible_howto.md#provision-the-servers
I get this error
```TASK [Gathering Facts] *********************************************************
ok: [54.89.133.106]
TASK [include] *****************************************************************
included: /home/ubuntu/cello/src/agent/ansible/roles/cloud_os/namenode/tasks/apply.yml for 54.89.133.106
changed: [34.205.43.130] => (item=172.30.0.64 fabric001)
changed: [54.89.133.106] => (item=172.30.0.64 fabric001)
TASK [Gathering Facts] *********************************************************
fatal: [35.170.79.111]: FAILED! => {"changed": false, "module_stderr": "Shared connection to 35.170.79.111 closed.\r\n", "module_stdout": "sudo: unable to resolve host fabric001\r\nTraceback (most recent call last):\r\n File \"/home/ubuntu/.ansible/tmp/ansible-tmp-1519669073.22-278970945054401/setup.py\", line 17, in
did all the instructions above
installed ansible, created the key, added to the agent, cloned the repo and copied both cloud & fabric .yml files (aws.yml and bc1st.yml to mycloud.yml and myfabric.yml)
then I added my aws ID & aws_secret_key added as an env variable
finally I ran `ansible-playbook -e "mode=apply env=mycloud cloud_type=aws" provcluster.yml`
and got the error I posted above :/ @tongli
you are following the all in one instruction?
i'm following the https://github.com/hyperledger/cello/blob/master/docs/worker_ansible_howto.md#provision-the-servers link you provided
hmmm, it seems to me that is not the case, you are doing 3 nodes.
yup, i'm not doing the all in one, i'm doing the one you did in your video
with fabric001 to fabric003
ok.
so you have one VM as your ansible controller and trying to create 3 VMs to stand up fabric network?
exactly
all 3 are currently running even after the error, but obviously can't access the builder k8s dashboard because of the error
after that error, did you actually see any new VMs coming up from aws console?
ok.
did you start up your ssh-agent?
yes
on ansible controller?
yup, it is running on the controller
in that case, you can try to manually ssh to one of the server to test your ssh-agent.
do this.
```ssh ubuntu@54.89.133.106```
i'm in
if your ssh agent is running correctly, you should be able to do this from your ansible controller
yup, I logged to fabric002
good. good. actually I think you are following the right steps and setting up right stuff.
the problem is this.
when you use a ubuntu 16.04 cloud image to create new VMs, there is a cloud-init script to install python during the provisioning step.
in many cases, the python installation does not finish after the machine is ready because apt install works in background.
to overcome this issue. you can do the following
1. create a VM on your aws cloud using ubuntu 16.04 image
2. once the VM is created, log in manually and do the following
```apt update```
```apt install python```
then you can save that as an image.
all right
a question though... VM as in EC2 instance ?
yeah. EC2 instance.
oh okay, i'll do it right away
once this is done, I am sure it will go as expected since all other steps are correct.
glad that you followed these steps.
by the way, should I use that VM as the ansible controller instead of my local VM ?
the python issue is more serious now because ubuntu 17.10 was out for awhile, and any VM boots up, ubuntu will want to check updates and things and want you to upgrade.
this is a big headache.
or should I use it as a template for the fabric nodes
once you create an image, you will want to use that image to create all fabric nodes.
change vars/aws/yml file to use your own image.
vars/aws.yml
ooooohhh
now that makes sense
i'll do it right away, thanks for taking the time so far @tongli
so that all the VMs will have python already in. you do not have to wait for anything.
I am planning to try using 17.10 but have no got time yet. but in any case, ansible requires python on the remote host, we will need python.
would this be okay @tongli ? http://prntscr.com/ik2lgw
Canonical, Ubuntu, 16.04 LTS, amd64 xenial image build on 2018-01-26
well, I think if you are doing this in aws, you will need to use a image that was provide as base to create your own new image which simply has python installed.
I have never tried to upload an image onto aws.
but 16.04 amd64 server is what we normally use.
so, I created the instance, ssh into it, install python then save it as a base for the fabric nodes, right ?
alright
yeah. exactly.
that is how new images get created on aws.
you start with aws provided images.
okay so everything happened successfully
only that I can't login to k8s dashboard on the fabric001 node
@tongli
should be 35.153.103.169:8080/ui, right ?
(35.153.103.169 is the builder node as my log indicates)
oh wait, provclusters.yml doesn't provide k8s dashboard
I need to keep following the github tutorial I guess
or destroy everything and run awsk8s.yml instead of provclusters.yml
oh, cool.
the port is the https://
@Dilip 25 if you want it to be really simple, destroy the 3 nodes.
and run this one command
```ansible-playbook -e "mode=apply" awsk8s.yml```
but you will have to make sure that vars/aws.yml and vars/bc1st.yml are all correct.
that command is a short cut to have everything done in one command.
which has k8s setup and fabric installed.
alright lemme try that
thanks @tongli
@Dilip 25 yeah, get rid of these 3 nodes first.
otherwise, it will create 3 more nodes and you will be paying a bit more.
the flavors that you choose should have 4 gb of memory
the free tier is way too small.
yeah I did t2.medium
let's see if it works
t2.medium is fine.
btw I did a little modification to allow image_id instead of image_name... in cello/src/agent/ansible/roles/cloud_aws/prepare/tasks/apply.yml:22 change ami_name to ami_id then in vars/aws.yml change ami_name to ami_id
because it's easier to point the AMI by id than by name
that is fine. if these changes are necessary, u can propose a patch set.
so we can make the change in the repo and everyone can benefit.
yeah that's a good idea
but you will have to make sure that we can do in either case.
will propose that when I get this working
yes that's the point
keep me posted , will be very happy to see it is working for you.
I am also planning to install composer on top of the fabric network which ansible sets up.
once that part is added, you will be able to start developing your business blockchain app after the script finishes.
that will be really really nice.
it worked! @tongli
only thing different with your video is that it asks me for kubeconfig or token
which should I choose ?
http://prntscr.com/ik4a49
use token,
the token is in the secrets/users directory file.
uhm this is odd
I enter the token, press SIGN IN but no response happens
even though on the network tab I can see a 200 OK response
@tongli or should I inject it in the HTTP headers?
weird thing is that I can skip it and it works flawlessly
yeah. you probably won't be able to do anything?
well i'm able to view every section
without the need to login
which i'd say it's a no-no
I searched for the token in secrets/user/token.csv
entered the first column (which is a encrypted string) to the textbox but with no luck
hmmm. that is what is supposed to be.
it is very strange that it would work without token.
yeah it kinda is
i'll try now without inserting the token at all
because since it throws a 200 OK when I press the SIGNIN button then it might mean that it's working but it's not hiding the login modal
hi,If I use the docker for mac ,how do I find the docker host address?
@baohua @hightall found couple of issues recently on ansible agent. fixed them by this patch , please take a look https://gerrit.hyperledger.org/r/#/c/18459/
@baohua @hightall thanks a lot.
hey @tongli I sent you a DM, although I post the same here... I want to start helping in the cello ansible project, preferably integrating composer & the REST server in the deployment script ... is there any info you can provide me to start looking? Thanks!
@baohua @hightall we are running large number of nodes , that patch set will really help. please review when you have few minutes. thanks.
(by the way, deployed the network again and t he auth is not working properly, http://prntscr.com/ik98up SIGNIN button does not work, and I got the token from the ansible controller VM @ `$AAROOT/secrets/users/token.csv`)
(by the way, deployed the network again and the auth is not working properly, http://prntscr.com/ik98up SIGNIN button does not work, and I got the token from the ansible controller VM @ `$AAROOT/secrets/users/token.csv`)
Hi I have setup fabric over multiple vm now how can i invoke chaincode on them
+2! @tongli
@tongli Sometimes I see fabric-cli container running sometimes no
@baohua Hi baohua, we are going to contribute kubernetes agent code to cello, and I noticed that the status of jira item "Support kubernetes as the underly platform(CE-7)" is done, should I open a brand new jira item or create sub-project under that item?
sure, open a new one.
Thanks~
and before that, it would be nice to help refine existing code to implemented from the same Base class.
@baohua, are you still around?
@baohua @hightall, made a stupid mistake and misplaced a double quote in the docker login command, fixed that in this patch, please take a look, sorry about this mistake. https://gerrit.hyperledger.org/r/#/c/18463/
Has joined the channel.
@tongli minor change needed: https://gerrit.hyperledger.org/r/#/c/18463/
sure.
@baohua fixed "Fix" https://gerrit.hyperledger.org/r/#/c/18463
+2
thx
@baohua is there any hyperledger meetup scheduled in China? Thinking about participating.
yes, we plan one on Mar 10 in Dalian, and hopefully another one in Beijing at the end of Mar
When I use ansible script to bring up containers there is a fabriccli container up and running to install and instantiate transaction after that it disappers when I try to bring it up again with `docker-compose -f file.yaml -up -d` it says container is already running when I do `docker -it fabriccli bash `I says it container is not running If I start it with `docker start fabriccli` I starts but never shows up
I am doing this to invoke chaincode
@baohua can you help me
sorry, cannot quite follow ur question. so which command is not as expected?
I am trying to invoke transaction on chaincode hence I need cli container to login and execute commands I am not able to bring up cli container
@baohua
@baohua this is what I am trying to do
Has joined the channel.
Has joined the channel.
@DarshanBc please take a look at the docker compose file on how to bring up the fabriccli.
the file should be in the working directory in /opt/gopath/bc1st/fabric/run
do not have the exact path, but it should be there on the first node. take a look at that and see how fabriccli is brought up, you can do the similar thing
Has joined the channel.
Hi, I have a question on how to connect Cello to an existing running fabric instance? I only see Cello as a tool to provision new fabric instances (worker nodes) but unable to connect to an instance which was not provisioned through Cello. Is this correct?
I have fabric instance node1 and would like it to be added as a worker node to my master. But there's no way for the master node to get the existing instance through the docker daemon.
@arunmitteam It can be done, but at this point that is somewhat outside the scope of what Cello does. It is probably quicker and simpler to simply provision a new cluster from the ground up with Cello.
Has joined the channel.
@bleggett Thanks for confirming!!
Is the ansible agent used to setup worker nodes or it is a separate tool (separate from Master-Worker architecture ) used to just setup fabric network ? the doc (http://hyperledger-cello.readthedocs.io/en/latest/ansible-agent-how-to/) doesn't mention the relation. Thank you
@Amir This doc is a little more up to date: https://github.com/hyperledger/cello/blob/master/docs/worker_ansible_howto.md If that doesn't answer your question, let me know and maybe we can clarify
@bleggett Thank you for the response. In fact I used that doc and I managed to setup fabric network on multiple physical machines. After setting up the network I was expecting to see active chains under cello operator dashboard (assuming that the ansible agent is configuring worker nodes ) and the fabric network on top these chains. But I didn’t saw anything under the operator dashboard.
I'm I correct or the ansible agent is separate from Master-Worker setup ?
@Amir I think you must manually point the Cello dashboard at your network.
@Amir I think you must manually point the Cello dashboard at your network, after you stand your network up with Ansible.
https://gerrit.hyperledger.org/r/18663
@lujiang add comments
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/18663/
https://gerrit.hyperledger.org/r/18681
+2
Has joined the channel.
Has joined the channel.
@arunmitteam ansible agent sets up a fabric network on multiple nodes but the network was not hooked up with the dashboard. It will be nice if someone started working on the integration.
I have started adding fabric composer onto the ansible created network.
@tongli ok thank you
Has joined the channel.
Has joined the channel.
Hello Everyone
I am excited about Cello project. I have experience working with Kubernetes, Docker and large enterprise cloud platform development
How can I get started contributing on Cello?
@ashishapy probably go through the docs a bit, then pick an area that you have a feel with, then start fixing up things. I could use some help with the ansible agent.
https://gerrit.hyperledger.org/r/18705
https://gerrit.hyperledger.org/r/#/c/18707/
add some comments!
@baohua recommit
cool
I really like the cello operator dashboard, it helps us to spin off fabric instances very quickly.
I have two questions - 1. What is user dashboard? When should I use it?
2. Can I use cello to create new channels? Or add peers to existing channel?
Has joined the channel.
Has joined the channel.
@hightall u're the appropriate to answer these questions, as the original author :)
@baohua :- I don't see `v0.8.0-beta` been cut for cello yet ?
@baohua :- I don't see `v0.8.0-beta` release tags for cello yet ?
Has joined the channel.
Has joined the channel.
Hi @tongli , I have same problem than @Dilip 25 fatal: [XX.XXX.XXX.XX]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Permission denied (publickey).\r\n", "unreachable": true}
and I try to create a new AMI with python installed
but still the same problem
Either your host does not support ssh or you do not have ssh agent running
Please look at the docs
So you are creating vms In aws?
yes
Were your vms created?
yes, I created a ubuntu 16.04 with apt install python
so I need to do a apt install open-ssh?
So your vms were all created and you just can not connect to them ?
If you are using aws image, ssh is there.
You can try manually ssh to one of the vms and see if you can do it.
ok I'm testing
Make sure that you have ssh agent running on the ansible controller which is the machine you run the script
ssh to fabric001 OK
Yeah. That means you vms are ok. Go ahead ssh-add your vms private key to ssh agent
You should be ok
Again, the doc has all the steps how to do it
http://hyperledger-cello.readthedocs.io/en/latest/ansible-agent-how-to/
this one?
There is an even more up to date one on github cello/docs directory
Last doc in that directoey
Take a look
https://github.com/hyperledger/cello/blob/master/docs/worker_ansible_howto.md
this one right?
Right
Take a look at ssh agent section. Should solve your issue
Has joined the channel.
@harsha just there: https://github.com/hyperledger/cello/releases/tag/v0.8.0-beta
hello all, I updated fabric-solo-4.yaml file (https://github.com/hyperledger/cello/blob/master/src/agent/docker/_compose_files/fabric-1.0/fabric-solo-4.yaml) to use couchDB as the state database (this is the file used to create chains from operator dashboard). But when I create chains it never starts the couchdb containers. any Idea on how to use couchDB ? thank you
There might be sth wrong in ur updated yaml file. maybe u can have a look at https://github.com/yeasy/docker-compose-files/blob/master/hyperledger_fabric/v1.0.6/docker-compose-2orgs-4peers-couchdb.yaml
https://gerrit.hyperledger.org/r/#/c/18765/ -- Request for review, update Makefile to comply by policy.
[ ](https://chat.hyperledger.org/channel/cello?msg=8yYahZ8djHPDA3hhX) @baohua Thank you for the reply. I will check with that file. In fact to test my file I did `docker-compose -f fabric-solo-4.yaml up -d` and everything starts fine but then this will not be associated with the dashboard.
Has joined the channel.
Has joined the channel.
@baohua @hightall need to add a flag to setup peers and orderers logging level. here is the patch, please take a look , thanks https://gerrit.hyperledger.org/r/#/c/18775/
@baohua @hightall we are running system test with 32 orgs and 64 peers now.
@baohua @hightall please take a look at this simple patch, needed it for our next round of tests. thanks https://gerrit.hyperledger.org/r/#/c/18775/
Has left the channel.
Has joined the channel.
@here Hyperledger Cello v0.8.0-beta is released, and thanks to every developer. Welcome to try and report issues: https://github.com/hyperledger/cello/releases/tag/v0.8.0-beta.
glad to here that
@tongli merged, thanks!
Thanks bro
@baohua good job!
Has left the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=T5cyDoaoAMXyySZqX) @baohua that's nice bro
@here Dear all, we will have weekly group meeting 8:30PM--9:00PM Friday evening (UTC+8), welcome to join: https://wiki.hyperledger.org/projects/cello. Thanks!
†Got it:relaxed:
@baohua 0830 - 0930, is 02AM in IST. Is there any other alternative timeslot, or is this agreed timings for all ?
oh, sorry, i though u are at US, we discussed by Asia team and US team. seems we should consider more developers...
btw, IST is?
and we also seek community help to find some free time slot at zoom system.
IST - India :)
Has joined the channel.
@harsha if i'm right, 8:30PM--9:00PM Friday evening (UTC+8) is 5:30--6:00PM in india?
[ ](https://chat.hyperledger.org/channel/cello?msg=8yYahZ8djHPDA3hhX) @baohua I tried to use these compose files (with both v1.0.6 and v1.1.0-rc1) to create fabric network from operator dashboard but the dashboard doesn't create the couchdb containers. It always create the following (4 peers, 2 ca, orderer/ kafkas, explorer and cli ).
When creating fabric network (cluster) from operator dashboard, is the dashboard using these files (https://github.com/hyperledger/cello/tree/master/src/agent/docker/_compose_files/fabric-1.0) ?
@baohua @hightall please see this issue and patch set. https://gerrit.hyperledger.org/r/#/c/18865/ https://jira.hyperledger.org/browse/CE-294?filter=-2
How do we create a new channel or invoke transaction in current businesschannel ?
@Amir yes, it uses the path.
@tongli +2,thanks!
Tmr's scrum meeting agenda can be found at https://wiki.hyperledger.org/projects/cello/meetings/20180309.
Has joined the channel.
Has joined the channel.
@harsha when create Active Chains, how to put peers to different machine? now, all peers、order、ca at the same machine
@liujinzhong Please use ansible way to deploy the peer/orderers/ca across diff machine.
@harsha thank you very very much!! Three days I read the code of cello,but so many problems
@harsha is there any demo or UrlLink can be introduced?
@liujinzhong
http://cello.readthedocs.io/en/latest/setup_worker_ansible/
http://cello.readthedocs.io/en/latest/worker_ansible_howto/#cello-ansible-agent-how-to
@harsha Get it ! OK,Let me see
[ ](https://chat.hyperledger.org/channel/cello?msg=e3PkxEAkmc9idrARS) @baohua Yes. you are right.. India is GMT + 5:30
any one tried to use an updated version of these files (https://github.com/hyperledger/cello/tree/master/src/agent/docker/_compose_files/fabric-1.0) to create chains from the operator dashboard ? I tried to include couchDB but the dashboard is not creating it. Thank you in advance
@Amir it's ok you do the testing, but a more recommended way is to open some jira issue for the bug, and submit a patchset, then people will feel easier to help understand the problem and situation.
Has joined the channel.
is cello-k8s ok or it is only draft?
@liujinzhong There are only some scripts so far, which can not be called by the Cello runtime
but the feature to support kubernetes agent is under developing now.
@Luke_Chen I read the document of setup_worker_ansible.md several times but cant't get what i want.
peer,order and so on deployed on different machines....this is what i need
the ansible agent is different from the native kubernetes agent.
@Luke_Chen when i using dashboard,only create host and chain.at the same time they are on the same local machine
@Luke_Chen cell,ansible,k8s....how to tie together?
There is a "agent" layer in cello's arch, some APIs are implemnted here to communicate with remote host, and the host can be the machine which is running the docker service or being the master the node of a kubernetes cluster.
There is a "agent" layer in cello's arch, some APIs are implemnted here to communicate with remote host, and the host can be the machine which is running the docker service or being the master node of a kubernetes cluster.
[ ](https://chat.hyperledger.org/channel/cello?msg=e3PkxEAkmc9idrARS) @baohua is the meeting on?
There are only 2 participants in zoom call.
the time is after 20 mins
@baohua Gotcha, 0600 PM IST, ty
@baohua Will we have some sort of playback for the Meeting ?
@baohua Will we have some sort of recorded for the Meeting ?
@baohua Will we have some sort of recording sessions made available for the meetings ?
@harsha oh, i think there's recording, but i cannot find the link, maybe you can ask ry. Btw, there will be minutes in the wiki page after the meeting.
@tongli here are previous quarterly report: https://wiki.hyperledger.org/groups/tsc/project-updates
great. I will take a look and create the page accordingly.
Has joined the channel.
@baohua @hightall here is the new page I created for the project update
https://wiki.hyperledger.org/groups/tsc/project-updates/cello-2018-mar
@baohua @hightall it is now showing gree on this page. https://wiki.hyperledger.org/groups/tsc/project-updates
let me know if you like to change anything.
@tongli 1). we have a cello meetup on Jan, 2018, may mention it. 2) And we may ask the TSC whether can apply for some vm instance to run the cello testbed.
btw, when will u visit China? beijing has a meetup at Mar 24, hope you can attend it.
I was referring the regular weekly meeting.
oh, the Mar meeting is way too close, how about next one? IN may?
I can add the request for more machines for test bed.
ok, and you can add some hyperlink into the wiki's content, in case people may find more details. E.g., release history, meetings
overall, it's good for me
I added the machine request in the plan section.
yeah, if you can add these links , that will be great. you want to add these links at the additional information section?
seems that will be a nice place to add.
oh, let me help
great. thanks.
:thumbsup:
So during the update, one would basically talk about the content on this page, is that the correct understanding?
yes, introduce the content, like a quick sync report to tsc
i've updated, feel free to keep refining.
ok, I see, I was sure what you meant by the links. looks very nice. :thumbsup:
sure
[ ](https://chat.hyperledger.org/channel/cello?msg=ukdTDy5FM2RdE7RF6) @baohua OK thank you. I will do that
Regarding the issue of using couchDB with the operator dashboard, I look the jira issues if someone added the case before and I found one from Jan added by @harsha (https://jira.hyperledger.org/browse/CE-250) , I updated the description on that issue. I will try if I can resolve the issue. Thank you
Regarding the issue of using couchDB with the operator dashboard, I looked the jira issues if someone added the case before and I found one from Jan added by @harsha (https://jira.hyperledger.org/browse/CE-250) , I updated the description on that issue. I will try if I can resolve the issue. Thank you
Thanks @Amir !
Has joined the channel.
Has joined the channel.
I apologize for the random question but what is the best approach for seeking assistance with cello?
I'm trying to find out about a 404 error we keep getting when using Docker. Seems a container is identified and then dies. the docker py function throwing the error is get_swarm_node_ip.
404 Client Error: Not Found ("No such container: f9ff9571ebe8455b98b85e8219c7a43d_peer0.org1.example.com is the exact error message
Has joined the channel.
Has joined the channel.
fix the incompatible with vsphere host type https://gerrit.hyperledger.org/r/#/c/19031/
[ ](https://chat.hyperledger.org/channel/cello?msg=dnFNaMHDy9ZiL65ep) @Amir FWIW, most of the work by me is already completed, however I kept it aside so it can be worked upon once v0.8.0 is released. this shouldn't be much of work, however you can always DM me on this issue. Let's see how we can take it ahead.
Hi Guys, I made some minor typo fixes in the docs. Hope you can review it. The gerrit item is at: https://gerrit.hyperledger.org/r/#/c/19047/
@baohua @hightall please review this patch which I simply updated the configuration to etcd and flanneld to a latest version. thanks https://gerrit.hyperledger.org/r/#/c/19067/
+2! @tongli
@baohua thanks a lot.
https://gerrit.hyperledger.org/r/#/c/19047/
+2! @yopep
thanks
https://gerrit.hyperledger.org/r/#/c/19031/ @Luke_Chen +2, thanks
@baohua thanks a lot.
@Luke_Chen https://gerrit.hyperledger.org/r/#/c/19031/ +2
https://gerrit.hyperledger.org/r/#/c/19047/ @yopep +2
Congrats to @yopep for his first merged patchset in project Hyperledger Cello: https://gerrit.hyperledger.org/r/#/c/19047/!
Congrats to @yopep for his first code contribution in project Hyperledger Cello: https://gerrit.hyperledger.org/r/#/c/19047/!
Congrats to @yopep for his first code contribution to project Hyperledger Cello: https://gerrit.hyperledger.org/r/#/c/19047/!
I hate to be annoying here but trying to wrap my head around this. Anyone encounter `404 Client Error: Not Found ("No such container: f9ff9571ebe8455b98b85e8219c7a43d_peer0.org1.example.com is the exact error message` when attempting to work with docker swarms
It looks like it's creating a container but does not have .example.com on the container name
lulz. I'll even take a "Docker isn't fully supported despite docs" answer. haha. I just feel completely in the dark at this point
lol
Has joined the channel.
I'm getting that issue too. There doesn't seem to be a whole lot of documentation on setting up a Docker swarm as a host.
get_swarm_node_ip
I don't think that Cello is ready for docker out of the box. The "hard-coded" example seems to be referencing peers by name that don't really get created by said name.
reference cluster.py
@baohua Is this true? Does Cello support docker swarms yet?
@AlexBA 6 @ChrisFowers if you use ansible agent, you can setup your own network anyway you like
does not matter you use multiple nodes or just one node.
please see the docs here.
https://github.com/hyperledger/cello/blob/master/docs/worker_ansible_howto.md
https://github.com/hyperledger/cello/blob/master/docs/setup_worker_ansible_allinone.md
if you follow these documents, you should be setup your own fabric network anyway you like.
in terms of docker swarm , ansible agent create a docker swarm like env (not exactly the docker swarm), it uses overlay network so that containers have their own IP address, it also sets up local dns and registrators so that peers and orderers can work with each other by using names.
if you follow these documents and have issues, I can certainly help with.
Right on. Thank you for the reply. I assume by that Docker support for swarms isn't quite there then?
Has joined the channel.
Builds subsequent to today will allow you to hide Jenkins spew in the Gerrit UI natively. You need to be using the old UI. use the "hide tagged comments" button. if you want to see how it works, examine this change using the old UI: https://gerrit.hyperledger.org/r/#/c/19059/
Has joined the channel.
@AlexBA 6 yes, we've tested cello with the swarm 1.12 version, but haven't tested with the latest one. It would be nice if you have any feedbacks on the new swarm cluster. thanks!
How do we create chain for Host which is on different network?
We used Ansible and could ping all the other worker nodes.Created a host for different VM but not able to add chain to the newly crated host.
Has joined the channel.
Has joined the channel.
Hi,
I am unable to add a docker host to cello admin dashboard.I am using Ubuntu 16.04 and did changes as per tutorial in /lib/systemd/system/docker.service
@pb check if docker port is open 2375
Has joined the channel.
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=L29dFh6zXswQG5LSu) @dinesh.rivankar
Thanks.I did -
sudo systemctl stop docker.service
$ sudo dockerd -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --api-cors-header='*' --default-ulimit=nofile=8192:16384 --default-ulimit=nproc=8192:16384 -D &
But encountered with a new problem ie unable to add chain i am getting "Failed to create cluster message".
Has joined the channel.
Has left the channel.
@pb what do you see in the logs when you try to create the chain ?
Has joined the channel.
Hi guys, created a bug ticket at: https://jira.hyperledger.org/browse/CE-299
It is about deleting a chain on a non-existing worker node. Can anyone take a peek and check if the expected behavior that I stated is correct. Thanks!
[ ](https://chat.hyperledger.org/channel/cello?msg=orNyPpzTMvFFNmgwB) @Amir
There is no error in logs which i viewed by make logs
But the status is still creating and no change at all.
@pb If you are using Chrome, can you open the developer console to check if there are any errors.
[ ](https://chat.hyperledger.org/channel/cello?msg=h9GKwnzX3zPzpxsqR) @yopep
ok..i checked its 404 not found !! while adding chain.
The terminal log is -
: 'fabric-solo-4.yaml', 'COMPOSE_PROJECT_NAME': 'd59e0daa7cb640dea97320e8b7581434', 'VM_ENDPOINT': 'tcp://192.168.20.247:2375', 'VM_DOCKER_HOSTCONFIG_NETWORKMODE': 'cello_net_solo', 'PEER_NETWORKID': 'd59e0daa7cb640dea97320e8b7581434', 'HLF_VERSION': '1.0.5', 'COMPOSE_PROJECT_PATH': '/opt/cello/fabric-1.0', 'CLUSTER_LOG_LEVEL': 'DEBUG', 'DOCKER_HOST': 'tcp://192.168.20.247:2375', 'CLUSTER_NETWORK': 'cello_net_solo', 'NETWORK_TYPES': 'fabric-1.0'}
Creating d59e0daa7cb640dea97320e8b7581434_peer1_org1 ... error
t/docker/_compose_files/fabric-1.0
cello-watchdog | Creating d59e0daa7cb640dea97320e8b7581434_orderer ...
cello-watchdog | Creating d59e0daa7cb640dea97320e8b7581434_ca_org2 ...
Creating d59e0daa7cb640dea97320e8b7581434_peer0_org2 ... error
cello-watchdog | Creating d59e0daa7cb640dea97320e8b7581434_mysql ...
Creating d59e0daa7cb640dea97320e8b7581434_peer1_org1 ...
cello-watchdog | Creating d59e0daa7cb640dea97320e8b7581434_peer0_org2 ...
Creating d59e0daa7cb640dea97320e8b7581434_peer0_org1 ... error
cello-watchdog | Creating d59e0daa7cb640dea97320e8b7581434_peer0_org1 ...
cello-watchdog |
cello-watchdog | ERROR: for d59e0daa7cb640dea97320e8b7581434_peer1_org1 Cannot start service peer1.org1.example.com: driver failed programming external connectivity on endpoint d59e0daa7cb640dea97320e8b7581434_peer1_org1 (7c02919dc8dd55513c50da8c5877f213794c32226fd3aef61ad6d03a458a54cb): Bind for 0.0.0.0:7053 failed: port is already allocated
cello-watchdog |
cello-watchdog | ERROR: for d59e0daa7cb640dea97320e8b7581434_peer0_org2 Cannot start service peer0.org2.example.com: driver failed programming external connectivity on endpoint d59e0daa7cb640dea97320e8b7581434_peer0_org2 (35d7f72c6bce0a8ac246abc7f2c6482549ac28ad688b575b75e3643d093dbf66): Bind for 0.0.0.0:7054 failed: port is already allocated
cello-watchdog |
cello-watchdog | ERROR: for d59e0daa7cb640dea97320e8b7581434_peer0_org1 Cannot start service peer0.org1.example.com: driver failed programming external connectivity on endpoint d59e0daa7cb640dea97320e8b7581434_peer0_org1 (9382aad8b4e2eddde345672106478fa18ba262e83d2f9e16f34154cbdad1fec6): Bind for 0.0.0.0:7051 failed: port is already allocated
cello-watchdog |
cello-watchdog | ERROR: for peer1.org1.example.com Cannot start service peer1.org1.example.com: driver failed programming external connectivity on endpoint d59e0daa7cb640dea97320e8b7581434_peer1_org1 (7c02919dc8dd55513c50da8c5877f213794c32226fd3aef61ad6d03a458a54cb): Bind for 0.0.0.0:7053 failed: port is already allocated
cello-watchdog |
cello-watchdog | ERROR: for peer0.org2.example.com Cannot start service peer0.org2.example.com: driver failed programming external connectivity on endpoint d59e0daa7cb640dea97320e8b7581434_peer0_org2 (35d7f72c6bce0a8ac246abc7f2c6482549ac28ad688b575b75e3643d093dbf66): Bind for 0.0.0.0:7054 failed: port is already allocated
cello-watchdog |
cello-watchdog | ERROR: for peer0.org1.example.com Cannot start service peer0.org1.example.com: driver failed programming external connectivity on endpoint d59e0daa7cb640dea97320e8b7581434_peer0_org1 (9382aad8b4e2eddde345672106478fa18ba262e83d2f9e16f34154cbdad1fec6): Bind for 0.0.0.0:7051 failed: port is already allocated
ok so it seems there were errors creating the fabric containers
there are port conflicts i believe.
@pb Check if there are fabric container running (docker ps -a ).. If there are old container running then kill those before creating new chain.
[ ](https://chat.hyperledger.org/channel/cello?msg=ttp7zCeYgFr83vr5n) @dinesh.rivankar
THanks i killed old containers and restart the master .But now unable to create new host .Log is host already full
Has joined the channel.
@pb what ur system configuration? did u set up the capacity option while creating host ?
@pb what's ur system configuration? did u set up the capacity option while creating host ?
[ ](https://chat.hyperledger.org/channel/cello?msg=ynMZkPctvoa5iMnor) @dinesh.rivankar
Here is my host configuration -
[ ](https://chat.hyperledger.org/channel/cello?msg=ynMZkPctvoa5iMnor) @dinesh.rivankar
Here is my host configuration -
Edit the Host Config
ID e5737cff456e4623a25fe87cda5ab79f
Name docker_host
Daemon URL tcp://192.168.20.247:2375
Capacity 1
Status active
Schedulable Autofill
Type docker
Log Level Debug
Log Type Local
Created Wed, 14 Mar 2018 05:32:23
Running Chains 1
@pb The error you are getting "Host xxx is already full", happens when you are creating a new chain when there is already an existing.
Reason for that is the defined Capacity in your case is set to 1.
you may have to delete the host and create a new one with an increased capacity. for example 5.
[ ](https://chat.hyperledger.org/channel/cello?msg=78gPDR8oAFnRtKzSW) @yopep
The add chain button is disabled .And the already added one's status is creating.
This is my chain information Id
530010d8f4524c99a2081375c2ef9492
Name
docker_host_0
Consensus
solo
Host
e5737cff456e4623a25fe87cda5ab79f
Create Time
2018-03-14T09:45:58.900000+00:00
Network Type
fabric-1.0
Size
4
Status
creating
Health
User Id
Apply Time
None
Release Time
None
Duration
0.0
Service URL
Hyperledger Explorer
@pb just to establish the setup, are you using at least 2 servers? 1 for cello master and 1 for cello worker?
[ ](https://chat.hyperledger.org/channel/cello?msg=YxZbuH3dbY9basSWf) @yopep
Actually both master and worker are running in same machine .I wish to manage my existing fabric sample network with cello .But how can i link my own sample network and chaincode with cello
it is my understanding that cello master and worker can't co-exist on the same server. can the other guys chime in? :)
i think that is the root cause of the errors that you are getting since master/worker are on the same machine.
[ ](https://chat.hyperledger.org/channel/cello?msg=MLzF5tgjk5Cp8imy8) @yopep
Now the status of chain is running
[ ](https://chat.hyperledger.org/channel/cello?msg=MLzF5tgjk5Cp8imy8) @yopep
Now the status of chain is running.
[ ](https://chat.hyperledger.org/channel/cello?msg=MLzF5tgjk5Cp8imy8) @yopep we can have master and worker node on same machine.
Has joined the channel.
Once a network is up and running with the Ansible agent, can it be added to the Cello dashboard?
I mean using docker and flanneld. Or does it need to be a kubernetes cluster?
Has joined the channel.
@AlexBA 6 the integration is not done. Need help to do that job.
we currently have a Jira task to do that.
need some help to actually integrate the dashboard with the network ansible agent sets up.
@all Dear all, we will have project meeting at 8:30PM--9:00PM (UTC+8) this Friday evening , welcome to join, and please help update the agenda at: https://wiki.hyperledger.org/projects/cello/meetings/20180316. Thanks!
hightall
hightall
hightall
tongli
Hyperledger Cello project
wiki.hyperledger.org/projects/cello
https://github.com/hyperledger/cello
Hi, is there a rest api for the user dashboard?
@ctan you mean api in user dashboard or in operator service? what's the purpose of the rest api?
in the user dashboard. we're using cello but with a different dashboard. so if there's an api for the uploading of smart contract we we're thinking of just calling that
Has joined the channel.
Has joined the channel.
@ctan oh, you mean you want to use cello user dashboard background api service integrate with the ui developed by yourself, currently in cello user dashboard is Front and back end separation design, but the session need be authorized.
Has joined the channel.
Thanks ..
guys, any chance that we can upgrade fabric version used in cello from 1.0.5 to 1.1-alpha?
guys, any chance that we can upgrade fabric version used in cello from 1.0.5 to 1.1.0?
i think fabric-1.1.0 is already released just today. right?
How to create a cluster of 2 chain running on different network?
Hi cello group
this cello project is the fabric project..
part
@tongli Hi
I am luke
@yopep fabric 1.1.0 is just out, we need some time to test if it's stable enough
it might be included in cello v0.9.0 release
Has joined the channel.
thanks @baohua. i went ahead and created the corresponding ticket. https://jira.hyperledger.org/projects/CE/issues/CE-305
Has joined the channel.
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/19341/
Has joined the channel.
@yopep thanks for the contribution at https://gerrit.hyperledger.org/r/#/c/19341/!
We now maintain all cello related images under the same cello naming prefix, which is to be unique and flexible to adjust based on other official images.
Hence, to update the cell-mongo image, i guess a better way is:
1) Remove installing mongo db in the base image Dockerfile;
2) Use mongo:3.4-jessie as the baseimage in the cello-mongo image building, and may remove the entrypoint.sh code.
This will be simpler to change, and won't need to update other code.
Thanks!
@baohua thanks. i'll adjust the codes accordingly.
In a kubernetes environment, should cello-operator-dashboard be part of the cluster? Or should it be deployed outside of the kubernetes cluster?
@yopep In my opinion, cello-operator-dashboard should be outside of the kubernetes clusters, we should only use kubernetes as a Cello host to maintain Fabric cluster.
thanks @Luke_Chen
@baohua @hightall please see this patch for making sure that the ca can use different image tags. https://gerrit.hyperledger.org/r/#/c/19415/
@all we need to prepare some video tutorial (each within 5 minutes), anyone volunteer is encouraged: 0. What is cello. 1. Quick start cello service. 2. How to use operator dashboard.
https://jira.hyperledger.org/browse/CE-313
Has joined the channel.
Has joined the channel.
Hi all ,When I am running the command "make start" , getting an error
error1.png
but there is no container running at 8081
@Unni_1994 based on the logs it seems that port 8081 is already used by a running process. You may issue the following command to verify
```
netstat -avn | grep 8081
```
on a centos server to get the process id
```
netstat -avnp | grep 8081
```
then
```
ps -ef | grep
@baohua @tongli Hi Guys, I'm trying to comprehend the design of Cello with respect to Kubernetes. Kindly comment on my observations/questions.
1. Currently Cello supports Docker (single node) and Docker Swarm. This is done via calls to docker-compose in src/agent/docker scripts. Can't comment on vSphere as I'm not familiar with it.
2. Support for Kubernetes is via CLI at src/agent/k8s scripts. It is not wired with the Operator Dashboard just yet.
3. There are ansible scripts that deploys fabric to different environments (Docker and Kubernetes)
- Is the intention of these scripts is to replace current agent implementation? Operator Dashboard will then call these ansible scripts to provision a fabric cluster instead?
4. There is an empty src/agent/k8s-operator folder
- Based on CE-68 the intention is to use an Kubernetes Operator (http://bit.ly/2FTDiDv) to provition a fabric network.
5. What is the planned design for supporting Kubernetes?
- Items #2, #3 and #4 seems to be in conflict in my opinion
thanks @yopep
Has joined the channel.
Has joined the channel.
Hi everyone!!! What functions does cello provide? By baas (as a service) does it means that we will be able to define network identities by it. Identity meaning no. of organizations , no. of peers and channel between them? Can someone clear me on this...
Hi everyone can i use cello to deploy bna archive via Hyperledger composer ?
@walnemer1976 integration with composer is in the works.
i have setup docker cluster using docker swarm , three worker nodes, i have successfully connect to manager node from cello operator dashboard , however when i add new chain
i receive successful message but noting created in the worker nodes
and dashboard show zero chains
it may be that there were errors during the creation of the chain. you can verify this by checking the cello-operator-dashboard logs
```
docker logs -f cello-operator-dashboard
```
[ ](https://chat.hyperledger.org/channel/cello?msg=pEcv7wXYwBcZxW9Bu) @divudivyansh It provisions a fabric network using docker, docker swarm, vsphere and soon k8s. By provision I mean it creates/teardown a fabric network with 2 orgs with 2 peers each. Haven't looked at the ansible scripts yet if it is capable of being flexible in terms of orgs and peers. There is also a user dashboard wherein you can deploy chaincodes and test function calls on the deployed chaincode but this is still a work in progress. Integration with Hyperledger Composer playground is also a work in progress.
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=8ACPnFYoecdrCLpjq) @yopep
[2018-03-20 13:44:41,573] ERROR [agent.docker.docker_swarm] [docker_swarm.py:273 get_swarm_node_ip()] - Exception happens when detect container host!
[2018-03-20 13:44:41,573] ERROR [agent.docker.docker_swarm] [docker_swarm.py:274 get_swarm_node_ip()] - 404 Client Error: Not Found ("No such container: 8534fda4ed764002bd4efb1711279c04_peer0.org1.example.com")
[2018-03-20 13:44:41,574] DEBUG [modules.cluster] [cluster.py:690 _get_service_ip()] - swarm host, ip =
[2018-03-20 13:44:41,574] ERROR [modules.cluster] [cluster.py:128 gen_service_urls()] - Error to find peer host url, cleanup
did you execute "make setup-worker" on the worker nodes? it seems to me that the docker images were not downloaded beforehand.
[ ](https://chat.hyperledger.org/channel/cello?msg=uqmXGAkSZp59CWXAB) @yopep yes i did
not sure then what is causing it.
[Would please send me the steps to configure cello work nodes ](https://chat.hyperledger.org/channel/cello?msg=xQQtz4oNRmw328hCq) @yopep
im getting this message in the logCompose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node
Hi every one, i have setup docker cluster using docker swarm , three worker nodes, i have successfully connect to manager node from cello operator dashboard , however when i add new chain
i receive successful message but noting created in the worker nodes
and dashboard show zero chains
@walnemer1976 did you configured docker daemon to listen on port 2375 as described here (http://hyperledger-cello.readthedocs.io/en/latest/installation_worker_docker/) ?
docker daemon on the worker node
yes i did
i mentioned i can connect to worker nodes from dashboard by providing IP:2375
But adding worker node to the dashboard doesn't guarantee a connection. what do you see if you do `docker -H Worker_Node_IP:2375 version` from the master node ?
Client:
Version: 17.12.1-ce
API version: 1.35
Go version: go1.9.4
Git commit: 7390fc6
Built: Tue Feb 27 22:17:40 2018
OS/Arch: linux/amd64
Server:
Engine:
Version: 17.12.1-ce
API version: 1.35 (minimum version 1.12)
Go version: go1.9.4
Git commit: 7390fc6
Built: Tue Feb 27 22:16:13 2018
OS/Arch: linux/amd64
Experimental: false
root@waleed-Worker02:~#
i have configured the docker swarm cluster and tested it, do i need to run anything in the worker node related to cello?
in the worker node you just need `make setup-worker`, nothing else
also note that the current version of cello does the setup only in one worker node, even if you have connected three nodes. If you need to use actual multi-node you have to use the ansible agent (Also note, the ansible agent is not connected to the dashboard, it is work in progress)
dose it matter if i run make setup-worker before i configure docker swarm ? or i need to configure swarm first then run make setup-worker?
I don't think the order matters (anyone correct me if I'm wrong), but you can always try to redeploy. To redeploy you can do `make stop`, remove /opt/cello and do make setup-master / make setup-worker on master and worker nodes respectively
Start compose project with name=a4e9d5e9440a47009a67051d3f773ef3
[2018-03-20 21:47:17,864] DEBUG [agent.docker.cluster] [cluster.py:53 create()] - Start compose project with name=a4e9d5e9440a47009a67051d3f773ef3
[2018-03-20 21:47:17,865] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:415 compose_up()] - Compose start: name=a4e9d5e9440a47009a67051d3f773ef3, host=SwarmServer01, mapped_port={'dashboard': 8150, 'peer1_org1_grpc': 7250, 'peer1_org2_event': 7750, 'ca_org2_ecap': 7950, 'peer0_org2_event': 7550, 'peer0_org2_grpc': 7450, 'peer0_org1_grpc': 7050, 'orderer': 8050, 'peer0_org1_event': 7150, 'peer1_org1_event': 7350, 'ca_org1_ecap': 7850, 'peer1_org2_grpc': 7650},config={'size': 4, 'consensus_plugin': 'solo', 'network_type': 'fabric-1.0'}
[2018-03-20 21:47:17,865] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:391 _compose_set_env()] - envs {'DOCKER_HOST': 'tcp://192.168.99.101:2375', 'COMPOSE_PROJECT_PATH': '/opt/cello/fabric-1.0', 'COMPOSE_FILE': 'fabric-solo-4.yaml', 'CLUSTER_NETWORK': 'cello_net_solo', 'CLUSTER_LOG_LEVEL': 'DEBUG', 'HLF_VERSION': '1.0.5', 'VM_ENDPOINT': 'tcp://192.168.99.101:2375', 'PEER_NETWORKID': 'a4e9d5e9440a47009a67051d3f773ef3', 'NETWORK_TYPES': 'fabric-1.0', 'VM_DOCKER_HOSTCONFIG_NETWORKMODE': 'cello_net_solo', 'COMPOSE_PROJECT_NAME': 'a4e9d5e9440a47009a67051d3f773ef3'}
[2018-03-20 21:47:17,865] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:427 compose_up()] - template path ./agent/docker/_compose_files/fabric-1.0
Creating a4e9d5e9440a47009a67051d3f773ef3_orderer ... done
Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.
Creating a4e9d5e9440a47009a67051d3f773ef3_peer1_org2 ... done
Creating a4e9d5e9440a47009a67051d3f773ef3_explorer ... done
Creating a4e9d5e9440a47009a67051d3f773ef3_ca_org2 ...
Creating a4e9d5e9440a47009a67051d3f773ef3_mysql ...
Creating a4e9d5e9440a47009a67051d3f773ef3_orderer ...
Creating a4e9d5e9440a47009a67051d3f773ef3_peer1_org2 ...
Creating a4e9d5e9440a47009a67051d3f773ef3_peer1_org1 ...
Creating a4e9d5e9440a47009a67051d3f773ef3_peer0_org2 ...
Creating a4e9d5e9440a47009a67051d3f773ef3_peer0_org1 ...
Creating a4e9d5e9440a47009a67051d3f773ef3_explorer ...
Creating a4e9d5e9440a47009a67051d3f773ef3_cli ...
[2018-03-20 21:47:30,515] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:433 compose_up()] - containers=[
[2018-03-20 21:47:30,574] DEBUG [modules.cluster] [cluster.py:690 _get_service_ip()] - swarm host, ip =
[2018-03-20 21:47:30,574] ERROR [modules.cluster] [cluster.py:128 gen_service_urls()] - Error to find peer host url, cleanup
[2018-03-20 21:47:30,575] DEBUG [modules.cluster] [cluster.py:329 delete()] - Delete cluster: id=a4e9d5e9440a47009a67051d3f773ef3, forced=True
[2018-03-20 21:47:30,578] INFO [modules.cluster] [cluster.py:820 db_update_one()] - filter {'state': 'active', 'id': 'a4e9d5e9440a47009a67051d3f773ef3'} operations {'status': 'deleting'}
[2018-03-20 21:47:30,578] INFO [modules.cluster] [cluster.py:823 db_update_one()] - k=set__status, v=deleting
[2018-03-20 21:47:30,585] DEBUG [modules.host] [host.py:417 get_active_host_by_id()] - check host with id = 8ca9715ee6a3469aa935e13a3d748216
[2018-03-20 21:47:30,589] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:602 compose_down()] - Compose remove a4e9d5e9440a47009a67051d3f773ef3 with worker_api=tcp://192.168.99.101:2375, config={'size': 4, 'consensus_plugin': 'solo', 'network_type': 'fabric-1.0', 'env': {'PEER0_ORG1_EVENT_PORT': '7150', 'CA_ORG1_ECAP_PORT': '7850', 'ORDERER_PORT': '8050', 'DASHBOARD_PORT': '8150', 'PEER1_ORG1_GRPC_PORT': '7250', 'PEER0_ORG2_GRPC_PORT': '7450', 'PEER0_ORG2_EVENT_PORT': '7550', 'PEER1_ORG2_EVENT_PORT': '7750', 'CA_ORG2_ECAP_PORT': '7950', 'PEER0_ORG1_GRPC_PORT': '7050', 'PEER1_ORG1_EVENT_PORT': '7350', 'PEER1_ORG2_GRPC_PORT': '7650'}}
[2018-03-20 21:47:30,589] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:391 _compose_set_env()] - envs {'DOCKER_HOST': 'tcp://192.168.99.101:2375', 'CLUSTER_NETWORK': 'cello_net_solo', 'DASHBOARD_PORT': '8150', 'PEER1_ORG1_GRPC_PORT': '7250', 'PEER0_ORG2_EVENT_PORT': '7550', 'CA_ORG2_ECAP_PORT': '7950', 'HLF_VERSION': '1.0.5', 'PEER_NETWORKID': 'a4e9d5e9440a47009a67051d3f773ef3', 'COMPOSE_PROJECT_PATH': '/opt/cello/fabric-1.0', 'COMPOSE_FILE': 'fabric-solo-4.yaml', 'PEER0_ORG1_EVENT_PORT': '7150', 'CA_ORG1_ECAP_PORT': '7850', 'VM_ENDPOINT': 'tcp://192.168.99.101:2375', 'ORDERER_PORT': '8050', 'NETWORK_TYPES': 'fabric-1.0', 'PEER0_ORG2_GRPC_PORT': '7450', 'COMPOSE_PROJECT_NAME': 'a4e9d5e9440a47009a67051d3f773ef3', 'PEER1_ORG2_EVENT_PORT': '7750', 'PEER0_ORG1_GRPC_PORT': '7050', 'CLUSTER_LOG_LEVEL': 'DEBUG', 'VM_DOCKER_HOSTCONFIG_NETWORKMODE': 'cello_net_solo', 'PEER1_ORG1_EVENT_PORT': '7350', 'PEER1_ORG2_GRPC_PORT': '7650'}
[2018-03-20 21:47:30,590] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:608 compose_down()] - environ({'DOCKER_HOST': 'tcp://192.168.99.101:2375', 'MONGODB_PORT': '27017', 'TLSCAA_PORT': '7059', 'CLUSTER_NETWORK': 'cello_net_solo', 'MONGO_HOST': 'mongo', 'DASHBOARD_PORT': '8150', 'STATIC_FOLDER': 'themes/basic/static', 'TLSCAP_PORT': '7058', 'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'HLF_VERSION': '1.0.5', 'PEER_NETWORKID': 'a4e9d5e9440a47009a67051d3f773ef3', 'COMPOSE_PROJECT_NAME': 'a4e9d5e9440a47009a67051d3f773ef3', 'ENABLE_EMAIL_ACTIVE': 'False', 'PEER0_ORG1_EVENT_PORT': '7150', 'TCAP_PORT': '7056', 'VM_ENDPOINT': 'tcp://192.168.99.101:2375', 'CLI_PORT': '7052', 'ORDERER_PORT': '8050', 'MONGO_URL': 'mongodb://cello-mongo:27017', 'PEER0_ORG2_GRPC_PORT': '7450', 'PEER1_ORG2_EVENT_PORT': '7750', 'PEER0_ORG1_GRPC_PORT': '7050', 'CLUSTER_LOG_LEVEL': 'DEBUG', 'GRPC_PORT': '7051', 'PEER1_ORG1_EVENT_PORT': '7350', 'DEBUG': 'True', 'EVENT_PORT': '7053', 'ECAP_PORT': '7054', 'ECAA_PORT': '7055', 'MONGO_DB': 'dev', 'REST_PORT': '7050', 'PEER1_ORG1_GRPC_PORT': '7250', 'PEER0_ORG2_EVENT_PORT': '7550', 'WERKZEUG_RUN_MAIN': 'true', 'CA_ORG2_ECAP_PORT': '7950', 'HOSTNAME': 'cello-operator-dashboard', 'COMPOSE_PROJECT_PATH': '/opt/cello/fabric-1.0', 'COMPOSE_FILE': 'fabric-solo-4.yaml', 'CA_ORG1_ECAP_PORT': '7850', 'TEMPLATE_FOLDER': 'themes/basic/templates', 'NETWORK_TYPES': 'fabric-1.0', 'LOG_LEVEL': 'DEBUG', 'TCAA_PORT': '7057', 'WERKZEUG_SERVER_FD': '6', 'PEER1_ORG2_GRPC_PORT': '7650', 'VM_DOCKER_HOSTCONFIG_NETWORKMODE': 'cello_net_solo', 'HOME': '/root'})
Stopping a4e9d5e9440a47009a67051d3f773ef3_explorer ... done
Stopping a4e9d5e9440a47009a67051d3f773ef3_cli ... done
Stopping a4e9d5e9440a47009a67051d3f773ef3_peer0_org1 ... done
Stopping a4e9d5e9440a47009a67051d3f773ef3_peer0_org2 ... done
Stopping a4e9d5e9440a47009a67051d3f773ef3_peer1_org2 ... done
Stopping a4e9d5e9440a47009a67051d3f773ef3_peer1_org1 ... done
Stopping a4e9d5e9440a47009a67051d3f773ef3_orderer ... done
Stopping a4e9d5e9440a47009a67051d3f773ef3_ca_org2 ... done
Stopping a4e9d5e9440a47009a67051d3f773ef3_mysql ... done
Stopping a4e9d5e9440a47009a67051d3f773ef3_ca_org1 ... done
Removing a4e9d5e9440a47009a67051d3f773ef3_explorer ... done
Removing a4e9d5e9440a47009a67051d3f773ef3_cli ... done
Removing a4e9d5e9440a47009a67051d3f773ef3_peer0_org1 ... done
Removing a4e9d5e9440a47009a67051d3f773ef3_peer0_org2 ... done
Removing a4e9d5e9440a47009a67051d3f773ef3_peer1_org2 ... done
Removing a4e9d5e9440a47009a67051d3f773ef3_peer1_org1 ... done
Removing a4e9d5e9440a47009a67051d3f773ef3_orderer ... done
Removing a4e9d5e9440a47009a67051d3f773ef3_ca_org2 ... done
Removing a4e9d5e9440a47009a67051d3f773ef3_mysql ... done
Removing a4e9d5e9440a47009a67051d3f773ef3_ca_org1 ... done
[2018-03-20 21:47:39,196] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:75 _clean_project_containers()] - Clean project containers, worker_api=tcp://192.168.99.101:2375, prefix=a4e9d5e9440a47009a67051d3f773ef3
[2018-03-20 21:47:39,204] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:97 _clean_project_networks()] - Clean project networks, worker_api=tcp://192.168.99.101:2375, prefix=a4e9d5e9440a47009a67051d3f773ef3
[2018-03-20 21:47:39,377] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:103 _clean_project_networks()] - Remove network id 7c1cfbd9b383abaf3ebeb348c2fb48549faa5596d99a565e0716d0e5ab062e0e
[2018-03-20 21:47:39,378] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:53 _clean_chaincode_images()] - clean chaincode images with prefix=a4e9d5e9440a47009a67051d3f773ef3
[2018-03-20 21:47:39,405] DEBUG [agent.docker.docker_swarm] [docker_swarm.py:58 _clean_chaincode_images()] - chaincode image id to removes=
[2018-03-20 21:47:39,411] INFO [modules.cluster] [cluster.py:820 db_update_one()] - filter {'state': 'active', 'id': 'a4e9d5e9440a47009a67051d3f773ef3'} operations {'api_url': '', 'status': 'running', 'service_url': {}, 'user_id': ''}
Has joined the channel.
I managed to reproduce your error, There is some issue when adding a host with `host type = swarm`. Try to add the host with `host type = docker` this should work at least
Thank you @yopep
@yopep luke is also working on the k8s agent, let's use this jira item to track the discussions: https://jira.hyperledger.org/browse/CE-283
thanks @baohua will take a look.
Hi everyone, any idea how to solve having cello working with docker Swarm cluster setup?
[ ](https://chat.hyperledger.org/channel/cello?msg=WKpbC7FZskNyFcN7N) @baohua Posted a comment on https://jira.hyperledger.org/browse/CE-286. Reposting it here so others who are interested are free to chip in. :)
```
Hi Luke Chan and allen yang,
I'd like to know the design on how do you go about implementing the code to deploy Fabric to a Kubernetes cluster. As I understand it there are three solutions existing in the Cello source codes.
Solution 1:
Use src/agent/k8s where deployment is via shell scripts
Solution 2:
Use src/agent/ansible where Ansible is used
Solution 3:
Use src/agent/k8s-operator where Kubernetes Operator will be used (No codes yet)
I'm asking because I would like to help in implementing it and so that our efforts can be harmonized.
```
@Luke_Chen
[ ](https://chat.hyperledger.org/channel/cello?msg=jQbTT2a47JdoYepAm) @walnemer1976 Can you try a single docker worker node? Based on the message from Amir it seems deploying to docker swarm has a bug for now.
@yopep well, we must assume that user already have a functional and configured kubernetes cluster before using kubernetes agent.
basically, our work is meant to intergrate `https://github.com/kubernetes-client/python` into Cello, and we created an agent base on the api from the `kubernetes-client python`
the api design is compatible with the module layer, which means module layer can call functions of this agent
thanks @Luke_Chen
thanks @Luke_Chen i think that clears it up. :)
Can we use cello on windows platform as of now? And how soon will cello integrate with composer
we don't need extra scripts to launch Fabric deployment on kubernetes cluster
@Luke_Chen I think your idea can also be implemented for docker swarm - via rest api calls. Currently it is based on some docker-compose.yml files which are somewhat static. What do you think?
@yopep also, we will release design document of kubernetes agent soon, please keep focus。
@yopep we use the static deployment file too
@Luke_Chen yes, some sharing design docs (e.g., google doc) is a good way to collaborate!
@yopep We have considerated let user config their Fabric network, but the the problem is, we need some tools to generate `crypto-config` and `channel-artifacts` which are not possible to be intergrated in Cello so far
@baohua Yes, will release as soon as possible
[ ](https://chat.hyperledger.org/channel/cello?msg=G8spXkq6dXjM2by5m) @yopep
[ ](https://chat.hyperledger.org/channel/cello?msg=G8spXkq6dXjM2by5m) @divudivyansh Haven't tried but in my opinion you can't because some of the tools used like make and python. And the scripts for configuring master and workers are all in bash scripts.
@divudivyansh perhaps what you can do is to install virtualbox with a running linux vm. then you configure cello in there.
@divudivyansh perhaps what you can do is to install virtualbox or vmware with a running linux vm. then you configure cello in there.
Has joined the channel.
@baohua @hightall made the necessary adjustments. kindly take a look
```
https://gerrit.hyperledger.org/r/#/c/19341/
```
@baohua @hightall made the necessary adjustments. kindly take a look
https://gerrit.hyperledger.org/r/#/c/19341/
https://gerrit.hyperledger.org/r/19525
https://gerrit.hyperledger.org/r/#/c/19515/
thanks
add my comments, thanks!
I am having DNS problems when running Fabric via Cello/Ansible. Everything seems to work until I try to join channel. I have containers on 2 VMs. The zookeepers (and others) cannot resolve addresses. "2018-03-20 18:52:36,844 [myid:] - WARN [main:QuorumPeer$QuorumServer@155] - Failed to resolve address: zookeeper3
java.net.UnknownHostException: zookeeper3: Temporary failure in name resolution". I can ping all the containers by IP address, so it appears that the network is in good shape.
Also I get this messages in fabriccli: "2018-03-20 18:53:25.599 UTC [grpc] Printf -> DEBU 003 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: Error while dialing dial tcp: lookup orderer1-org3 on 208.43.80.246:53: server misbehaving"; Reconnecting to {orderer1-org3:7050
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/19541/
@tonym1 looks like that you dns and/or registrator maybe down. you can check to see if these containers are still running.
@tongli skydns.fabric001 and registrator.fabric001 are up and running on one VM and registrator.fabric002 is up and running on the other. I don't know how to test them. I tried doing a nslookup like this ... "ubuntu@tony-work1:~$ docker exec -it skydns.fabric001 nslookup zookeeper1
Server: 208.43.80.246
Address: 208.43.80.246#53
** server can't find zookeeper1: SERVFAIL
"
@tonym1 that is correct. dns only runs on one machine. registrator are running on all nodes.
hmmm. your node IP looks strange.
your node IP should be in 172.x.x.x range.
looks like that your overlay network was not setup right.
@tongli The node_ip above (208.43.80.246) is the public ip of the Softlayer VM that is fabric001. There is a private IP 10.x.x.x but I could never get that to work - I assumed because 10.x is non-routeable. docker0 and flannel0 both have 172.16.x.x addresses.
I added this issue (https://jira.hyperledger.org/projects/CE/issues/CE-319?filter=allopenissues) following this error (https://chat.hyperledger.org/channel/cello?msg=cqfD8p3XH67gPLJdN)
Has joined the channel.
@baohua @harsha made some adjustments to https://gerrit.hyperledger.org/r/#/c/19341/ based on your comments.
thanks!
@harsha would like to have ur confirmation first :)
@harsha am not really familiar with ppc64le and s390x arch. so i just restored those files and for any improvements for those arch let's tackle it on another ticket. :)
Has joined the channel.
Has joined the channel.
Clipboard - March 22, 2018 1:46 PM
Clipboard - March 22, 2018 1:46 PM
Is this the success of the cello project?
I am a novice
Has joined the channel.
Hello, I am Alan Ortiz, I am new in this project too, I decided to start working on the video about "what is cello", I want to ask you for advice about what points should I highlight and if it would be good to use powtoon for that, thanks in advanced.
There is no documentation for the analysis of the cello project code.
But I can't understand the code of the cello project. How does the cello manage fabric?
我想学习cello和fabric
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=rLraDKJH5DZjCMEyG) @baohua Just checking it now :)
@danfeng yes, for the first step, we have only the user documentation and design documentation. no code analysis is available yet, but would welcome a lot if you have interested to help contribute that :)
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/19541/
https://gerrit.hyperledger.org/r/19607
@hightall should wait till CI passed before posting here
https://gerrit.hyperledger.org/r/#/c/19607/
Has joined the channel.
Has joined the channel.
Has joined the channel.
Has joined the channel.
Has joined the channel.
@baohua can you paste the zoom link here? thanks.
Never mind, found it.
is the project meeting right now? or I am late
@ashishapy it was done, and meeting info is at https://wiki.hyperledger.org/projects/cello/meetings/20180323
https://gerrit.hyperledger.org/r/#/c/19671/
@baohua I joined zoom link https://zoom.us/j/248676399 (received through email from Neil Yue) & no one was there.
I feel need of Design Doc so that others can also understand & contribute. I don't see in minutes of meeting
I was trying to fix this issue (https://jira.hyperledger.org/browse/CE-250). I cloned the git, make the changes I though and uploaded it to my server where I wanted to test, but i still can't see my changes (even simple log outputs). I'm I missing something ?
@ashishapy have u subscribed to the maillist hyperledger-cello@lists.hyperledger.org? the minutes is there.
@Amir pls follow the documentation at http://cello.readthedocs.io/en/latest/
[ ](https://chat.hyperledger.org/channel/cello?msg=uqkHQHft8j47z9XDB) @baohua yes, I have. I see minutes. Thanks
Has joined the channel.
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/19525/
https://gerrit.hyperledger.org/r/#/c/19671/
Thanks @baohua
How do you use fabric to develop block chain applications?Any good articles or references, thank you.
Clipboard - 2018年3月26日下午4点40分
不是在这个目录下面运行吗?
@baohua Just to re-confirm... Cell Scrum Meeting is 8:30 AM EST on Friday, right?
Which is 6:00PM IST?
I missed the last one, don't want to miss it again.
Has joined the channel.
Does anyone else have an issue running "make start" where the command can't be found?
(for the master node)
Has joined the channel.
Ah nevermind, needed to install gettext
Has joined the channel.
Hi, is it true that Cello and Explorer both provide dashboards? If so, how are they different?
[ ](https://chat.hyperledger.org/channel/cello?msg=49FQNDEoiPSz2DoRX) @randallj Hey, yes if you deploy Cello using the Makefile you will get Explorer dashboard. Cello and Explorer are separate projects with different functionalities. Cello is focused on managing the underlining infrastructure for blockchain network. Explorer is used to visualize the contents of blockchain network
Good to know. Thank you, @Amir !
Can a user model their business blockchain network through Cello or can they only do that in Composer?
I don't think cello will be used to model business blockchain network. But I think integration with Composer is under development
Where can i read up or follow on cello and composer integration.
@sillysachin I did some work, but not fully functional please refer this link https://jira.hyperledger.org/browse/CE-245
I'm going to finish the rest of work once the v0.8.0 is released
https://gerrit.hyperledger.org/r/#/c/19799/
+2
Has joined the channel.
Has joined the channel.
Thanks again, @Amir !
Has joined the channel.
Has joined the channel.
Hi @baohua , does cello support docker swarm based deployment yet? Since I need a multi-host deployment, swarm would be really easy to operate. If not is there a road-map I was not able to find it on JIRA or anywhere else. The script inside worker_node for swarm seems to be a work in progress.
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/19857/
+1!
@AshishMishra 1 already!
have u met any problem to add swarm as new host?
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=rL7CiBf6qbwZ3oZCA) @baohua there is this issue on swarm host https://jira.hyperledger.org/projects/CE/issues/CE-319?filter=allopenissues
Has joined the channel.
Hi all, I have cello set up, and I dont know if it is correct, but My users cant create or join a chain is this normal?
@all
Dear all, we will have project meeting at 8:30PM--9:00PM (UTC+8) this Friday evening , welcome to join, and please help update the agenda at: https://wiki.hyperledger.org/projects/cello/meetings/20180330. Thanks!
@all Dear all
Thanks to all the contributors, Hyperledger Cello v0.8.0 is released after the v0.8.0-beta release three weeks ago.
This release mainly fixed bugs and enhanced documentation.
More details can be found at https://github.com/hyperledger/cello/releases/tag/v0.8.0.
https://wiki.hyperledger.org/projects/cello.
Thanks!
Has joined the channel.
Has joined the channel.
@baohua For upcoming release will there be a planned release cycle, or release would happen based on feature enhancement(s) ?
yes, the 0.x.0 release is around quarterly, while the 0.x.y release is base on feature/bug.
btw, i enable the released flag, you may help revert it back now: https://github.com/hyperledger/cello/releases/tag/v0.8.0
Sure Thing.
@baohua , yes I was not able to add swarm nodes as worker nodes. I feel there is lack of documentation for swarm type nodes. I could only add docker type nodes.
https://gerrit.hyperledger.org/r/19909 -- Request for update.
https://gerrit.hyperledger.org/r/19909 -- Request for review
@harsha posted a comment.. You have to make IS_RELEASE=false
[ ](https://chat.hyperledger.org/channel/cello?msg=HELEXk7y8KnQ5n6tZ) @rameshthoomu done
Has joined the channel.
Has joined the channel.
Has joined the channel.
fabric-kubernetes.png
@Nghiadt I see good discussion here https://jira.hyperledger.org/browse/FAB-3339
K8s managing Chaincode container initiated by peer.
Someone has to confirm that it is resolved. @greg.haskins @baohua @tongli @chawlanikhil24
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=QtnkeFoPw4u9SeGoT) Need one more +2
@harsha +2 merged
Anyone has faced this issue before ?
git review
Problem running 'git remote update gerrit'
Fetching gerrit
ssh: connect to host gerrit.hyperledger.org port 29418: Connection timed out
fatal: Could not read from remote repository.
Request for review : https://gerrit.hyperledger.org/r/20063
[ ](https://chat.hyperledger.org/channel/cello?msg=JLMj2SRop4HwWjKdk) Issue resolved by using https connection
@dinesh.rivankar ci failed, pls check the log link from the ci's comments: https://gerrit.hyperledger.org/r/#/c/20063/
that report is not accurate, need to check it's detailed log
@dinesh.rivankar
can u see the comments by the hyperledger jobbuilder? https://gerrit.hyperledger.org/r/#/c/20063/
there's links to the failure, and it's jenkins
[ ](https://chat.hyperledger.org/channel/cello?msg=M6Ha42W5JvE8dNf6X) @baohua yes on log it says Error: Remove JIRA URLs from commit message
09:46:17 operator-dashboard-test | test_search_user (test_user_management.UserManagementTestCase) ... [2018-04-02 09:46:17,263] ERROR [modules.user.user] [user.py:99 get_by_username()] - get user exc User matching query does not exist.
@Nghiadt if you use ansible agent which can be found here https://github.com/hyperledger/cello/tree/master/src/agent/ansible, the agent sets up k8s and deploy fabric onto it.
@Nghiadt that work has been done long time ago. you can find doc for that here. https://github.com/hyperledger/cello/blob/master/docs/worker_ansible_howto.md
@baohua @hightall create this issue https://jira.hyperledger.org/browse/CE-332 and added this patch. https://jira.hyperledger.org/browse/CE-332, please review. this is to add common connection profile generation and composer playground.
[ ](https://chat.hyperledger.org/channel/cello?msg=43eHq9xip6E8saCbG) @tongli thank you, Tongli. I'll check it out.
[ ](https://chat.hyperledger.org/channel/cello?msg=d2zXdNvqutKC2hors) @tongli By the way, do you have any comment about this doc? https://github.com/hyperledger/cello/tree/master/src/agent/k8s . I set up kube-dns NodePod and ngixn which work each other. But problem not solved.
Has joined the channel.
hey all, I was wondering if someone could take a look at the error at the bottom of this gist: https://gist.github.com/collincusce/8efc21f216dad2114c4afd7e734e7209
I can't figure out why I get it... I think there's something wrong with the network config? But not sure what or why.
also having issues with 1.1.0 but I can get bc1st to work on 1.0.0 (diagnosed that last week, but lost the details on that) ... had to include the bin at the bottom to get bc1st example to work. When I modify the example as follows, though, it bombs
no idea why
also great project @baohua , it's going to make things a lot easier going forward in the future
Has joined the channel.
Great you love it!
i checked the gist, not find too many clues now. If you wanna try 1.1 setup, let me find some link for u today.
i checked the gist, not find too many clues now. If you wanna try 1.1 setup, maybe can have a look at this: https://github.com/yeasy/docker-compose-files/tree/master/hyperledger_fabric/v1.1.0
https://github.com/yeasy/docker-compose-files/tree/master/hyperledger_fabric/v1.1.0
@hightall has prepared some quick start tutorial video, welcome anyone help comment: https://www.youtube.com/watch?v=LVGW1fuZ98c
Has joined the channel.
@dinesh.rivankar can u help elaborate the process of the crypto config generation?
i guess this is not that easy to just put the binary into operator dashboard.
Would you like to help create some google doc as the design spec? including the idea, steps, challenges and solutions.
Sure. we wanted to include the crypto-gen tool in the cello-operator-dashboard and during chain creation API call wanted to invoke the tool.
based on the crypto-config.yaml for network we wanted to create crypto material
several issues come to me to this topic: 1) how to define the organization structure? 2) how to distribute the crypto materials in a cluster env?
Hence some design spec will be a good start. feel free to share the link here after initializing the first version.
will welcome more people to help comment
was thinking to define same way as fabric-solo-4.yaml is defined... sure
give me 1 day time and will submit the design doc
thanks!
[ ](https://chat.hyperledger.org/channel/cello?msg=fLdseMzFZj5jtoGLC) @baohua Great video, really helps for starters. But I think he missed the `make setup-master` command before running `make start`
@Amir Good point, would you like to help add the comments to https://jira.hyperledger.org/projects/CE/issues/CE-315? @hightall
Sure
Dynamic Generation of Crypto Material : https://docs.google.com/document/d/1jfA2FoU6C5XRCWT6hasxOFr4FGaxdf-nqIgVnyXA_-0/edit?usp=sharing
cool, added it into the project wiki: https://wiki.hyperledger.org/projects/cello
@baohua @hightall please review this patch to add connection profile and composer playground.
https://gerrit.hyperledger.org/r/#/c/18683/
if this patch set merges, and if you use ansible agent, you should have composer playground sets up after everything finishes.
@tongli +2
@hights
@hightall thanks bro.
:relaxed:
hmmmm so @baohua I dont know how to use that information you provided on the yeasy github. I was trying to make my own version of bc1st.yml but ... I can't see what's wrong at all. It looks right? But I keep getting this "ordererOrganization" error that is at the bottom of this: https://gist.github.com/collincusce/8efc21f216dad2114c4afd7e734e7209
which file in https://github.com/yeasy/docker-compose-files/tree/master/hyperledger_fabric/v1.1.0 did you want me to look at?
@collincusce if you are using bc1st.yml, it suggests to me that you are using ansible agent for setting up your network.
yesss
@collincusce can you tell me what error did you see and when that happened?
it's at the bottom of the gist
there's three files
it happens when I run setupfabric
I can provision, I can initicluster, but I can't get that network setup to work ... says there's files that need to be copied, but they're already there?
or somesuch?
@collincusce give me few minutes, let me finish the email, then I can take a look at it.
@collincusce which cloud are you using? AWS?
aws yes
I looked at your aws.yml file, that looks fine to me.
where you using just one command like this to run the playbooks or few steps? ```ansible-playbook -e "mode=apply" aws.yml```
I mean did you use one command like that or you used the step command?
I can but I also can run it separately, that's how I know it's wrong at the setupfabric step
either is fine.
I want to know some details so that I can help you quickly.
not a problem!! Ask away!!
so the first two steps, you did not have any errors when you run the playbooks, right?
like provisioning and initcluster
correct
zero errors
I watched, I checked
sure, good. that is what it should behave.
it created 4 hosts in aws
now, let me look at your bc1st.yml file,
sure! Gimme a sec
you have 4 nodes I assume.
no that uses 3
but I did modify to make 4 nodes for this test... when I run bc1st I get an error because it tries to access something in network {} for the fabric004 node but does
it does not fail the process, though
that is fine.
itkeeps going
you can leave one node alone
yeah. let me look at your bc1st.yml
https://gist.github.com/collincusce/8efc21f216dad2114c4afd7e734e7209
bc1st_copy.yml was added
is this the location https://gist.github.com/collincusce/8efc21f216dad2114c4afd7e734e7209
oh, good. let me take a look.
from the fabric_config.yml file, you are using all 4 nodes.
yes
if you have these two files aws_config.yml and fabric_config.yml in the vars directory, I assume that you used the right command?
should be
can you paste your provcluster and initcluster commands here.
?
do not paste your password or key here please.
from $AAROOT
ansible-playbook -e "mode=apply env=aws-us-west-2c-oregon cloud_type=aws" provcluster.yml
ansible-playbook -i run/runhosts -e "mode=apply env=aws-us-west-2c-oregon env_type=k8s" initcluster.yml
that env is a copy of what you're seeing, I j ustnamed different in gist
ansible-playbook -i run/runhosts -e "mode=apply env=bc1st_test deploy_type=k8s" setupfabric.yml
ok, so you aws_config.yml becomes aws-us-west-2c-oregon.yml?
ansible-playbook -i run/runhosts -e "mode=apply env=fabric_conffig deploy_type=k8s" setupfabric.yml
ansible-playbook -i run/runhosts -e "mode=apply env=fabric_config deploy_type=k8s" setupfabric.yml
yea
I just lazily named them in gist
that is fine, this should work.
yeah
weird error?
so you actually wanted to use k8s.
tbh for now I'm not picky
I used k8s but can do something else
at this point, you should be able to access the k8s dashboard, can you do that?
that is fine.
it should work.
gimme one moment, I have to see the state of the system
I sent my original messages yesterday afternoon, things have happened since then
but yes I could
at one point
if you will repeat this work (after you wipe clean everything), after step 2 (initcluster), you should be able to access the dashboard,
gah what's that port # again
which should give you good idea if the cluster and k8s is working appropriately.
32334 I think
to hit the very first node.
yeah, 32334 is the right port.
ah needed https
got it
yes
works
so I hate to do this but I h ave a meeting in 5 mins
yeah. log in and see the state of your cluster.
haha.
ok, so what is the error that you saw when you do setupfabric?
two pods failed on fabric001
specifically orderer1st-orgc
and
orderer1st-orgd
the error from ansible playbook run?
this might be because I was messing with it, this looks like bc1st stuff
I'll tell you what
noticed that you have 4 nodes for kafka and zookeeper,
I'll run bc1st again
and log i
it
generally speaking, these clusters should use odd number of nodes.
like 3 or 5
oh?
but does that explain the orderer issue
that looks like a file copy issue
not exactly sure.
need to see the exact error.
it's at the bottom of the file
even number of nodes is fine but in production, you should not do that.
I'll replicate it again, destroying the hosts
split brain issues.
ah
yeah.
this is a PoC
that is fine.
even number of nodes should work.
not recommended though.
also, it is generally a good idea, to have orderers in its own organization.
wew, oh ok... I wonder what else I dont know...
but the way you setup, should also work.
we just want to set up 3 channels with 4 orgs
just not how it is normally setup.
each org connects to orga
each org would also want their own composer rest api
say all your orderers go to orgo?
you are in luck because I just merged a patch set to deploy composer playground
and generating the connection profile.
but that work has not be done for k8s.
you should use flanneld setup.
at least I have not tested against k8s yet.
if you use the latest code, you should see something like this after you successfully deploy everything.
Screen Shot 2018-04-03 at 10.33.44 AM.png
notice that the ip address 192.168.56.101 is my private vmbox vm address.
@collincusce you will be able to connect to that the carauction-network to start the transaction.
it is awsome.
dude you're amazing thanks
@tongli sorry I was away in that meeting
I'm going to send you a separate gist with the output from the bc1st_test.yml
https://gist.github.com/collincusce/b8c532b9a8fece6cf50411f31a536bea
running it on the scrubbed
is the patch available with a standard git pull?
Has joined the channel.
@tongli cool and I added the 4 node version: https://gist.github.com/collincusce/8e0887e7f2efcba98896f6076fcd062a
seems to me that you are still using the old image
you changed to 1.0.0, you probably want to use 1.1.0
also I think this url to get the fabric binary may need to be updated to point to the latest one.
https://nexus.hyperledger.org/content/repositories/releases/org/hyperledger/fabric/hyperledger-fabric/linux-amd64-1.0.0/hyperledger-fabric-linux-amd64-1.0.0.tar.gz
use this one instead. https://nexus.hyperledger.org/content/repositories/releases/org/hyperledger/fabric/hyperledger-fabric/linux-amd64-1.1.0/hyperledger-fabric-linux-amd64-1.1.0.tar.gz
and change to base image tag to be x86_64-1.1.0
then I think you should simply run this command
ansible-playbook -e "mode=applu" aws.yml
ok!
@tongli what I'm doing is a fresh local copy of cello on my machine to use the ansible scripts to get jank into the cloud. Cleaned up images and all.
yeah.
the 1.1.0 version didnt work with what I was using.... so I'm going to switch after this new pull and using flanneld
oh. that is fine.
should not matter.
it did :-/
no idea why though, so ... but yeah when we switched to that version and that binary things started working
I mean from ansible agent perspective.
oh ok
running master-setup now
the binary should match the fabric release.
notice there is version info in the url?
if you use both what I gave to you, it will work fine.
you may had a mismatch
you may have a mismatch
Screenshot from 2018-04-03 14-16-09.png
and I git cloned from.... git clone https://github.com/hyperledger/cello.git
hmmm.
this is what I'm doing now
is this correct?
you are trying to use the cello container.
just following this: http://cello.readthedocs.io/en/latest/setup_master/
I normally do not do that since that container was not setup to run ansible.
hmmm well ok let me ask a direct question
use this procedure to get ansible controller up running.
I have Companies A, B, C, and D. Company A is the bossman. It owns the stuff and has its own portal and such. B, C, and D are companies which interact with A
https://github.com/hyperledger/cello/blob/master/docs/worker_ansible_howto.md#install-ansible
I want to create a PoC which deploys a network for all three companies. Each company can talk to A, but none of them can see information from each other.
I want to create a PoC which deploys a network for all four companies. Each company can talk to A, but none of them can see information from each other.
I was going to display this through rest calls that exist on the host of B,C, and D
can talk to A meaning?
channels will be A->B; A->C; A->D
NOT B->C and so forth
so it's a decentralized network from the perspective of company A, that said in reality B could take to C, just not see the A->B sstuff
cello only create one channel for verification purposes, once you setup fabric network, you can create channels as you see fit.
if that is the case, I would think you want 3 channels,
yes
1st = A + B, 2nd = A + C, 3rd = A + D.
yes!
:-)
so transactions should be done that way.
yes
B,C And D have no interaction in terms of transactions.
yes
just a proof of concept
so, what is the problem?
also forgive me, I come from the ethereum space, so if this doesn't make sense let me know
the problem is ... so I want to throw up the network that can support that on multiple hosts... I'd even love to automate the channel creation through tasks
channel separate orgs, I think that is the right way.
but one thing at a time
yeah so that's why I have the setupfabric config I sent you earlier: https://gist.github.com/collincusce/8efc21f216dad2114c4afd7e734e7209
so you create channels, and join selected the orgs to the channel.
if you look at the network section
remember that's what sparked all of this... that network layout wasn't working
4 hosts each representing 1 org
if I need to have 3 that's ok :-) or 5
that is not a problem.
yeah figured especially since it's 1:1 channels
the orderer org should not matter.
orderers can all belong to say orgo.
it is just a service.
ok
it is the channel most important.
and who is in a channel.
yeah but I can't get past setupfabric if you recall
that network layout bombed?
even if two org peers stay on the same node, that is fine.
well I'll try again with new cello on my local system using the 1.1.0
setupfabirc does not create channels.
still pulling down stuff
right
it does not, I have to build that
it only creates a test channel to verify if the network created fine.
just trying to get the network setup in AWS
for the configuration that you want, you need to create these channels and joins peers on your own.
(with the playground rest api if possible!)
cli maybe
ok
(composer rest api)_
cello, sets up playground and creates the connection profile assumes that all orgs will join the test channel.
cool, so did I do wrong by running setup-master?
if you are using ansible agent, you do not need any of these stuff.
ok
sweet
setup ansible controller, run the playbooks, you got your own network.
that is it.
so in bc1st.yml
"GERRIT_REFSPEC: "refs/tags/v1.0.5" # 1.0.5"
should that be 1.1.0 by chance?
leave what gerrit_respec unchanged.
oh... wait hmm
ok
got it
go ahead use base_image_tag
and this bin
https://nexus.hyperledger.org/content/repositories/releases/org/hyperledger/fabric/hyperledger-fabric/linux-amd64-1.1.0/hyperledger-fabric-linux-amd64-1.1.0.tar.gz
alright deploying with "ansible-playbook -e "mode=apply" aws.yml"
alright deploying with "ansible-playbook -e "mode=apply" aws-oregon.yml"
Screenshot from 2018-04-03 14-51-07.png
that happens every once in a while
Screenshot from 2018-04-03 14-56-46.png
that's what I get with the base stuff out of the box
soooo I guess flanneld doesn't exist?
was having this problem before early last week, but switching to k8s fixed it
bombed, same ordererOrganization error from this: https://gist.github.com/collincusce/8efc21f216dad2114c4afd7e734e7209
trying once more
yeah still having that problem @tongli
Screenshot from 2018-04-03 15-40-25.png
keeps happening on that network I sent you in gist: https://gist.github.com/collincusce/8efc21f216dad2114c4afd7e734e7209
so I removed one... annnd same problem
@collincusce I think this is a problem in ansible agent. please file a bug.
@collincusce at the same time, to let you go on, you can put all orderers in one org like orgo.
so peers and orderers won't share same org. that way, I believe you can keep going.
Has joined the channel.
@all this week's meeting will be cancelled due to the holidays, feel free to discuss offline :)
Has left the channel.
@dinesh.rivankar added some comments: https://jira.hyperledger.org/browse/CE-330
[ ](https://chat.hyperledger.org/channel/cello?msg=bZBwtPNq58Kz8uHC3) @baohua commented https://jira.hyperledger.org/browse/CE-330?focusedCommentId=42496&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-42496
I want to deploy fabric-chaincode_example02 in cello user dashboard
Clipboard - April 4, 2018 12:51 PM
what parameter can write ?
Has joined the channel.
Has joined the channel.
Has joined the channel.
Has joined the channel.
@baohua @tongli https://jira.hyperledger.org/browse/IN-22
Has joined the channel.
Can I deploy Hyperledger Fabric to kubernetes?
Hi all,
I tried the first command make start ,i am getting an error
cello1.png
there is nothing running in the port 8081
@rjones, I will let Chris know about the costs, will they be on aws?
[ ](https://chat.hyperledger.org/channel/cello?msg=ZSE9T9CRyLvFadDTw) @kpbird Yes, absolutely.
[ ](https://chat.hyperledger.org/channel/cello?msg=C2LPDkChR6CFQ6JKR) @Unni_1994 did you run `make setup-master` first ?
@tongli yes, AWS as we discussed. The expenditure need to be approved.
@rjones can I just need to tell Chris to approve or disapprove or there is a process?
@tongli I'll ask. I don't think it's something Chris can do on his say so
k. let me know the results. thanks.
@Amir I changed the port number from 8081 to 9081 Now it is working
good, that port must have been used by something else
@tongli @baohua please consider using https://www.cncf.io/community/infrastructure-lab/ for a cello test net. It would cost no money to Hyperledger.
@rjones, will look into it. Thx
@hightall pls have a look.
Has joined the channel.
I wish I have seen these two videos before. Helped me to understand Cello project better
https://www.youtube.com/watch?v=ndLuUSK3XTs
https://www.youtube.com/watch?v=FV3rN_kwPnA
by @tongli
https://gerrit.hyperledger.org/r/#/c/20241/
Add comments
https://gerrit.hyperledger.org/r/#/c/20241/
Hi @baohua how about I create another k8s_agent dir to store my codes? there are too many thing under k8s dir
`k8s` is the appropriate name to be aligned with other agents. can u check the k8s dir, they are some setup scripts of k8s, and should not affect u? And feel free to update the content too.
ok I will try to make some adjustment
Clipboard - 2018年4月8日下午4点16分
@baohua I created a setup-scripts dir to store the former setup scripts, is that ok ?
pls verify if any doc/code links to these scripts. i guess u can mainly check the `ansible` agent and the doc links.
sure
@baohua I have verfied that there is no `ansible` code nor docs link to these scripts.
great, then go ahead! thanks
thank you
https://gerrit.hyperledger.org/r/#/c/20241/、
https://gerrit.hyperledger.org/r/#/c/20241/
Clipboard - 2018年4月8日晚上6点20分
please help
the deploy chaincode-example02 parameter is right ?
Clipboard - 2018年4月8日晚上6点27分
I met above error when I invoked.
@hightall pls help check
@baohua Dynamic crypto creation proposal : https://docs.google.com/document/d/1HbXk0Tr7ONTJND-rz06wtHPv2FgXTzKPB8DwNvKBSsA/edit?usp=sharing
Has left the channel.
Has joined the channel.
Hello, guyes. Help me please. What a versions of docker and docker-compose i can use (maximum) for latest version of cello?
Hello, guys. Help me please. What a versions of docker and docker-compose i can use (maximum) for latest version of cello?
I suspect versions in documentation little bit is not actual.
Has joined the channel.
@baohua, for each docker image that we build on docker hub, someone has to create an automated build on docker hub, is that correct?
@baohua, if that is correct, who is doing that, I like to add a cello ansible agent image.
oh, you can add a dockerfile as other images in the source code, and then ask ry's help to create the auto-build
ah, I see.
@baohua thanks for your info.
https://gerrit.hyperledger.org/r/#/c/20241/
@baohua with this change https://gerrit.hyperledger.org/r/#/c/20317/, cello build is now taking 3 mins (Reduced almost 6 mins :wink: )
@baohua with this change https://gerrit.hyperledger.org/r/#/c/20317/, cello build is now taking 3 mins to finish (Reduced almost 6 mins :wink: )
@rameshthoomu what was done to reduce the time?
Simply using different images?
caching the cello images instead of pulling them for every build
cool, i guess it should apply to all projects that use Docker images?
And i think even with the `latest` tag, it should check to pull images every time, right? In case the dockerhub version is updated.
Has joined the channel.
Hi together, I have some questions about Cello as we need something to provision a Fabric consortium network. We have a couple of organisations involved in the Fabric/Composer network, let's say 7. All of them should have their very own control about one (or more) peer Fabric nodes. Only one channel is needed with one Composer business network definition deployed.
Is this also possible with Cello? The documentation reads like Cello sees all added infrastructure as a blockchain cluster. All infrastructure is maintained/controlled by the ones who work with Cello. But the organisations themselves do not really have control about their very own peer node.
Is Cello considering those idiomatic architectures or is the usage of Cello really forcing me to do a "centralised blockchain operations"?
Thanks :-)
ls
@jverhoelen currently, cello helps deploy the blockchain networks and add no limit on the cluster's ownership. It will depends on how people manage the underly infrastructure, i.e., is some server is controlled by an org A, then A can manage the peers in that server. We also consider to connect Cello to help manage some existing blockchain network, hope that feature can help in this scenario, too.
Hi Folks , I am an Phd Candidate in www.nitrr.ac.in my Linkedind Profile is https://www.linkedin.com/in/eramitg/ for sake of earning an Phd Degree i was proposed Blockchain Technology research work area to my guide so oom I request all of you gyus ,please guide me and assign me some research oriented task so that we mutullay benifited research related to Hyperledger Umbrella Project , All of you feel free to catch me on twitter or skype to https://twitter.com/eramitg1 or amitg.iitb skype id also in Zoom to in Zoom ID 3649222703 or whatsapp +917773011100 Regards
Has joined the channel.
Has joined the channel.
Hello All
Is cello stable and is there any documentation or tutorial for its implementation?
Has joined the channel.
Hi this is the design doc of Kubernetes agnet welcome for comments https://docs.google.com/document/d/1Z52BC95Jfx8i_i9XB18PBVnHt17YU3PeyIq0nWclF8g/edit?usp=sharing
@Luke_Chen would you like to add into tmr meeting's agenda? https://wiki.hyperledger.org/projects/cello/meetings/20180413
ok
@all Deal all, The Cello project meeting will be held at 12:30 (UTC)/20:30 (UTC+8) this Friday. https://wiki.hyperledger.org/projects/cello/meetings/20180413
+1
Has joined the channel.
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/20497/
Has joined the channel.
+2!
@hightall Could you please review my patch? https://gerrit.hyperledger.org/r/#/c/20241/
@Luke_Chen yes, +2
Thank you
@hightall Will we disable the UI code under the cello/src/themes/basic/ in the future?
@Luke_Chen yes, after the react theme rewrite done, basic & vue will be removed
@hightall ok, will it spend 2~3 weeks to rewrite the UI ?
@Luke_Chen yes, at least.
@hightall Thanks, I'm looking forward to your new UI.
https://wiki.hyperledger.org/projects/cello/meetings/20180413
https://github.com/hyperledger/cello/blob/master/src/agent/ansible/vars/bc1st.yml
https://docs.google.com/document/d/1HbXk0Tr7ONTJND-rz06wtHPv2FgXTzKPB8DwNvKBSsA/edit
@baohua The purpose of using NFS server is not only for sharing the Certificate files, but also the channel-artifacts and explorer setup files, so it may be little difficult to take out the NFS server, beside, user may have multiple worker nodes, NFS server can be the shared storage medium for these worker nodes.
and the NFS server will only against the kubernetes agent users, othere users don't need this NFS server.
Hey @baohua as per discussion with you in zoom meeting as mainatner and developer i can contribute in your cell/fabric sdk project.
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/20561/
@eramitg sure, well come. here are some information for help: https://wiki.hyperledger.org/projects/cello
@eramitg sure, welcome. here are some information for help: https://wiki.hyperledger.org/projects/cello
Thanks you sir
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/20535/
Has joined the channel.
the design doc has been refined, welcome for comments. https://docs.google.com/document/d/1Z52BC95Jfx8i_i9XB18PBVnHt17YU3PeyIq0nWclF8g/edit
Dear all, i'm updating a new version of cello introduction slides, feel free to give your comments at https://docs.google.com/presentation/d/1sq4hnTRpH9qaPJe96f6Egx9LclbCC5KcQjnBmv2LO68/edit. Thanks!
Has joined the channel.
@baohua Added my comments to above URL.
Has joined the channel.
resolved, thanks!
https://gerrit.hyperledger.org/r/#/c/20589/
Has joined the channel.
Hi, how come the directory structure in `cello-for-ste/src/agent/ansible/roles` does not match the recommended ansible directory layout for roles? ie, there seems to be no `roles/
Hi, how come the directory structure in `cello/src/agent/ansible/roles` does not match the recommended ansible directory layout for roles? ie, there seems to be no `roles/
@john.d.sheehan that was done on purpose. We support apply and destroy actions.
can not use main.yml. what is your concern?
Thanks @tongli , no concern. New to both ansible and cello so was just surprised
@john.d.sheehan ok. good. if you like to use ansible agent to stand up fabric network, please see the document here.
https://github.com/hyperledger/cello/blob/master/docs/worker_ansible_howto.md
thank you @tongli
@john.d.sheehan you are very welcome
Has joined the channel.
Has joined the channel.
People, any ideas why this is happening?```I clone cello using git clone https://gerrit.hyperledger.org/r/cello and then edit the exit
```
People, any ideas why this is happening?
-- I clone cello using "git clone https://gerrit.hyperledger.org/r/cello"
-- I edit ~/cello/src/agent/ansible/vars/aws.yml accordingly and I don't touch ~/cello/src/agent/ansible/vars/bc1st.yml
-- I do "ansible-playbook -e "mode=apply" awsk8s.yml"
PLAY [change /etc/hosts files so that each node knows other nodes] ****************************************************************************************************
TASK [Gathering Facts] ************************************************************************************************************************************************
fatal: [@@@@@@@.128]: FAILED! => {"changed": false, "module_stderr": "Shared connection to @@@@@@@.128 closed.\r\n", "module_stdout": "sudo: unable to resolve host fabric001\r\n/bin/sh: 1: /usr/bin/python: not found\r\n", "msg": "MODULE FAILURE", "rc": 127}
ok: [@@@@@@@.148]
ok: [@@@@@@@.131]
and it goes on
TASK [include] ********************************************************************************************************************************************************
included: /home/ubuntu/cello/src/agent/ansible/roles/cloud_os/namenode/tasks/apply.yml for @@@@@@@.148
included: /home/ubuntu/cello/src/agent/ansible/roles/cloud_os/namenode/tasks/apply.yml for @@@@@@@.131
changed: [@@@@@@@.148] => (item=10.0.0.218 fabric001)
changed: [@@@@@@@.131] => (item=10.0.0.218 fabric001)
changed: [@@@@@@@.148] => (item=10.0.0.93 fabric002)
changed: [@@@@@@@.131] => (item=10.0.0.93 fabric002)
changed: [@@@@@@@.148] => (item=10.0.0.83 fabric003)
TASK [Setup /etc/hosts for the node] **********************************************************************************************************************************
changed: [@@@@@@@.131] => (item=10.0.0.83 fabric003)
till I get
TASK [Start the flanneld service] *************************************************************************************************************************************
fatal: [@@@@@@@.148]: FAILED! => {"changed": false, "msg": "Unable to start service flanneld: Job for flanneld.service failed because a timeout was exceeded. See \"systemctl status flanneld.service\" and \"journalctl -xe\" for details.\n"}
fatal: [@@@@@@@.131]: FAILED! => {"changed": false, "msg": "Unable to start service flanneld: Job for flanneld.service failed because a timeout was exceeded. See \"systemctl status flanneld.service\" and \"journalctl -xe\" for details.\n"}
and
and PLAY RECAP ************************************************************************************************************************************************************
@@@@@@@.131 : ok=40 changed=23 unreachable=0 failed=1
@@@@@@@.128 : ok=0 changed=0 unreachable=0 failed=1
@@@@@@@.148 : ok=40 changed=23 unreachable=0 failed=1
cloud : ok=30 changed=9 unreachable=0 failed=0
fabric001 : ok=6 changed=1 unreachable=0 failed=0
fabric002 : ok=6 changed=1 unreachable=0 failed=0
fabric003 : ok=6 changed=1 unreachable=0 failed=0
More info:
ubuntu@fabric002:~$ systemctl status flanneld.service
● flanneld.service - Fabric Flanneld Service
Loaded: loaded (/lib/systemd/system/flanneld.service; enabled; vendor preset: enabled)
Active: activating (start) since Mon 2018-04-16 22:01:07 UTC; 49s ago
Main PID: 12869 (flanneld)
Tasks: 7
Memory: 10.9M
CPU: 94ms
CGroup: /system.slice/flanneld.service
└─12869 /opt/fabric/bin/flanneld
Apr 16 22:01:51 fabric002 flanneld[12869]: timed out
Apr 16 22:01:51 fabric002 flanneld[12869]: E0416 22:01:51.575475 12869 main.go:349] Couldn't fetch network config: 100: Key not found (/coreos.com) [6]
Apr 16 22:01:52 fabric002 flanneld[12869]: timed out
Apr 16 22:01:52 fabric002 flanneld[12869]: E0416 22:01:52.581464 12869 main.go:349] Couldn't fetch network config: 100: Key not found (/coreos.com) [6]
Apr 16 22:01:53 fabric002 flanneld[12869]: timed out
Apr 16 22:01:53 fabric002 flanneld[12869]: E0416 22:01:53.584717 12869 main.go:349] Couldn't fetch network config: 100: Key not found (/coreos.com) [6]
Apr 16 22:01:54 fabric002 flanneld[12869]: timed out
Apr 16 22:01:54 fabric002 flanneld[12869]: E0416 22:01:54.587921 12869 main.go:349] Couldn't fetch network config: 100: Key not found (/coreos.com) [6]
ubuntu@fabric002:~$ journalctl -xe
Apr 16 22:02:00 fabric002 flanneld[12869]: E0416 22:02:00.609675 12869 main.go:349] Couldn't fetch network config: 100: Key not found (/coreos.com) [6]
Apr 16 22:02:01 fabric002 flanneld[12869]: timed out
Apr 16 22:02:01 fabric002 flanneld[12869]: E0416 22:02:01.613025 12869 main.go:349] Couldn't fetch network config: 100: Key not found (/coreos.com) [6]
Apr 16 22:02:02 fabric002 flanneld[12869]: timed out
Apr 16 22:02:02 fabric002 flanneld[12869]: E0416 22:02:02.616412 12869 main.go:349] Couldn't fetch network config: 100: Key not found (/coreos.com) [6]
Apr 16 22:02:03 fabric002 flanneld[12869]: timed out
Apr 16 22:02:03 fabric002 flanneld[12869]: E0416 22:02:03.619548 12869 main.go:349] Couldn't fetch network config: 100: Key not found (/coreos.com) [6]
Apr 16 22:02:03 fabric002 etcd[12564]: health check for peer e20231f48cc1bd6d could not connect: dial tcp 10.0.0.218:2380: getsockopt: connection refused
Apr 16 22:02:04 fabric002 flanneld[12869]: timed out
Apr 16 22:02:04 fabric002 flanneld[12869]: E0416 22:02:04.622935 12869 main.go:349] Couldn't fetch network config: 100: Key not found (/coreos.com) [6]
@hightall is this fixed? https://jira.hyperledger.org/browse/CE-195
@baohua haven't see this error now.
ok, then pls close it.
ok
@baohua Got some comments from the community, could you please take a look? https://docs.google.com/document/d/1Z52BC95Jfx8i_i9XB18PBVnHt17YU3PeyIq0nWclF8g/edit
i've seen them, and seems introducing fabric-ca is a good idea?
It's good idea, but it still need some config file to start the fabric-ca, am I right?
and If we want to introduce explorer and composer later, we need to find a place to store all these start up files, and share among the Kubernetes workers.
Has joined the channel.
ERROR: for cello-nginx Cannot start service nginx: driver failed programming external connectivity on endpoint cello-nginx (d9b2389396e1aa0bfc310b6e35ad5574391df34982757d7145ab727d7ba56c56): Bind for 0.0.0.0:8080 failed: port is already allocated
Can any one help to find a solution for this?
It seems an application has already taken over your 8080 port, shut it down an restart Cello again.
Query chains failed!?
Why does this occur on Dashboard
@Luke_Chen correct, and i suppose in k8s, we can set some config with the started service using annotation or secret? Is this some possible way?
@baohua I think it's difficult without using shared storage
Has joined the channel.
hi everyone, I am trying to setup my cello local environment using the tutorial https://cello.readthedocs.io/en/latest/tutorial/ everything works (all docker containers are up) and can access the operator dashboard
the problem comes when I try to create a new docker host (I am on a Mac) I get this error: cello-operator-dashboard | [2018-04-17 11:35:56,757] DEBUG [modules.host] [host.py:91 create()] - Create host: name=dam, worker_api=127.0.0.1:2375, host_type=docker, capacity=1, log=local/, autofill=false, schedulable=false
cello-operator-dashboard | [2018-04-17 11:35:56,767] ERROR [agent.docker.docker_swarm] [docker_swarm.py:173 check_daemon()] - Exception in check_daemon Error while fetching server API version: HTTPConnectionPool(host='127.0.0.1', port=2375): Max retries exceeded with url: /version (Caused by NewConnectionError('
I have tried enabling the docker daemon API following using this: docker run -d -v /var/run/docker.sock:/var/run/docker.sock -p 127.0.0.1:2375:2375 bobrik/socat TCP-LISTEN:2375,fork UNIX-CONNECT:/var/run/docker.sock
$ docker -H 127.0.0.1:2375 info
last command works and it seems the API is working
any help would be appreciated :)
Has joined the channel.
@frustrated-chain seems the network config is not fetched? maybe @tongli could help with more details.
thank you @baohua, I figured that subnet mask is wrong in clonned /cello/src/agent/ansible/vars/aws.yml :```
container_network: {
Network: "172.16.0.0/16",
SubnetLen: 24,
SubnetMin: "172.16.0.0",
SubnetMax: "172.16.255.0",
Backend: {
Type: "udp",
Port: 8285
}```
Let me check if it fixes the problem.
thank you @baohua, I figured that subnet mask is wrong in cloned /cello/src/agent/ansible/vars/aws.yml :```
container_network: {
Network: "172.16.0.0/16",
SubnetLen: 24,
SubnetMin: "172.16.0.0",
SubnetMax: "172.16.255.0",
Backend: {
Type: "udp",
Port: 8285
}```
Let me check if it fixes the problem.
Has joined the channel.
Good afternoon! The http://cello.readthedocs.io/en/latest/setup_master/ documentation states that the hardware requirement is "8c16g100g". Does that mean 8 CPUs, 16 GB RAM and 100GB drive?
Has joined the channel.
Has left the channel.
@baohua and @tongli this is what happens when I do the same process using aws.yml (and not awsk8s.yml)```
TASK [Create crypto-config file] ************************************************************************************
changed: [34.207.117.108]
TASK [Create the certificate files] *********************************************************************************
fatal: [34.207.117.108]: FAILED! => {"changed": false, "cmd": "/opt/gopath/bc1st/src/github.com/hyperledger/fabric/build/bin/cryptogen generate --config crypto-config.yml --output keyfiles", "msg": "[Errno 2] No such file or directory", "rc": 2}
to retry, use: --limit @/home/ubuntu/cello/src/agent/ansible/aws.retry
PLAY RECAP **********************************************************************************************************
34.207.117.108 : ok=96 changed=41 unreachable=0 failed=1
34.224.38.81 : ok=62 changed=33 unreachable=0 failed=0
54.144.200.113 : ok=62 changed=33 unreachable=0 failed=0
cloud : ok=28 changed=9 unreachable=0 failed=0
fabric001 : ok=6 changed=1 unreachable=0 failed=0
fabric002 : ok=6 changed=1 unreachable=0 failed=0
fabric003 : ok=6 changed=1 unreachable=0 failed=0
```
@baohua @tongli. This is what happens when I do the same process with aws.yml (and not awsk8s.yml):```
TASK [Create crypto-config file] ************************************************************************************
changed: [@@@@@.108]
TASK [Create the certificate files] *********************************************************************************
fatal: [@@@@@.108]: FAILED! => {"changed": false, "cmd": "/opt/gopath/bc1st/src/github.com/hyperledger/fabric/build/bin/cryptogen generate --config crypto-config.yml --output keyfiles", "msg": "[Errno 2] No such file or directory", "rc": 2}
to retry, use: --limit @/home/ubuntu/cello/src/agent/ansible/aws.retry
PLAY RECAP **********************************************************************************************************
@@@@@.108 : ok=96 changed=41 unreachable=0 failed=1
@@@@@.81 : ok=62 changed=33 unreachable=0 failed=0
@@@@@.113 : ok=62 changed=33 unreachable=0 failed=0
cloud : ok=28 changed=9 unreachable=0 failed=0
fabric001 : ok=6 changed=1 unreachable=0 failed=0
fabric002 : ok=6 changed=1 unreachable=0 failed=0
fabric003 : ok=6 changed=1 unreachable=0 failed=0
```
@baohua @tongli This is what happens when I do the same process with aws.yml (and not awsk8s.yml):```
TASK [Create crypto-config file] ************************************************************************************
changed: [@@@@@.108]
TASK [Create the certificate files] *********************************************************************************
fatal: [@@@@@.108]: FAILED! => {"changed": false, "cmd": "/opt/gopath/bc1st/src/github.com/hyperledger/fabric/build/bin/cryptogen generate --config crypto-config.yml --output keyfiles", "msg": "[Errno 2] No such file or directory", "rc": 2}
to retry, use: --limit @/home/ubuntu/cello/src/agent/ansible/aws.retry
PLAY RECAP **********************************************************************************************************
@@@@@.108 : ok=96 changed=41 unreachable=0 failed=1
@@@@@.81 : ok=62 changed=33 unreachable=0 failed=0
@@@@@.113 : ok=62 changed=33 unreachable=0 failed=0
cloud : ok=28 changed=9 unreachable=0 failed=0
fabric001 : ok=6 changed=1 unreachable=0 failed=0
fabric002 : ok=6 changed=1 unreachable=0 failed=0
fabric003 : ok=6 changed=1 unreachable=0 failed=0
```
Has joined the channel.
I wish to make a possible controversial change to Gerrit. I'm asking all the teams that use Gerrit. May I require JIRA links in all Gerrit changes? this isn't something I can enable on a per-project basis.
Has joined the channel.
@rjones you mean that for each gerrit commit, needs to include the jira issue number like now?
yes
it would be enforced by Gerrit
right now, you get an advisory warning when you don't have one
```remote: Hint: insert one or more issue-id anywhere in the commit message.
remote: Issue-ids are strings matching ([A-Z][A-Z0-9]{1,9}-\d+)
remote: and are pointing to existing tickets on its-jira Issue-Tracker
```
that is what you see today
@hightall would you like to have a look at https://jira.hyperledger.org/browse/CE-300?
@rjones I think it's a good idea, as we already recommend people to follow this rule, and the format is `[CE-xxx] some commit headline`
@rjones I think it's a good idea, as we already recommend people to follow this rule, and the format is "[CE-xxx] some commit headline"
thank you, I agree
by the way, an old requirement, is it possible to send some auto-notification in the rocketchat channel, when some commit is submitted or merged?
by the way, an old requirement, is it possible to send some auto-notification in the rocketchat channel, when some commit is submitted, passed-ci or merged (the gerrit link will be posted to jira item)?
currently, we have to do this manually...
It is something that needs work. We now have a hubot instance running
so what is needed is to plug hubot into jira
sure, look forward to seeing any progress on this, will save lots of time
https://gerrit.hyperledger.org/r/#/c/20535/
+2
@baohua Thank you
@frustrated-chain seems to me that you did not configure where to get the bin of the fabric.
@frustrated-chain if you place your bc1st.yml file, I can take a look.
[ ](https://chat.hyperledger.org/channel/cello?msg=xCqA2whJWYMCpHZqn) @mvillalobos to check the docker demon in the worker node you have to pass the worker node IP. You did `docker -H 127.0.0.1:2375 info` this checks the localhost try `docker -H YOUR-WORKER-NODE-IP:2375 info`
If that works the cello-master should see the worker
Good afternoon! Is Docker 17.0+ still experimental?
Should we be using the Cello Jira instance "Cello-Feature Design" for documentation issues? I'd like to contribute some patches as I'm working through the documents.
Who should I add as reviewers to https://gerrit.hyperledger.org/r/#/c/20737/ ?
Hi @baohua , I talked to @tkuhrt last week about some code we (Luxoft) would like to contribute to the Hyperledger foundation. We have developed a yaml driven automation tools to configure and deploy a fabric network. All is written in Java. Our code has been leveraged on several projects, some in pilot production.
Tracy suggested I talk to you about where we might fit in the cello project. There might be some crossover with cello but also useful new features. Also, we did a code walkthrough and a demo in the Netherlands with @mpiekarska and @sstone1 and it sounds like composer might also leverage some of our work. I can provide more details if you like. It would be helpful to schedule a meeting to go over what we have and what the next steps might be.
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/20727/
@tgehrke thanks for the info, sure, would you like to join our cello meeting on Friday and discuss that? 8:30 PM at UTC+8. Pls let me know if that time works. Besides, is there any doc or code we can have a look before the discussion? :)
@here Dear all, we will have project meeting at 8:30PM--9:00PM (UTC+8) this Friday evening , welcome to join, and please help update the agenda at: https://wiki.hyperledger.org/projects/cello/meetings/20180420. Thanks!
Has joined the channel.
Cello's new mail list is ready now (old data has been migrated): https://lists.hyperledger.org/g/cello/topics
@hightall as jiang @lujiang suggested, should we close this first? https://jira.hyperledger.org/browse/CE-300#add-comment
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=fqKRk7gaJshHn682s) @mvillalobos Did you found the solution ? I have exactly the same issue
[ ](https://chat.hyperledger.org/channel/cello?msg=upi689WKGyBmJDq9k) @mvillalobos Did you found the solution ? I have exactly the same issue
https://gerrit.hyperledger.org/r/#/c/20767/
Has joined the channel.
@baohua the component that might fit into cello is https://github.com/Luxoft/fabric-utils There is also https://github.com/Luxoft/fabric-skeleton that uses fabric-utils. I think composer was interested in fabric-skeleton. . . I will do my best to join your meeting on Friday
@baohua the component that might fit into cello is https://github.com/Luxoft/fabric-utils There is also https://github.com/Luxoft/fabric-skeleton that uses fabric-utils. I think composer was interested in fabric-skeleton. . . I will join your meeting on Friday, looking forward to talking.
thanks, let's discuss then @tgehrke
i will add them to the meeting agenda
Sounds good. I hope to get my team mate Aleksander to join as well.
Sounds good. I hope to get my team mate Aleksander to join as well. @baohua
sure
@hightall +2
Has left the channel.
Has joined the channel.
Has joined the channel.
(When) Will Cello be available for s390x ?
@a.ochs cello docker-images for s390x are available from v0.8.0 onwards
Are you looking for anything specific as such ?
And a native Version?
Is the worker api is working on Mac OS for you ?
Is the worker api working on Mac OS for you ? I get the two dashboard without problem
But when I want to create a new host it's not possible. I tried different IP same issue...
I am new to Cello. I just watched the two great videos below re. how to create a 3 worker K8 cluster on AWS to run a 2 Organization network with 3 peers each all in the same K8 cluster. I would like some help understanding how Cello could help to create a real decentralized network where Organizations run peers on separate K8 clusters in the same or different clouds.
https://www.youtube.com/watch?v=ndLuUSK3XTs
https://www.youtube.com/watch?v=FV3rN_kwPnA
I am new to Cello. I just watched the two great videos below re. how to create a 3 worker K8 cluster on AWS to run a 2 Organization network with 3 peers each all in the same K8 cluster. I would like some help understanding how Cello could help to create a real decentralized network where Organizations run peers on separate K8 clusters in the same or different clouds.
https://www.youtube.com/watch?v=ndLuUSK3XTs
https://www.youtube.com/watch?v=FV3rN_kwPnA
I am new to Cello. I just watched the two great videos below re. how to create a 3 worker K8 cluster on AWS to run a 2 Organization network with 3 peers each all in the same K8 cluster.
Can somebody help me understand how Cello could help to create a real decentralized network where Organizations run peers on separate K8 clusters in the same or different clouds?
https://www.youtube.com/watch?v=ndLuUSK3XTs
https://www.youtube.com/watch?v=FV3rN_kwPnA
I am new to Cello. I just watched the two great videos below re. how to create a 3 worker K8 cluster on AWS to run a 2 Organization network with 3 peers each all in the same K8 cluster.
Can somebody help me understand how Cello could be leveraged to create a real decentralized network where Organizations run peers on separate K8 clusters in the same or different clouds?
https://www.youtube.com/watch?v=ndLuUSK3XTs
https://www.youtube.com/watch?v=FV3rN_kwPnA
I am new to Cello. I just watched the two great videos below re. how to create a 3 worker K8 cluster on AWS to run a 2 Organization network with 3 peers each all in the same K8 cluster.
Can somebody help me understand how Cello could be leveraged to create a real decentralized HL Fabric network where Organizations run peers on separate K8 clusters in the same or different clouds?
https://www.youtube.com/watch?v=ndLuUSK3XTs
https://www.youtube.com/watch?v=FV3rN_kwPnA
I am new to Cello. I just watched the two great videos below re. how to create a 3 worker K8 cluster on AWS to run a 2 Organization network with 3 peers each all in the same K8 cluster.
https://www.youtube.com/watch?v=ndLuUSK3XTs
https://www.youtube.com/watch?v=FV3rN_kwPnA
Can somebody help me understand how Cello could be leveraged to create a real decentralized HL Fabric network where Organizations run peers on separate K8 clusters in the same or different clouds?
https://www.youtube.com/watch?v=ndLuUSK3XTs
https://www.youtube.com/watch?v=FV3rN_kwPnA
I am new to Cello. I just watched the two great videos below re. how to create a 3 worker K8 cluster on AWS to run a 2 Organization network with 3 peers each all in the same K8 cluster.
https://www.youtube.com/watch?v=ndLuUSK3XTs
https://www.youtube.com/watch?v=FV3rN_kwPnA
Can somebody help me understand how Cello could be leveraged to create a real decentralized HL Fabric network where Organizations run peers on separate K8 clusters in the same or different clouds?
Has joined the channel.
Has joined the channel.
Has joined the channel.
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/20863/
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/20873/
+2!
Has joined the channel.
Has joined the channel.
@baohua I'm sorry I missed the meeting on Friday. I got the times mixed up. I will fir sure join this Friday. In the mean time is there anything I might provide? Have you had a chance to talk a look at the source?
@baohua I'm sorry I missed the meeting on Friday. I got the times mixed up. I will for sure join this Friday. In the mean time is there anything I might provide? Have you had a chance to talk a look at the source?
@tgehrke thanks. We have discussed the issue last week, and feeling it's quite useful to one of our planed feature: allow user input the network parameters through the dashboard, and then generate some network template automatically. Then the agents (docker,k8s) can take the template to start a new network.
Since we will cancel this week's meeting due to the labor days holidays, let's discuss here.
One question is that most of cello's code is implemented in Python, is it possible to help create a python version based on the existing fabric-skeleton tool?
@baohua we though fabric-utils was the component that fit into cello. That part wraps the fabric SDK to setups the crypto configuration. Calling fabric-skeleton uses fabric-utils. I don't think translating the Java code into Python is something we want to do on our side. Is it a problem supporting Java in cello?
@baohua we though fabric-utils was the component that fit into cello. That part wraps the fabric SDK to setup the crypto configuration. Calling fabric-skeleton uses fabric-utils. I don't think translating the Java code into Python is something we want to do on our side. Is it a problem supporting Java in cello?
Would it be possible to schedule a meeting at another time? I'd like to include our development lead. Most of the team is in St Petersburg so their time zone is closer to yours.
@baohua At one point @sstone1 was thinking parts of fabric-skeleton might be useful in composer. Do the cello and composer project leverage each others work?
Not yet, but we've seen some possibility to integrate composer into cello. We can discuss in next week's meeting, hope not too late.
And, there's no tech problem to support java in cello, but you know, multiple languages implementation may make the project hard to maintain in open-source way. The core dev team is focusing on python and javascript.
besides, it is not a 'translating', but more like take the existing code as reference, and implement the necessary functionality in cello. Does that make sense? Thanks!
@baohua Yes I understand how it gets hard with multiple languages. That's the reason we implemented in Java, we don't develop our solutions in Python. I look forward to talking with you and Aleksandr I'll see if I can get him to respond on rocket chat today.
sure, thanks! and we will be open for any volunteers that can help on this, too!
@baohua while creating a Fabric cluster in Cello, there are two things need to do at module layer: 1. call the relative agent to create container 2. wait until the return from the agnet layer and update the db
base on the current design of the module layer, the first operations I mentioned above will block the return of the module layer, if the container need to take few mins to create that will strongly affect user experience
this is actually happening in Kubernetes agent
so I propose to put the tow operations I mentioned above to a Thread, how do you think?
@Luke_Chen you mean to create fabric using k8s?
yes
I think it's fine just check the response code by k8s API, and use some health field to track whether the network is up successfully?
e.g., make it async
It need 1~2 mins to create a Fabric cluster
if the k8s API response OK, then we will think the cluster is in "creating" status.
once the health check passed, then can mark it as 'ready'?
```containers = self.cluster_agents[worker.type] \
.create(cid, mapped_ports, self.host_handler.schema(worker),
config=config, user_id=user_id)
if not containers:
logger.warning("failed to start cluster={}, then delete"
.format(name))
self.delete(id=cid, record=False, forced=True)
return None```
cluster creation will block here
It seem we need to put the response code to `containers` varible unless we add an i`f statement` to check weather the host type is kubernetes
``` # start compose project, failed then clean and return
logger.debug("Start compose project with name={}".format(cid))
containers = self.cluster_agents[worker.type] \
.create(cid, mapped_ports, self.host_handler.schema(worker),
config=config, user_id=user_id)
if not containers:
logger.warning("failed to start cluster={}, then delete"
.format(name))
self.delete(id=cid, record=False, forced=True)
return None
# creation done, update the container table in db
for k, v in containers.items():
container = Container(id=v, name=k, cluster=cluster)
container.save()
# service urls can only be calculated after service is created
service_urls = self.gen_service_urls(cid, peer_ports, ca_ports,
orderer_ports, explorer_ports)
# update the service port table in db
for k, v in service_urls.items():
service_port = ServicePort(name=k, ip=v.split(":")[0],
port=int(v.split(":")[1]),
cluster=cluster)
service_port.save()
# update api_url, container, user_id and status
self.db_update_one(
{"id": cid},
{
"user_id": user_id,
'api_url': service_urls.get('rest', ""),
'service_url': service_urls,
'status': NETWORK_STATUS_RUNNING
}
)```
more specific, is it a good idea to put above code to a Thread?
the codes are from cello/src/module/cluster.py lines 271-306
Has joined the channel.
If k8s agent return a response code during the cluster creation, there must be a `if` statement to suite it need, rather than we wiil put the response code to `containers` varible which cause confusion
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/20953/
@Luke_Chen are u saying `self.cluster_agents[worker.type].create()` is a blocking call?
i guess we can change it to some async call.
if the call takes too much time.
@hightall +2!
@baohua thanks
@baohua correct, I am working on it.
thanks :)
Hi all, any idea on the health issue statement failed after created a cluster ? ( Kafka and solo, worker already setup, host created )
Hi all, any idea on the health issue statement failed after created a cluster ? ( same issue with Kafka and solo, worker already setup, host created )
https://stackoverflow.com/questions/50018228/cello-run-everything-on-one-clean-ubuntu-server-fails-with-404
where can i post issues related to cello
Has joined the channel.
hey, im trying to setup a simple cello deployment with a docker backend and i tried adding a new chain a couple of times now, the issue i cant get past is that cello seems to want to pull an image from docker repo that doesnt even exist (i checked): hyperledger/fabric-ca
cello-issue.png
@tongli it seems related to ansible agent, can u help have a look?
https://stackoverflow.com/questions/50018228/cello-run-everything-on-one-clean-ubuntu-server-fails-with-404
sorry it does exist... strange though
any ideas what could be the cause?
have u followed the tutorial and run the `make setup` in both master nodes and worker nodes?
from the error message, it seems u do not have required docker images in worker nodes.
ah i see
thanks this will probably help
@baohua @Luke_Chen Hi guys, I think I need to better understand the cello goals. The fabric-skeleton code we have is something we use to deploy a "production-like" fabric network. It is intended to launch multiple VMs and distribute the services across the network. The time it takes to set everything up can be quite long. We were looking at what @sstone1 and his team were doing with composer which primarily works with a sandbox and thought fabric-skeleton might be a good fit for doing a distributed network. Is deploying a distributed network something that fits into cello? The modules in fabric-utils just abstract the sdk calls and could be used for a smaller deployment. The fabric-utils might be more useful to cello. . .
@baohua @Luke_Chen Hi guys, I think I need to better understand the cello goals. The fabric-skeleton code we have is something we use to deploy a "production-like" fabric network. It is intended to launch multiple VMs and distribute the services across the network. The time it takes to set everything up can be quite long. We were looking at what @sstone1 and his team were doing with composer which primarily works with a sandbox and thought fabric-skeleton might be a good fit for doing a distributed network. Is deploying a distributed network something that fits into cello? The modules in fabric-utils just abstract the sdk calls and could be used for a smaller deployment. The fabric-utils might be more useful to cello. . .
@tgehrke Cello is mean to facilitate the application development among the Fabric network, and it has role called `operator` who is responsible to delpoy and maintain Fabric network, more important `operator` can deploy Fabric network on different platform like Docker swarm and Kubernetes, check this link for detials.
https://docs.google.com/presentation/d/1sq4hnTRpH9qaPJe96f6Egx9LclbCC5KcQjnBmv2LO68/edit#slide=id.p15
Has joined the channel.
Has joined the channel.
@tgehrke @Luke_Chen thanks for the comments. Cello's goal is to help provision and operate blockchain networks ( think about cello, who plays music :) ), which we also call it some management tools. Now it is capable to support deploying blockchains into various physical infrastructures using the corresponding agent, like VM (using vcenter or ansible), baremetal (ansible), docker, swarm, k8s. Although we do not focus on how to install swarm or k8s, but the project is quite open to features that help people use blockchain in easy ways. Feel free to raise more questions or comments, thanks!
@Luke_Chen @baohua thanks that helps. Do you see the code we provided as useful? It's mostly Ansible driven. It builds out a kafka base network. Is that something you are already doing? We also have some tooling to provision a network using AWS ec2.
yes, i believe it's useful, and we already have some ansible agent in cello, would you like to have a look?https://github.com/hyperledger/cello/tree/master/src/agent/ansible
thanks!
thanks
thanks - This looks very close. I see you have provisioning for azure and aws
Has joined the channel.
Hello everyone, I implemented a Fabric-Solution using Composer and I do have a deployable bna file which already worked on a local dev environment. Now I want to deploy a network using multiple AWS instances. Using Cello with ansible worker nodes seems to be good idea.
Am I on the right track?
I updated the vars/aws.yml and set the env variable. Wheh I run "ansible-playbook -e "mode=apply" aws.yml" it stops after a few seconds with "fatal: [cloud]: FAILED! => {"changed": false, "msg": "boto required for this module, install via pip or your package manager"}"
even after I installed bote using 'npm install -g boto'
solved this boto issue. installed using pip
now stuck with "[cloud]: FAILED! => {"msg": "could not locate file in lookup: /home/ubuntu/.ssh/fd.pub"}"
Has joined the channel.
there is not such a file in .ssh. Should that have been created in any step before?
hi, saschaka, if you wanna use our ansible agent, would you like to follow the guideline at http://cello.readthedocs.io/en/latest/setup_worker_ansible_allinone/? Thanks!
Has joined the channel.
@baohua the user-dashboard use the node ?
@baohua the user-dashboard use the node ? what's the relationship host,cluster?
mainly javascript
@hightall might provide more details
@asaningmaxchain123 yes, it's written in node with express framework.
host is some resource pool that can be used by cello to run blockchain on, e.g., a k8s cluster, a vm cloud
cluster is a blockchain network instance
https://cello.readthedocs.io/en/latest/
the cluster can deploy in different host?
the cluster can deploy in different host? use the docker tcp 2375 port to manager docker?
no, currently, one cluster can only be deployed into one host
[ ](https://chat.hyperledger.org/channel/cello?msg=aAS8KsT5JZbvQg5pN) @baohua in the future,it will support?
[ ](https://chat.hyperledger.org/channel/cello?msg=tyjcy2iwjcjuZCbLb) @hightall thx
can u help elaborate why need to deploy one cluster across, like multiple k8s clusters?
can this help @asaningmaxchain123 http://cello.readthedocs.io/en/latest/terminology/
@baohua I opened a ticket pertaining to cello, cello-analytics, and cmonit that you may have encountered: https://github.com/yeasy/cmonit/issues/1
I'm not sure why there's a connection init error to the worker node
I just added to the ticket the results of running `docker -H WORKER_NODE_IP:2375 info`
One thing I noticed on cmonit is that you mention containers should have `label=monitor=true` but don't specify where this would go. Not sure if it's pertinent to finding a solution.
Separate question/thought. It doesn't appear as though there's been any updates in regards to adding support of v1.1 since March 15th. At a minimum there should be a branch that works on compatibility, no?
@coffeeplease it's for future usage in monitoring
https://gerrit.hyperledger.org/r/#/c/20861/
+2!
Has joined the channel.
Has joined the channel.
@baohua when i use `make docker` cmd to build image,the watchdog,engine images doesn't contain,i will pull the images for the dockerhub
@baohua when i use `make docker` cmd to build image,the watchdog,engine images doesn't contain,i will pull the images for the dockerhub,i think you should add it
Has joined the channel.
Hello, I'm trying to run the cello ansible aws.yml and I'm stuck with the etcd not starting: TASK [Enable and start etcd services] ***********************************************************************************************************************
fatal: [13.55.43.142]: FAILED! => {"changed": false, "msg": "Unable to start service etcd: Job for etcd.service failed because the control process exited with error code. See \"systemctl status etcd.service\" and \"journalctl -xe\" for details.\n"}
fatal: [13.55.23.167]: FAILED! => {"changed": false, "msg": "Unable to start service etcd: Job for etcd.service failed because the control process exited with error code. See \"systemctl status etcd.service\" and \"journalctl -xe\" for details.\n"}
changed: [13.211.162.44]
this then leads to flanneld not finding its keys
when running journalctl -xe on the etcd host I see an error: couldn't find local name "fabric001" in the initial cluster configuration
I'm using the master branch from github
@baohua would you be able to give me some hints? I'm doing this in a fresh AWS account and not changed much in the vars/aws.yml file other than user key and region
OK, I did a complete fresh start. New ubuntu server and followed this guide: https://github.com/hyperledger/cello/blob/master/docs/worker_ansible_howto.md
I got past the etcd problem, but now I'm stuck again with the same problem that @frustrated-chain reported above:
Clipboard - April 29, 2018 9:48 AM
so somehow the cryptogen binary is neither built nor downloaded
I've created a JIRA issue for this: https://jira.hyperledger.org/browse/CE-350
hi, just newly setup a master node and a worker node
when i add a host in dashboard, get this error
cello-operator-dashboard | [2018-04-29 02:07:46,687] ERROR [agent.docker.docker_swarm] [docker_swarm.py:311 setup_container_host()] - 403 Client Error: Forbidden ("cannot create network 06fc2852be7cd033fff8fee16a347b4b508a651736fb2a7ced779402d00abd4b (br-06fc2852be7c): conflicts with network a7d5f8dd42a9d07a2f46114a1bba5a9dec08d2d77667085d3dae43fdc0876132 (br-a7d5f8dd42a9): networks have overlapping IPv4")
`cello-operator-dashboard | [2018-04-29 02:07:46,687] ERROR [agent.docker.docker_swarm] [docker_swarm.py:311 setup_container_host()] - 403 Client Error: Forbidden ("cannot create network 06fc2852be7cd033fff8fee16a347b4b508a651736fb2a7ced779402d00abd4b (br-06fc2852be7c): conflicts with network a7d5f8dd42a9d07a2f46114a1bba5a9dec08d2d77667085d3dae43fdc0876132 (br-a7d5f8dd42a9): networks have overlapping IPv4")`
my worker node is a centos7 box, with docker v18.04-ce and swarm disabled
my centos box has a existed bridge network, is there a way to config the worker network from this conflict?
https://gerrit.hyperledger.org/r/21183
Has joined the channel.
Has joined the channel.
I have an odd question (new to Ansible here so making this hard for me) When trying to deploy a network I'm getting an error for `org_anchor` being undefined from https://github.com/hyperledger/cello/blob/master/src/agent/ansible/roles/deploy_compose/certsetup/tasks/apply.yml#L154
I'm trying to find reasons for why this might be the case.
It's really not obvious to me
the only thing I can imagine is that the default we are giving it an empty object
Ah the more I dig into this it looks as if there is an issue with the anchor_peers. Either I'm not getting any anchor peers to pull from or I'm not getting one for that org
Looks like anchor_peers is empty and thus the root of my issues. We are going to continue to look deeper into this. I'll report back on the cause when I find it. If anyone knows of a quick place I should look please let me know.
I think we have isolated this down (credit to one of our developers Alex)
``` - name: Get peer object list
set_fact:
peers: |
{{ peers | default([]) + [{'org':item.split('@')[1].split('.')[-1],
'name':item.split('@')[1] | replace('.','-'),
'role':item.split('@')[0]}] }}
with_items: "{{ peerls }}"```
This seems to be only adding one peer the the list of peers and overwriting that peer each time a new one is created for the list.
Syntactically this looks right though. So I'm confused as to why this is happening
@ChrisFowers what does your configuration file look like?
the empty list should work ok.
@ChrisFowers the code above is not doing what you suspected to do.
```
network: {
oceantest001: {
cas: ["ca.speedydelivery"],
peers: ["anchor@peer.speedydelivery", "worker@peer.speedydelivery"],
orderers: ["orderer.speedydelivery"],
zookeepers: ["zookeeper1"],
kafkas: ["kafka1"]
},
oceantest002: {
cas: ["ca.arrayofblocks"],
peers: ["worker@peer.arrayofblocks", "anchor@peer.arrayofblocks"],
orderers: ["orderer.arrayofblocks"],
zookeepers: ["zookeeper2"],
kafkas: ["kafka2"]
}
},```
The code to set_fact for peers seems to only have `worker@peer.speedydelivery` for oceantest001 and `worker@peer.speedydelivery` for oceantest002
The code to set_fact for peers seems to only have `worker@peer.speedydelivery` for oceantest001 and `anchor@peer.arrayofblocks` for oceantest002
When we run the play
So for ocean test one we don't ever get the anchor peer
It looks as though as the play builds that list from the config it keeps replacing the one record in the array it's creating
¯\_(ツ)_/¯
[ ](https://chat.hyperledger.org/channel/cello?msg=Ev2GBM5MXWc9tAxaB) originally.. you're right
The default empty object is fine afterall
The real issue I see is that the ouput of `pearls` is
```
[
"anchor@peer.speedydelivery",
"worker@peer.speedydelivery"
]
```
While the output of pears only contains the ` worker@peer.speedydelivery` peer. The anchor peer is not there
So when we are trying to get the anchor peer from that list. It does not exist.
basically we are failing somehere here:
```
- name: Get peer object list
set_fact:
peers: |
{{ peers | default([]) + [{'org':item.split('@')[1].split('.')[-1],
'name':item.split('@')[1] | replace('.','-'),
'role':item.split('@')[0]}] }}
with_items: "{{ peerls }}"
```
However the code here looks to be right to me... So I don't really understand why peers ends up missing anchor@peer.speedydelivery
Per the Ansible docs on appending lists... this looks right to me
Btw... @tongli thank you for taking the time to reply to me earlier today. :joy: Wasn't sure if this was active anymore.
@ChrisFowers by looking at your config, I think the problem is that you have same peer name
"worker@peer.arrayofblocks", "anchor@peer.arrayofblocks"
your anchor and worker peer are the same peer.
If you want to have 2 peers there, you should have it like the following
"worker@peer1st.arrayofblocks", "anchor@peer2nd.arrayofblocks"
notice that the word before @ is really the role of a peer.
the name is actually the words after @ sign.
what worker@peer.arrayofblocks actually means the peer's name is peer in org arrayofblocks and is a worker.
vs anchor@peer.arrayofblocks, means the peer's name is also peer in org arrayofblocks, but it is allocated as anchor.
with your configuration, I think it is no surprising that you only see one peer, instead of multiple ones.
please change that as I suggested, then you should be fine.
glad that you are actually getting things going.
@ChrisFowers ask questions here, and I will try my best to help.
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/21193/
Has joined the channel.
Hello, I am using cello to develop a block-chain application. Are there anyway to write application which use RESTFul or API to query/Invoke with Cello server? Thanks!
@baohua @hightall When you get a chance, please review this patch https://gerrit.hyperledger.org/r/21297 for this jira issue https://jira.hyperledger.org/browse/CE-351
Thanks.
@tongli Thank you for the assist. Will try to adjust and let you know how it goes. Again than you
Has joined the channel.
Has joined the channel.
@tongli +2
https://gerrit.hyperledger.org/r/#/c/21317/
https://gerrit.hyperledger.org/r/#/c/21317/ +2!
Thanks very much! @baohua @hightall
Has joined the channel.
Hi All , I am not able to access the user dashboard on the 8081 port , it says 401 unauthorized
static/js/dist/home/index.js
i am able to access the dashboard on the 8080 port
Has joined the channel.
Has joined the channel.
Hi all. I am using Cello Master service in s390 .When I run "setup.sh " , I got errors showing me "Error response from daemon: manifest for hyperledger/cello-baseimage:s390x-latest not found" . I checked the docker hub.There do not exist s390 images with version "latest" . Can we add this s390-latest image into official repo ?
Hi all. Does cello support fab1.1 now ?
When i an trying to apply a new chain , i am getting the error as {success: false, message: "System maintenance, please try again later!"}
Has joined the channel.
@bjwswang experimentally support
@kiranarshakota what's the log?
@baohua { code: 400,
13:16:21 0|index | data: {},
13:16:21 0|index | error: 'No available res for 5aeb0a5fef4a2f00134e146e',
13:16:21 0|index | status: 'FAIL' } } }
oh, do you have the log when creating the new chain?
strange, this seems not directly related
your post log is related to chain_apply, means no available chain in the pool when you wanna apply a new one
@baohua in the operator dashboard logs i see this error , Not find matched available cluster for 5ae70a79ef4a2f00051d578e
are you starting the service following our tutorial?
yes
http://cello.readthedocs.io/en/latest/
i am trying to use this , https://github.com/hyperledger/cello/blob/master/docs/tutorial.md
it seems the chain started error, do you have the log when create a new chain?
The steps i followed are , 1) Go to operator dashboard , 1a) Create a Host , 1b) Add a chain
i can see the status of chain as running
The i go to the user dashboard and do a Apply new Chain , then i am getting this error
@baohua do you want operator dashboard logs or user dashboard logs
oh, the service log. can be found by `make log`
btw, is ur host active?
yes it is active
@baohua , in the Active chains , the Status shows as running , but the health shows as FAIL
oh, that means the chain is started, but the health check is not passed.
after starting, cello will try to verify the blockchain service's health by checking whether the port is open to connect.
what port should i open
@baohua what all ports from the member node should i open
you need to check the worker nodes by `docker ps`
to see the port mapped
and make sure each blockchain container running successfully
@baohua , Thanks now i am able to Apply the chain
I didn't open the ports required
nice to know that
Has joined the channel.
@tongli let's carry the discussion of the test net over here
@tongli I pointed you to the CNCF cluster: https://www.cncf.io/community/infrastructure-lab/
@tongli I see someone filed a request: https://github.com/cncf/cluster/issues/62
@rjones, taitao did that.
that is fine. he is one of the cello maintainers.
@tongli that request was closed with a request for more information. I think there is clear synergy between Cello (especially with k8s) but the request was not very clear
@rjones, that is fine. we can address that. thx
@tongli if you use a successful request as a template, it might help you understand what CNCF and Packet want: https://github.com/cncf/cluster/issues/66
https://github.com/cncf/cluster/issues/65
ok. great. that is helpful. will follow that.
https://github.com/cncf/cluster/issues/55
OK.
Has left the channel.
@baohua @hightall I have made some changes to the Makefile so that the ansible agent container will be available on docker hub. Please review https://gerrit.hyperledger.org/r/21401 . thanks.
+2!
https://gerrit.hyperledger.org/r/21193
https://gerrit.hyperledger.org/r/#/q/project:cello
@baohua thanks for the +2. do we have a meeting tomorrow morning?
i'm having issue debugging why settinup cello via ansible-agent http://cello.readthedocs.io/en/latest/setup_worker_ansible_allinone/ fails ..
i'm having issue debugging why setting up cello via ansible-agent http://cello.readthedocs.io/en/latest/setup_worker_ansible_allinone/ fails..
i'm having issue debugging why setting up cello via ansible-agent http://cello.readthedocs.io/en/latest/setup_worker_ansible_allinone/ fails, I am trying to stand this up on a single node, with etcd/flanneld/skydns, I see DNS isn't resolving as such, operations are failing as such.
i'm having issue debugging why setting up cello via ansible-agent http://cello.readthedocs.io/en/latest/setup_worker_ansible_allinone/ fails, I am trying to stand this up on a single node, with etcd/flanneld/skydns, I see DNS isn't resolving as such, chaincode installation operations are failing as such.
@tongli yes, haitao already sent the mail notification, have u got it?
The meeting channel is changed to https://zoom.us/my/hyperledger.community
@baohua @hightall I missed the flag to actually post the image to docker hub. made the changes in this patch, please take a look. https://gerrit.hyperledger.org/r/21411
@tongli oh, yeah, +2
thanks.
https://wiki.hyperledger.org/projects/cello/faq
https://gerrit.hyperledger.org/r/#/c/21415/
Has joined the channel.
Hello, could someone please look into https://jira.hyperledger.org/browse/CE-350 ?
@hightall could you please merge this patch? https://gerrit.hyperledger.org/r/#/c/21193/
Has joined the channel.
@hightall when I used the `DEV=True make start` to start Cello, the operator-dashboard will come out this log
```Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 2295, in wsgi_app
response = self.handle_exception(e)
File "/usr/local/lib/python3.5/dist-packages/flask_restful/__init__.py", line 273, in error_router
return original_handler(e)
File "/usr/local/lib/python3.5/dist-packages/flask_restful/__init__.py", line 273, in error_router
return original_handler(e)
File "/usr/local/lib/python3.5/dist-packages/flask_restful/__init__.py", line 273, in error_router
return original_handler(e)
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1741, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.5/dist-packages/flask/_compat.py", line 35, in reraise
raise value
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1815, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.5/dist-packages/flask_restful/__init__.py", line 273, in error_router
return original_handler(e)
File "/usr/local/lib/python3.5/dist-packages/flask_restful/__init__.py", line 273, in error_router
return original_handler(e)
File "/usr/local/lib/python3.5/dist-packages/flask_restful/__init__.py", line 273, in error_router
return original_handler(e)
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1713, in handle_user_exception
return self.handle_http_exception(e)
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1644, in handle_http_exception
return handler(e)
File "/app/dashboard.py", line 75, in page_not_found
return render_template('404.html'), 404
File "/usr/local/lib/python3.5/dist-packages/flask/templating.py", line 134, in render_template
return _render(ctx.app.jinja_env.get_or_select_template(template_name_or_list),
File "/usr/local/lib/python3.5/dist-packages/jinja2/environment.py", line 869, in get_or_select_template
return self.get_template(template_name_or_list, parent, globals)
File "/usr/local/lib/python3.5/dist-packages/jinja2/environment.py", line 830, in get_template
return self._load_template(name, self.make_globals(globals))
File "/usr/local/lib/python3.5/dist-packages/jinja2/environment.py", line 804, in _load_template
template = self.loader.load(self, name, globals)
File "/usr/local/lib/python3.5/dist-packages/jinja2/loaders.py", line 113, in load
source, filename, uptodate = self.get_source(environment, name)
File "/usr/local/lib/python3.5/dist-packages/flask/templating.py", line 58, in get_source
return self._get_source_fast(environment, template)
File "/usr/local/lib/python3.5/dist-packages/flask/templating.py", line 86, in _get_source_fast
raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: 404.html
```
@hightall when I used the `DEV=True make start` to start Cello, the operator-dashboard will come out this error
```Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 2295, in wsgi_app
response = self.handle_exception(e)
File "/usr/local/lib/python3.5/dist-packages/flask_restful/__init__.py", line 273, in error_router
return original_handler(e)
File "/usr/local/lib/python3.5/dist-packages/flask_restful/__init__.py", line 273, in error_router
return original_handler(e)
File "/usr/local/lib/python3.5/dist-packages/flask_restful/__init__.py", line 273, in error_router
return original_handler(e)
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1741, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.5/dist-packages/flask/_compat.py", line 35, in reraise
raise value
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1815, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.5/dist-packages/flask_restful/__init__.py", line 273, in error_router
return original_handler(e)
File "/usr/local/lib/python3.5/dist-packages/flask_restful/__init__.py", line 273, in error_router
return original_handler(e)
File "/usr/local/lib/python3.5/dist-packages/flask_restful/__init__.py", line 273, in error_router
return original_handler(e)
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1713, in handle_user_exception
return self.handle_http_exception(e)
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1644, in handle_http_exception
return handler(e)
File "/app/dashboard.py", line 75, in page_not_found
return render_template('404.html'), 404
File "/usr/local/lib/python3.5/dist-packages/flask/templating.py", line 134, in render_template
return _render(ctx.app.jinja_env.get_or_select_template(template_name_or_list),
File "/usr/local/lib/python3.5/dist-packages/jinja2/environment.py", line 869, in get_or_select_template
return self.get_template(template_name_or_list, parent, globals)
File "/usr/local/lib/python3.5/dist-packages/jinja2/environment.py", line 830, in get_template
return self._load_template(name, self.make_globals(globals))
File "/usr/local/lib/python3.5/dist-packages/jinja2/environment.py", line 804, in _load_template
template = self.loader.load(self, name, globals)
File "/usr/local/lib/python3.5/dist-packages/jinja2/loaders.py", line 113, in load
source, filename, uptodate = self.get_source(environment, name)
File "/usr/local/lib/python3.5/dist-packages/flask/templating.py", line 58, in get_source
return self._get_source_fast(environment, template)
File "/usr/local/lib/python3.5/dist-packages/flask/templating.py", line 86, in _get_source_fast
raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: 404.html
```
@hightall when I used the `DEV=True make start` to start Cello, the operator-dashboard will come out with this error
```Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 2295, in wsgi_app
response = self.handle_exception(e)
File "/usr/local/lib/python3.5/dist-packages/flask_restful/__init__.py", line 273, in error_router
return original_handler(e)
File "/usr/local/lib/python3.5/dist-packages/flask_restful/__init__.py", line 273, in error_router
return original_handler(e)
File "/usr/local/lib/python3.5/dist-packages/flask_restful/__init__.py", line 273, in error_router
return original_handler(e)
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1741, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.5/dist-packages/flask/_compat.py", line 35, in reraise
raise value
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1815, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.5/dist-packages/flask_restful/__init__.py", line 273, in error_router
return original_handler(e)
File "/usr/local/lib/python3.5/dist-packages/flask_restful/__init__.py", line 273, in error_router
return original_handler(e)
File "/usr/local/lib/python3.5/dist-packages/flask_restful/__init__.py", line 273, in error_router
return original_handler(e)
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1713, in handle_user_exception
return self.handle_http_exception(e)
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1644, in handle_http_exception
return handler(e)
File "/app/dashboard.py", line 75, in page_not_found
return render_template('404.html'), 404
File "/usr/local/lib/python3.5/dist-packages/flask/templating.py", line 134, in render_template
return _render(ctx.app.jinja_env.get_or_select_template(template_name_or_list),
File "/usr/local/lib/python3.5/dist-packages/jinja2/environment.py", line 869, in get_or_select_template
return self.get_template(template_name_or_list, parent, globals)
File "/usr/local/lib/python3.5/dist-packages/jinja2/environment.py", line 830, in get_template
return self._load_template(name, self.make_globals(globals))
File "/usr/local/lib/python3.5/dist-packages/jinja2/environment.py", line 804, in _load_template
template = self.loader.load(self, name, globals)
File "/usr/local/lib/python3.5/dist-packages/jinja2/loaders.py", line 113, in load
source, filename, uptodate = self.get_source(environment, name)
File "/usr/local/lib/python3.5/dist-packages/flask/templating.py", line 58, in get_source
return self._get_source_fast(environment, template)
File "/usr/local/lib/python3.5/dist-packages/flask/templating.py", line 86, in _get_source_fast
raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: 404.html
```
and the broswer is showing nothing
and the browser is showing nothing
@Luke_Chen did use the newest code from master branch?
I did
```commit 8076ef079013ed1fe4121222156394ea810da470
Author: Haitao Yue
Has joined the channel.
I will test it tonight.
thanks
hi all
do we plan to support fabric 1.1?
@giniz we do
https://jira.hyperledger.org/projects/CE/issues/CE-305?filter=allopenissues
nice to hear that
Am new and trying to set up worker using docker
I did steps in tutorial but got this issue when adding host
> ERROR [agent.docker.docker_swarm] [docker_swarm.py:173 check_daemon()] - Exception in check_daemon Error while fetching server API version: HTTPConnectionPool(host='127.0.0.1', port=2375): Max retries exceeded with url: /version (Caused by NewConnectionError('
I can run curl http://127.0.0.1:2375/version
> {"Platform":{"Name":""},"Components":[{"Name":"Engine","Version":"17.12.0-ce","Details":{"ApiVersion":"1.35","Arch":"amd64","BuildTime":"2017-12-27T20:09:53.000000000+00:00","Experimental":"false","GitCommit":"c97c6d6","GoVersion":"go1.9.2","KernelVersion":"4.4.0-1052-aws","MinAPIVersion":"1.12","Os":"linux"}}],"Version":"17.12.0-ce","ApiVersion":"1.35","MinAPIVersion":"1.12","GitCommit":"c97c6d6","GoVersion":"go1.9.2","Os":"linux","Arch":"amd64","KernelVersion":"4.4.0-1052-aws","BuildTime":"2017-12-27T20:09:53.000000000+00:00"}
did I miss something ?
am using latest code, \did I miss something ?
am using latest code, did I miss something ?
it seems to be a network problem
it seems to be a network problem with my localnet
I can configure successfully on aws
hi
I have a concern about user and operator
as mentioned in docs the user dashboard mapped to port 8081, but why I saw container mapping to 8080
> CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
> 26161f57d465 hyperledger/cello-user-dashboard "bash -c 'cd /usr/ap…" 24 minutes ago Up 24 minutes 0.0.0.0:8081->8080/tcp cello-user-dashboard
> 69be25099052 hyperledger/cello-operator-dashboard "/bin/sh -c 'if [ \"$…" 24 minutes ago Up 24 minutes 0.0.0.0:8080->8080/tcp cello-operator-dashboard
and while I tried to login with user account on port 8080 it send me to operator dashboard
and while I tried to login with user account on port 8080 it sent me to operator dashboard
Clipboard - May 7, 2018 11:54 AM
8081(host port):8080(container port)
You should use 8081 port
yes, that is my concern
why default compose file maps user dashboard 8081:8080
is it on purpose or an issue
sorry I don't see any problem with this
what is your concerned?
I did't see any problem with this
it's my mistake
my local proxy restrict 8081
:joy: ok
:(
hi @Luke_Chen
yes?
do we need any special steps before apply chain
I got
> {"success":false,"message":"System maintenance, please try again later!"}
I create 1 host with 2 chains on operator board
any logs from operator dashboard?
> WARNING [modules.cluster] [cluster.py:452 apply_cluster()] - Not find matched available cluster for 5aefd8a7ef4a2f0009502515
a warning
Clipboard - May 7, 2018 12:43 PM
it's running, isn't it?
yes it running
yes it's running
Clipboard - May 7, 2018 12:46 PM
all fabric containers on worker host are up
great
but do you know why Apply chain action return that response
@giniz you may ask @hightall for assitance
@giniz you may ask @hightall for assistance
thanks @Luke_Chen
I found the problem
after enabling debug log and check source, my problem is the cluster healthcheck
cluster Running but healthcheck failed due to securitygroup on my aws instance
Oh I see, the ports are baned by rules of the iptables
Am I right?
yes :D
:joy: Thanks for the reporting, I will write it down to FAQ
:thumbsup:
I paste the link here, in case you have other instresting problems, https://wiki.hyperledger.org/projects/cello/faq
ok, thank @Luke_Chen
hi me again
hi, me again
when instantiate chaincode or invoke transaction, I got this error
> 2018-05-07T08:31:20.046218749Z 08:31:20 0|index | [2018-05-07 08:31:20.007] [ERROR] invoke-chaincode - Failed to send transaction and get notifications within the timeout period.
2018-05-07T08:31:20.046223121Z 08:31:20 0|index | [2018-05-07 08:31:20.007] [ERROR] invoke-chaincode - Failed to order the transaction. Error code: undefined
2018-05-07T08:31:20.046253758Z 08:31:20 0|index | TypeError: Cannot read property 'getConnectivityState' of undefined
2018-05-07T08:31:20.046258540Z 08:31:20 0|index | at ClientDuplexStream.
```2018-05-07T08:31:20.046218749Z 08:31:20 0|index | [2018-05-07 08:31:20.007] [ERROR] invoke-chaincode - Failed to send transaction and get notifications within the timeout period.
2018-05-07T08:31:20.046223121Z 08:31:20 0|index | [2018-05-07 08:31:20.007] [ERROR] invoke-chaincode - Failed to order the transaction. Error code: undefined
2018-05-07T08:31:20.046253758Z 08:31:20 0|index | TypeError: Cannot read property 'getConnectivityState' of undefined
2018-05-07T08:31:20.046258540Z 08:31:20 0|index | at ClientDuplexStream.
on the worker machine, I saw somes container for executing chaincode
>CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
>47fdba3538dc dev-peer1.org1.example.com-b01ccb391001490981a889299656c1a9-5af00cd13a9cc80060e0671a-v0-946c4c29edec32f089d9aa8d1f7070976e9e666dc4236afe5d0da1c6e7328654 "chaincode -peer.add…" 17 minutes ago Up 17 minutes dev-peer1.org1.example.com-b01ccb391001490981a889299656c1a9-5af00cd13a9cc80060e0671a-v0
>09ba36919cf0 dev-peer0.org1.example.com-b01ccb391001490981a889299656c1a9-5af00cd13a9cc80060e0671a-v0-65dff8b09c7897bd5cb44d60c59d18f8c2b8af3d86a53c9e99cbebf422c9d032 "chaincode -peer.add…" 17 minutes ago Up 17 minutes dev-peer0.org1.example.com-b01ccb391001490981a889299656c1a9-5af00cd13a9cc80060e0671a-v0
>18e2dc960459 dev-peer0.org1.example.com-b01ccb391001490981a889299656c1a9-5af00bea3a9cc80060e06719-v0-568acb5a06df18a895a46a40d6e8ac62b63ffdae2c09702e298d69632b664bfd "chaincode -peer.add…" 25 minutes ago Up 25 minutes dev-peer0.org1.example.com-b01ccb391001490981a889299656c1a9-5af00bea3a9cc80060e06719-v0
>e88930f8a8c3 dev-peer1.org1.example.com-b01ccb391001490981a889299656c1a9-5af00bea3a9cc80060e06719-v0-affdd5f4d049afc2883ef88b025d56b3aa157b4fab690604f1c042ab14a78272 "chaincode -peer.add…" 25 minutes ago Up 25 minutes dev-peer1.org1.example.com-b01ccb391001490981a889299656c1a9-5af00bea3a9cc80060e06719-v0
on the worker machine, I saw somes container for executing chaincode
```CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
47fdba3538dc dev-peer1.org1.example.com-b01ccb391001490981a889299656c1a9-5af00cd13a9cc80060e0671a-v0-946c4c29edec32f089d9aa8d1f7070976e9e666dc4236afe5d0da1c6e7328654 "chaincode -peer.add…" 17 minutes ago Up 17 minutes dev-peer1.org1.example.com-b01ccb391001490981a889299656c1a9-5af00cd13a9cc80060e0671a-v0
09ba36919cf0 dev-peer0.org1.example.com-b01ccb391001490981a889299656c1a9-5af00cd13a9cc80060e0671a-v0-65dff8b09c7897bd5cb44d60c59d18f8c2b8af3d86a53c9e99cbebf422c9d032 "chaincode -peer.add…" 17 minutes ago Up 17 minutes dev-peer0.org1.example.com-b01ccb391001490981a889299656c1a9-5af00cd13a9cc80060e0671a-v0
18e2dc960459 dev-peer0.org1.example.com-b01ccb391001490981a889299656c1a9-5af00bea3a9cc80060e06719-v0-568acb5a06df18a895a46a40d6e8ac62b63ffdae2c09702e298d69632b664bfd "chaincode -peer.add…" 25 minutes ago Up 25 minutes dev-peer0.org1.example.com-b01ccb391001490981a889299656c1a9-5af00bea3a9cc80060e06719-v0
e88930f8a8c3 dev-peer1.org1.example.com-b01ccb391001490981a889299656c1a9-5af00bea3a9cc80060e06719-v0-affdd5f4d049afc2883ef88b025d56b3aa157b4fab690604f1c042ab14a78272 "chaincode -peer.add…" 25 minutes ago Up 25 minutes dev-peer1.org1.example.com-b01ccb391001490981a889299656c1a9-5af00bea3a9cc80060e06719-v0
```
the log of these container containing such as:
> docker logs 09ba36919cf0
ex02 Init
Aval = 10, Bval = 20
ex02 Invoke
Aval = 0, Bval = 30
ex02 Invoke
Aval = -10, Bval = 40
the log of these container containing such as: docker logs 09ba36919cf0
ex02 Init
Aval = 10, Bval = 20
ex02 Invoke
Aval = 0, Bval = 30
ex02 Invoke
Aval = -10, Bval = 40
the log of these container containing such as:
```docker logs 09ba36919cf0
ex02 Init
Aval = 10, Bval = 20
ex02 Invoke
Aval = 0, Bval = 30
ex02 Invoke
Aval = -10, Bval = 40
```
I mean at the chain, the action still got invoked and value changed but at UI error was shown
Has joined the channel.
@Luke_Chen https://gerrit.hyperledger.org/r/21493 this patch fix dev mode error
https://gerrit.hyperledger.org/r/#/c/21493/
merged!
@baohua, @hightall WE have two directories one is called cello/docker and one is called cello/dockerhub. why do we have two and what are the purposes of each? thanks. Ry told me that the new ansible agent docker file should be in the dockerhub directory.
maybe this need ask @baohua , dockerhub is used for images auto build on docker hub, docker dir is only for docker image build locally, I think it's redundancy. @tongli
@hightall I will look at the Makefile and see if we can remove this duplicates. it does not make sense to me, I may have missed some thing important.
@tongli ok
@tongli the reason is that the original docker dir is used for support multiple archs like ppc and s390. while Docker Hub only support x86 now. If you wanna dockerhub supports auto-build, then put under dockerhub as ry suggested.
we can not combine them together?
@baohua I am not seeing why there should be duplicates.
no, we cannot. as ppc and s390 have very different requirement from Dockerhub, hence we haven't find easier solution for that yet.
@tongli we need docker build locally, because dockerhub build images from git code, but something you need local file, such as requirements.txt locally, so need use make docker instead of make dockerhub
oh yeah
if we stop the support on ppc and s390, then we can only have one dir. but i think we can keep them for now.
hmm. let me work with ry on this.
@tongli Hoping you can help me on this , trying to deploy fabric on K8 via `ansible-playbook -i run/runhosts -e "mode=apply env=vb deploy_type=k8s" setupfabric.yml`
```TASK [Gathering Facts] **************************************************************************************************************************************************************
fatal: [xx.xxx.xx.xx]: FAILED! => {"failed": true, "msg": "the field 'remote_user' has an invalid value, which appears to include a variable that is undefined. The error was: 'fabric' is undefined"}
to retry, use: --limit @/home/ibmadmin/gopath/src/github.com/hyperledger/cello/src/agent/ansible/setupfabric.retry
PLAY RECAP **************************************************************************************************************************************************************************
xxx.xx.xx.xx : ok=0 changed=0 unreachable=0 failed=1
cloud ```
@tongli Hoping you can help me on this , trying to deploy fabric on K8 via `ansible-playbook -i run/runhosts -e "mode=apply env=vb deploy_type=k8s" setupfabric.yml`
```TASK [Gathering Facts] **************************************************************************************************************************************************************
fatal: [xx.xxx.xx.xx]: FAILED! => {"failed": true, "msg": "the field 'remote_user' has an invalid value, which appears to include a variable that is undefined. The error was: 'fabric' is undefined"}
to retry, use: --limit @/home/ibmadmin/gopath/src/github.com/hyperledger/cello/src/agent/ansible/setupfabric.retry
```
Can you point me on how/what has to be tweaked ?
@harsha are you doing this in vb or aws?
will need to look at your configuration files.
I am doing this in vb.
oh. ok. in VB, it is a bit tricky, because VB has no many ways for your to control the VM.
in your VB env, you will need to have a vm which has docker and the user setup before you run the script.
it will be also helpful to look at your vb.yml file.
remove your password in it if you have it.
Tried clearing off `ssh_pass` under `vars/vb.yml` file, but I end up with same error as above.
Enabling additional verbose logging with `-vvv` doesn't give anything yield as such.
` "msg": "the field 'remote_user' has an invalid value, which appears to include a variable that is undefined. The error was: 'fabric' is undefined"
`
Enabling additional verbose logging with `-vvv` doesn't give anything yield as such.
``` "msg": "the field 'remote_user' has an invalid value, which appears to include a variable that is undefined. The error was: 'fabric' is undefined"
```
Enabling additional verbose logging with `-vvv` doesn't give anything yield as such.
```"msg": "the field 'remote_user' has an invalid value, which appears to include a variable that is undefined. The error was: 'fabric' is undefined" ```
Enabling additional verbose logging with `-vvv` doesn't yield anything info. as such.
```"msg": "the field 'remote_user' has an invalid value, which appears to include a variable that is undefined. The error was: 'fabric' is undefined" ```
Has joined the channel.
hello
can anyone help me on my case?
I tried deploy master and work on 2 aws instances with allow all traffics between
I deployed chaincode in fabric-sample package to test
installation successfully but when instantiating or invoking actions I got this error on master
```
cello-user-dashboard | 01:48:19 0|index | [2018-05-08 01:48:19.308] [ERROR] instantiate-chaincode - Failed to send instantiate transaction and get notifications within the timeout period. TypeError: Cannot read property 'getConnectivityState' of undefined
cello-user-dashboard | 01:48:19 0|index | [2018-05-08 01:48:19.309] [ERROR] instantiate-chaincode - Failed to order the transaction. Error code: undefined
cello-user-dashboard | 01:48:19 0|index | TypeError: Cannot read property 'getConnectivityState' of undefined
cello-user-dashboard | 01:48:19 0|index | at ClientDuplexStream.
@giniz which version are u using? can u elaborate the operation steps?
@baohua, am using the lates source from repo
first I setup and run master on one instance, then I run steps to setup worker using docker on another instance and configure network to allow all traffics between them
I create a host with that worker, add new chain with SOLO consensus via operator dashboard
then I apply chain and install sample chain code to that chain via user dashboard
the error happens when instantiating or invoking actions that create transaction
Query method works well
as I see in log the action was invoked successfully in peer node, but timeout happen at orderer
as I see in log the action was invoked successfully in peer node, but timeout happened at orderer
user dashboard log
```
cello-user-dashboard | 02:24:45 0|index | [2018-05-08 02:24:45.892] [INFO] invoke-chaincode - transaction proposal was good
cello-user-dashboard | 02:24:45 0|index | [2018-05-08 02:24:45.892] [DEBUG] invoke-chaincode - Successfully sent Proposal and received ProposalResponse: Status - 200, message - "OK", metadata - "", endorsement signature: 0D =p��`@E�J�o�:��zT�L[U@�9� /��W+���8Y�CL�%a�*�����>�w��
.....
cello-user-dashboard | 02:24:45 0|index | [2018-05-08 02:24:45.949] [ERROR] invoke-chaincode - Failed to send transaction and get notifications within the timeout period.
cello-user-dashboard | 02:24:45 0|index | [2018-05-08 02:24:45.953] [ERROR] invoke-chaincode - Failed to order the transaction. Error code: undefined
cello-user-dashboard | 02:24:45 0|index | TypeError: Cannot read property 'getConnectivityState' of undefined
cello-user-dashboard | 02:24:45 0|index | at ClientDuplexStream.
orderer log
```
2018-05-08T02:24:45.990000152Z 2018-05-08 02:24:45.989 UTC [orderer/common/blockcutter] Ordered -> DEBU 1359 Enqueuing message into batch
2018-05-08T02:24:46.050487815Z 2018-05-08 02:24:46.050 UTC [grpc] Printf -> DEBU 135a transport: http2Server.HandleStreams failed to read frame: read tcp 172.21.0.2:7050->10.2.0.225:51304: read: connection reset by peer
2018-05-08T02:24:46.050786667Z 2018-05-08 02:24:46.050 UTC [orderer/common/broadcast] Handle -> WARN 135b Error reading from stream: rpc error: code = Canceled desc = context canceled
2018-05-08T02:24:46.050799213Z 2018-05-08 02:24:46.050 UTC [orderer/main] func1 -> DEBU 135c Closing Broadcast stream
2018-05-08T02:24:47.990518981Z 2018-05-08 02:24:47.990 UTC [orderer/solo] main -> DEBU 135d Batch timer expired, creating block
2018-05-08T02:24:47.990556864Z 2018-05-08 02:24:47.990 UTC [fsblkstorage] retrieveBlockByNumber -> DEBU 135e retrieveBlockByNumber() - blockNum = [4]
```
from the UI of hyperledger explorer port 8150, transaction and block was produced, but the response to user dashboard UI show error
from the UI of hyperledger explorer port 8150, transaction and block was produced, but the response to user dashboard UI port 8081 show error
yes, seems there's problem when connecting to the orderer.
can u test in ur container (172.21.0.2), that can connect to orderer address (10.2.0.225) directly?
can u test in ur container (172.21.0.2), that can connect to address (10.2.0.225) directly?
@baohua, IP 172.21.0.2 belong to 5f57ad230f004445b7f802654045f9df_peer0_org1 container and IP 10.2.0.225 is cello master
yes, the error happens when peer connects to orderer service (10.2.0.225:51304)
the traffics from cello master can reach 172.21.0.2 successfully
because the Query request can be invoked normally
but the Invoke request fails
invoke will involve the orderer node, while query doesn't
yes, anything I need to check here?
btw, can u confirm 172.21.0.2:7050 is peer, not orderer? as peer node should listen on port 7051 by default
while orderer uses the port 7050
```
ubuntu@ip-10-2-0-20:~$ docker ps | grep 7050
dabf92d05548 hyperledger/fabric-peer:1.0.5 "peer node start" About an hour ago Up About an hour 0.0.0.0:7050->7051/tcp, 0.0.0.0:7150->7053/tcp 5f57ad230f004445b7f802654045f9df_peer0_org1
e28556ada415 hyperledger/fabric-orderer:1.0.5 "orderer" About an hour ago Up About an hour 0.0.0.0:8050->7050/tcp 5f57ad230f004445b7f802654045f9df_orderer
```
sorry, my mistake it's the IP of orderer
ok, then that makes sense. there must be some thing wrong when client tried to connect to the orderer. can u test manually using the client?
and be noticed that use the 1.0.5 sdk, not 1.1
@baohua, I think this issue may be related to this post https://stackoverflow.com/questions/50134379/error-invoking-chaincode-using-node-js-sdk-typeerror-cannot-read-property-get
ok, can u help create some jira issue, we will track the problem.
@hightall we should take care of this node sdk version problem.
@baohua seems that it's because grpc newest version
ok, we need to fix version to be stable
ok, we need fixed version to be stable
ok
hi @baohua, when will cello provide support for fabric 1.1?
the current plan is for v0.9 (around end of this June)
thanks
hi @hightall, do we use the Dockerfile in dockerhub/latest/user-dashboard to build image for user-dashboard?
@giniz yes
I just created a local image without modifying Dockerfile or package.json in that dir
but when run I got this error
```
2018-05-08T07:34:41.544833719Z 07:34:41 0|index | [2018-05-08 07:34:41.543] [INFO] index.js - Server started>>>
2018-05-08T07:34:41.695846069Z 07:34:41 0|index | [2018-05-08 07:34:41.694] [INFO] index.js - Connection to bc_dashboard established.
2018-05-08T07:35:31.551808280Z 07:35:31 0|index | Error: Cannot find module './api.js'
2018-05-08T07:35:31.583401473Z 07:35:31 0|index | at Function.Module._resolveFilename (module.js:538:15)
2018-05-08T07:35:31.589718738Z 07:35:31 0|index | at Function.Module._load (module.js:468:25)
2018-05-08T07:35:31.589742576Z 07:35:31 0|index | at Module.require (module.js:587:17)
2018-05-08T07:35:31.589747261Z 07:35:31 0|index | at require (internal/module.js:11:18)
```
do I need to rebuild image for other services or I miss something here?
because as I check the latest image of user-dashboard (3 days ago) in dockerhub containing the issue of grpc (1.11.3) with fabric-client
and a fix for this has been provided recently, so I need to create a local image of user-dashboard containing the fix in fabric-client
@hightall do you have any suggestion for mycase?
@giniz you can use make docker build local image, oh, looks like there no Dockerfile.in in docker dir, or can you move the dockerfile into the docker dir and try to build user-dashboard locally?
am using the dockerfile at https://github.com/hyperledger/cello/tree/master/dockerhub/latest/user-dashboard
do you mean I need to use another file or move this file to another dir?
@hightall I tried copy Dockerfile in above path to cello/user-dashboard and built image here but still got error
I do have two serves running the BYFN example chaincode on different servers. Can I use Cello to spin up a blockchain ntwork?
Anyone know how is it possible to modify the READ/WRITE access to a peer ? I saw that there are two possibility one on the Chaincode, the other one on the config file. can anyone help me to find the right file?
Is there a link with this ? :
`You can also see that the endorsement policy has been modified to -P "OR ('Org1MSP.peer','Org2MSP.peer','Org3MSP.peer')". `
The idea is to limit the access as for example in the sample : `parameter 1 : A` -> can invoke or query but ` parameter 1: B` can only query
Anyone know how is it possible to modify the READ/WRITE access to a peer ? I saw that there are two possibility one on the Chaincode, the other one on the config file. can anyone help me to find the right file?
Is there a link with this ? :
`You can also see that the endorsement policy has been modified to -P "OR ('Org1MSP.peer','Org2MSP.peer')". `
The idea is to limit the access as for example in the sample : `parameter 1 : A` -> can invoke or query but ` parameter 1: B` can only query
Has joined the channel.
@baohua @hightall please see this dockerfile to post the ansible agent docker image onto docker hub. https://gerrit.hyperledger.org/r/21533
it seems to me that DockerHub can not really produce a source code tree of a project, you will have to do everything in your Dockerfile. that is probably the most of the reason why we have two sets of docker files.
Seems to me that won't change for awhile.
Hey guys! I got a quick question - what's the right way to deploy chaincode past Ansible deployment?(I got a simple network on AWS with kafka etc) Is that what cello dashboard is for or is there another way? Sorry if it's a stupid question =/
@tongli +2
Thanks.
now we should have a container image to drive ansible deployment a bit easier.
https://gerrit.hyperledger.org/r/#/c/21541/
+2!
Has joined the channel.
"I'm pretty stumped. Does Cello need to be on the same machine and the swarm manager? I ask because I keep getting this when I attempt to create a host => https://gist.github.com/tdespenza/5ef86bac33a60e7958bf5e9045d28496."
@tdespenza not really, but the cello master node service needs to be able to access the swarm manager API.
hello
today, am trying to deploy a chain with kafka consensus and deploying contract
however I got an issue when instantiating chaincode on worker machine, the container for chaincode was created but cannot start because the network for that container not exist
```
2018-05-09T06:33:28.857078951Z 06:33:28 0|index | error: [client-utils.js]: sendPeersProposal - Promise is rejected: Error: 2 UNKNOWN: Error starting container: API error (404): {"message":"network ef1a625590c34deab55318cebf8147ca_default not found"}
2018-05-09T06:33:28.878644439Z 06:33:28 0|index | at new createStatusError (/node_modules/grpc/src/client.js:64:15)
2018-05-09T06:33:28.900450076Z 06:33:28 0|index | at /node_modules/grpc/src/client.js:583:15
2018-05-09T06:33:28.975728147Z 06:33:28 0|index | error: [client-utils.js]: sendPeersProposal - Promise is rejected: Error: 2 UNKNOWN: Error starting container: API error (404): {"message":"network ef1a625590c34deab55318cebf8147ca_default not found"}
2018-05-09T06:33:28.975763898Z 06:33:28 0|index | at new createStatusError (/node_modules/grpc/src/client.js:64:15)
2018-05-09T06:33:28.975769505Z 06:33:28 0|index | at /node_modules/grpc/src/client.js:583:15
2018-05-09T06:33:28.975773289Z 06:33:28 0|index | [2018-05-09 06:33:28.973] [ERROR] instantiate-chaincode - instantiate proposal was bad
2018-05-09T06:33:28.975777305Z 06:33:28 0|index | [2018-05-09 06:33:28.974] [ERROR] instantiate-chaincode - instantiate proposal was bad
2018-05-09T06:33:28.975781149Z 06:33:28 0|index | [2018-05-09 06:33:28.974] [ERROR] instantiate-chaincode - Failed to send instantiate Proposal or receive valid response. Response null or status is not 200. exiting...
2018-05-09T06:33:28.975785234Z 06:33:28 0|index | [2018-05-09 06:33:28.974] [ERROR] instantiate-chaincode - Failed to order the transaction. Error code: undefined
```
@baohua have you seen any similar issue like my case?
which underly platform are u using? docker or swarm?
am using docker
can u run `docker network ls` in the worker node (added host)
and `docker ps |grep ef1a625590c34deab55318cebf8147ca`
to see if the network starts successfully.
the network has not been created
```
ubuntu@ip-10-2-0-20:~$ docker network ls
NETWORK ID NAME DRIVER SCOPE
33ad6f3b1e64 bridge bridge local
b37ab36d0ebf cello_net_kafka bridge local
d75566a3bc8e docker_default bridge local
80ade1f4baed host host local
aefe0680a68c none null local
ubuntu@ip-10-2-0-20:~$ docker ps | grep ef1a625590c34deab55318cebf8147ca_default
```
Hi all !
Any idea about this
`
10:23:54 0|index | [2018-05-09 10:23:54.103] [ERROR] Join-Channel - TypeError: Cannot read property 'getConnectivityState' of undefined
10:23:54 0|index | at EventHub._checkConnection (/node_modules/fabric-client/lib/EventHub.js:479:39)
10:23:54 0|index | at EventHub.registerBlockEvent (/node_modules/fabric-client/lib/EventHub.js:658:8)
10:23:54 0|index | at Promise (/opt/data/users/admin/chains/5af2cc368231490035b4bfee/lib/join-channel.js:83:8)
10:23:54 0|index | at new Promise (
`
10:23:54 0|index | [2018-05-09 10:23:54.103] [ERROR] Join-Channel - TypeError: Cannot read property 'getConnectivityState' of undefined
10:23:54 0|index | at EventHub._checkConnection (/node_modules/fabric-client/lib/EventHub.js:479:39)
10:23:54 0|index | at EventHub.registerBlockEvent (/node_modules/fabric-client/lib/EventHub.js:658:8)
10:23:54 0|index | at Promise (/opt/data/users/admin/chains/5af2cc368231490035b4bfee/lib/join-channel.js:83:8)
10:23:54 0|index | at new Promise (
`10:23:54 0|index | [2018-05-09 10:23:54.103] [ERROR] Join-Channel - TypeError: Cannot read property 'getConnectivityState' of undefined
10:23:54 0|index | at EventHub._checkConnection (/node_modules/fabric-client/lib/EventHub.js:479:39)
10:23:54 0|index | at EventHub.registerBlockEvent (/node_modules/fabric-client/lib/EventHub.js:658:8)
10:23:54 0|index | at Promise (/opt/data/users/admin/chains/5af2cc368231490035b4bfee/lib/join-channel.js:83:8)
10:23:54 0|index | at new Promise (
`10:23:54 0|index | [2018-05-09 10:23:54.103] [ERROR] Join-Channel - TypeError: Cannot read property 'getConnectivityState' of undefined
10:23:54 0|index | at EventHub._checkConnection (/node_modules/fabric-client/lib/EventHub.js:479:39)
10:23:54 0|index | at EventHub.registerBlockEvent (/node_modules/fabric-client/lib/EventHub.js:658:8)
10:23:54 0|index | at Promise (/opt/data/users/admin/chains/5af2cc368231490035b4bfee/lib/join-channel.js:83:8)
10:23:54 0|index | at new Promise (
` 10:23:54 0|index | [2018-05-09 10:23:54.103] [ERROR] Join-Channel - TypeError: Cannot read property 'getConnectivityState' of undefined
10:23:54 0|index | at EventHub._checkConnection (/node_modules/fabric-client/lib/EventHub.js:479:39)
10:23:54 0|index | at EventHub.registerBlockEvent (/node_modules/fabric-client/lib/EventHub.js:658:8)
10:23:54 0|index | at Promise (/opt/data/users/admin/chains/5af2cc368231490035b4bfee/lib/join-channel.js:83:8)
10:23:54 0|index | at new Promise (
Hi all, ive setup a fabric network using cello ansible agent. is there any look at the peer proccesses etc. i just see some dokcer containers in the aws instances where network is setup
Hi all, ive setup a fabric network using cello ansible agent. is there any way to look at the peer proccesses etc. i just see some dokcer containers in the aws instances where network is setup
Hitachi@123
@here, cello developers, currently at ci we do very little for our build. I am planning to add at least the container build target as part of the ci gating process. I tried that process which takes about 4 minuets with all the docker containers getting built. I think that is an acceptable time. If no objections, I will make that change in a patch set.
@baohua @hightall The jira issue for that was created here https://jira.hyperledger.org/browse/CE-356. and the patch is here https://gerrit.hyperledger.org/r/21615. please review and comment
looks fine
https://gerrit.hyperledger.org/r/#/c/21613/
@baohua @hightall after added the docker target, it indeed timedout, on my own machine , it only took 4 minutes to get everything done. (without any base images).
but at the gate, it obviously took more than 15 minutes, the build failed.
I am debating if it is a good thing to build the docker containers for every patch set.
but I am thinking we should do some check at least, how about randomly pick one container to built? vs pick a specific container to build vs build all containers?
let me know what you think
I think this is one task that we discussed at the last meeting, we can talk about it at this week's meeting.
Has joined the channel.
Hello, some newbie doubts, in case I configure "docker host" Cello will deploy all containers only in that host (it will not use the other hosts available on the list to balance the amount of containers)?
I added several docker hosts as independent workers but I'm seeing that when adding chains, all containers go to the same host
https://gerrit.hyperledger.org/r/#/c/21617/
Has joined the channel.
+2
@hightall the tsc update for cello is at May 24 night, would you like to present?
I created a ticket for kafka explorer port duplication (8150) when using docker worker https://jira.hyperledger.org/browse/CE-358
thanks @giniz , added potential solution design
https://gerrit.hyperledger.org/r/#/c/21661/
@giniz this is the patchset
thanks @baohua
I tried to deploy fabric using ansible
and it created three vm's and seems it has installed fabric
how do i create channel, deploy chaincode onto this network?
https://gerrit.hyperledger.org/r/#/c/21665/ --- @baohua @tongli @hightall Request for review
added comments
https://gerrit.hyperledger.org/r/#/c/21671/
https://gerrit.hyperledger.org/r/21673 - @baohua @tongli @hightall request for review
+2!
@here Dear all, we will have project meeting at 8:30PM--9:00PM (UTC+8) this Friday evening , welcome to join, and please help update the agenda at: https://wiki.hyperledger.org/projects/cello/meetings/20180511. Thanks!
:thumbsup:
Has left the channel.
Has joined the channel.
@tongli Hi, trying to deploy fabric on K8 via `ansible-playbook -i run/runhosts -e "mode=apply env=aws deploy_type=k8s" setupfabric.yml`
fatal: [xx.xx.xxx.xx]: FAILED! => {"changed": false, "cmd": "/opt/gopath/myfabric/src/github.com/hyperledger/fabric/build/bin/cryptogen generate --config crypto-config.yml
--output keyfiles", "msg": "[Errno 2] No such file or directory", "rc": 2
Hi, trying to deploy fabric on K8 via `ansible-playbook -i run/runhosts -e "mode=apply env=aws deploy_type=k8s" setupfabric.yml`
fatal: [xx.xx.xxx.xx]: FAILED! => {"changed": false, "cmd": "/opt/gopath/myfabric/src/github.com/hyperledger/fabric/build/bin/cryptogen generate --config crypto-config.yml
--output keyfiles", "msg": "[Errno 2] No such file or directory", "rc": 2
Hi, trying to deploy fabric on K8 via `ansible-playbook -i run/runhosts -e "mode=apply env=aws deploy_type=k8s" setupfabric.yml`
TASK [Create the certificate files] ******************************************************************************************************************************************************************
fatal: [xx.xx.xxx.xx]: FAILED! => {"changed": false, "cmd": "/opt/gopath/myfabric/src/github.com/hyperledger/fabric/build/bin/cryptogen generate --config crypto-config.yml
--output keyfiles", "msg": "[Errno 2] No such file or directory", "rc": 2
Hi, trying to deploy fabric on K8 via `ansible-playbook -i run/runhosts -e "mode=apply env=aws deploy_type=k8s" setupfabric.yml`
TASK [Create the certificate files] ******************************************************************************************************************************************************************
fatal: [xx.xx.xxx.xx]: FAILED! => {"changed": false, "cmd": "/opt/gopath/myfabric/src/github.com/hyperledger/fabric/build/bin/cryptogen generate --config crypto-config.yml
--output keyfiles", "msg": "[Errno 2] No such file or directory", "rc": 2
Has joined the channel.
cello-user-dashboard | 09:03:49 0|index | [2018-05-11 09:03:49.013] [DEBUG] Helper - org1 peer2 { requests: 'grpcs://192.168.1.109:7250',
cello-user-dashboard | 09:03:49 0|index | events: 'grpcs://192.168.1.109:7350',
cello-user-dashboard | 09:03:49 0|index | 'server-hostname': 'peer1.org1.example.com',
cello-user-dashboard | 09:03:49 0|index | tls_cacerts: '/usr/app/src/src/config-template/fabric/4peers/fixtures/channel/crypto-config/peerOrganizations/org1.example.com/peers/peer1.org1.example.com/tls/ca.crt' }
cello-user-dashboard | 09:03:49 0|index | [2018-05-11 09:03:49.030] [DEBUG] Helper - [crypto_ecdsa_aes]: ecdsa signature: Signature {
cello-user-dashboard | 09:03:49 0|index | r:
@luckydogchina we've fixed a version mismatching problem with the sdk client, can u try with the latest code?
ok, thanks
:grinning:
@Gvedam yeah, make sure that you configure it right in your vars/bc1st.yml file
@Gvedam you may have this. https://github.com/hyperledger/cello/blob/master/src/agent/ansible/vars/bc1st.yml#L85 being ""
it should point to a place where the cryptogen can be downloaded.
https://nexus.hyperledger.org/content/repositories/releases/org/hyperledger/fabric/hyperledger-fabric-build/linux-amd64-ff6f6dbffed57efb9f7d8886b4a47949ce2d4396/hyperledger-fabric-build-linux-amd64-ff6f6dbffed57efb9f7d8886b4a47949ce2d4396.tar.gz
@Gvedam you can set bin to that value. which is the 1.1.0 binary.
Has joined the channel.
Has left the channel.
So I switched to kubernetes and when I set up a host I get this
`[2018-05-12 02:09:19,495] ERROR [modules.host] [host.py:177 create()] - Failed to setup Host kubernetes
[2018-05-12 02:09:19,495] ERROR [modules.host] [host.py:178 create()] - 'NoneType' object has no attribute 'get'
[2018-05-12 02:09:19,983] INFO [resources.host_api] [host_api.py:37 hosts_list()] - /hosts_list method=GET
[2018-05-12 02:09:19,986] INFO [modules.host] [host.py:248 list()] - filter data {}`
Any reason why this could be happening?
So I switched to kubernetes and when I set up a host I get this
[2018-05-12 02:09:19,495] ERROR [modules.host] [host.py:177 create()] - Failed to setup Host kubernetes
[2018-05-12 02:09:19,495] ERROR [modules.host] [host.py:178 create()] - 'NoneType' object has no attribute 'get'
[2018-05-12 02:09:19,983] INFO [resources.host_api] [host_api.py:37 hosts_list()] - /hosts_list method=GET
[2018-05-12 02:09:19,986] INFO [modules.host] [host.py:248 list()] - filter data {}
Any reason why this could be happening?
@tdespenza kubernetes type is not ready yet.
@tdespenza kubernetes type is not readly yet.
@tdespenza kubernetes type is not ready yet.
Ooh. Didn't know that. Thanks.
Know of any timeline?
@tdespenza It should be ready within this month
Cool. Thanks.
[ ](https://chat.hyperledger.org/channel/cello?msg=dMpT9Z4uqzKwvAjkY) @tongli it works!! the network is setup. But how do i perform any transactions on it?
when im trying to open the dashboard on 32334 its not getting connected and when i curl i get the following error
ubuntu@fabric002:~$ curl https://localhost:32334
curl: (60) server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
More details here: http://curl.haxx.se/docs/sslcerts.html
[ ](https://chat.hyperledger.org/channel/cello?msg=dMpT9Z4uqzKwvAjkY) @tongli it works!! the network is setup. But how do i perform any transactions on it?
when im trying to open the dashboard on 32334 its not getting connected and when i curl i get the following error
ubuntu@fabric002:~$ curl https://localhost:32334
curl: (60) server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
More details here: http://curl.haxx.se/docs/sslcerts.html
@Gvedam if you use ansible agent, the fabric network is not integrated with cello dashboard. it gives you a multi node fabric network and created a channel.
joined all the peers to that channel. at that point, you should start writing your own apps.
or deploy your own apps to the network.
if you are using docker (not k8s), the composer should have been setup for you, you can use composer to see things.
2018-05-14_21-52-53.mp4
@Luke_Chen is this ui right?
https://gerrit.hyperledger.org/r/#/c/21759/
Has joined the channel.
@hightall That is very good, thanks.
+2!
@baohua, @hightall, please review this jira issue and patch.
https://jira.hyperledger.org/browse/CE-363
https://gerrit.hyperledger.org/r/#/c/21815/
this is a continued effort to allow ansible agent to deploy fabric onto existing k8s cluster which was not setup by using ansible agent.
User User_1 added by tgehrke.
Has joined the channel.
Has joined the channel.
Hi there, I'm trying to set up a master node and worker node network using docker as shown in the docs, but when trying to create a worker node, I get this error from cello-operator-dashboard: `cello-operator-dashboard | [2018-05-15 19:02:20,310] ERROR [agent.docker.docker_swarm] [docker_swarm.py:173 check_daemon()] - Exception in check_daemon Error while fetching server API version: HTTPConnectionPool(host='127.0.0.1', port=2375): Max retries exceeded with url: /version (Caused by NewConnectionError('
Hi there, I'm trying to set up a master node and worker node network using docker as shown in the docs, but when trying to create a worker node, I get this error from cello-operator-dashboard: `cello-operator-dashboard | [2018-05-15 19:02:20,310] ERROR [agent.docker.docker_swarm] [docker_swarm.py:173 check_daemon()] - Exception in check_daemon Error while fetching server API version: HTTPConnectionPool(host='127.0.0.1', port=2375): Max retries exceeded with url: /version (Caused by NewConnectionError('
Hi there, I'm trying to set up a master node and worker node network using docker as shown in the docs, but when trying to create a worker node, I get this error from cello-operator-dashboard:
`cello-operator-dashboard | [2018-05-15 19:02:20,310] ERROR [agent.docker.docker_swarm] [docker_swarm.py:173 check_daemon()] - Exception in check_daemon Error while fetching server API version: HTTPConnectionPool(host='127.0.0.1', port=2375): Max retries exceeded with url: /version (Caused by NewConnectionError('
Hi there, I'm trying to set up a master node and worker node network using docker as shown in the docs, but when trying to add the worker node as a host, I get this error from cello-operator-dashboard:
`cello-operator-dashboard | [2018-05-15 19:02:20,310] ERROR [agent.docker.docker_swarm] [docker_swarm.py:173 check_daemon()] - Exception in check_daemon Error while fetching server API version: HTTPConnectionPool(host='127.0.0.1', port=2375): Max retries exceeded with url: /version (Caused by NewConnectionError('
Has left the channel.
I'm running both on MacOSX, and followed the steps for the docker daemon as well
I would appreciate any help!
Has joined the channel.
@tongli +2
@hightall thanks
Has joined the channel.
Has joined the channel.
@JordanDearsley why the worker_api is tcp://127.0.0.1:2375? that means the local addr inside the container.
Clipboard - May 17, 2018 2:15 PM
Clipboard - May 17, 2018 2:15 PM
Clipboard - May 17, 2018 2:15 PM
Clipboard - May 17, 2018 2:15 PM
Clipboard - May 17, 2018 2:15 PM
Clipboard - May 17, 2018 2:15 PM
Clipboard - May 17, 2018 2:15 PM
Clipboard - May 17, 2018 2:15 PM
[ ](https://chat.hyperledger.org/channel/cello?msg=L69pqmzegfwRgqCXE) @tongli I used docker compose and have setup a network. however inside an aws instance , i see no ports configured for peer.
the orderer is on 7050, CA on 7054 but none for Peer. is it some problem with the setup?
[ ](https://chat.hyperledger.org/channel/cello?msg=3vhEtqBhwFrkMTE4T) @baohua Thank you ! Fixed
Hey sorry! One more question. Is it possible to deploy a .bna via composer playground to a cello network? If so, how?
Hey sorry! One more question. Is it possible to deploy a BNA via composer playground to a cello network? If so, how?
What IP's would I put for the peers, orderers, etc?
Hey sorry! One more question. Is it possible to deploy a BNA via composer playground to a cello network? If so, how?
What IP's would I put for the peers, orderers, etc?
@JordanDearsley if you use cello ansible agent, it does that already.
it will set you up with composer playground and hook it up with the fabric network it creates.
Hi, I'm having timeout errors on container creation, currently using Swarm but had the same issue using only Docker, error is: ERROR: for ab78989a2eeb4380b1a3a13c8f3a7084_cli HTTPConnectionPool(host='192.168.200.21', port=2375) : Read timed out. (read timeout=60)
I've been trying to check where to raise the timeout to a superior number but couldn't find the exact place where to configure this
[ ](https://chat.hyperledger.org/channel/cello?msg=C4W5DTGsZH84oswQ9) @tongli Okay thanks! I'll give it a go
just to clarify, containers are created until the timeout reaches and at that time, the process stops and it just remove all the created containers
Timeout error
@baohua @hightall I added the persistent volume to the k8s cluster that ansible agent sets up. Here are the jira issue and patch. please review. Thanks. https://jira.hyperledger.org/browse/CE-366 https://gerrit.hyperledger.org/r/21959
+2!
@baohua Thanks a lot.
@tongli HI when I am launching the fabric network through cello using Ansible, the ports of my peer container are not getting mapped properly which might be leading to connect failed response.
@tongli HI when I am launching the fabric network through cello using Ansible, the ports of my peer container are not getting mapped properly which might be leading to connect failed response.
https://chat.hyperledger.org/channel/cello?msg=A7FjwdpkKsuFpcJEA
@Gvedam if you are using ansible agent, it uses overlay network, there is no port mapping whatsoever.
[ ](https://chat.hyperledger.org/channel/cello?msg=ej4oFJ4zCLazRwNJn) @tongli ok...so how does 7050/tcp, 5984/tcp mappings listed? How do I connect to the peer from outside?
@Gvedam currently the fabric network sets up by ansible agent expect your apps running on the same network.
@tongli hello, do you have any idea why the timeout error I'm getting when trying to deploy cello on docker or swarm ?
printscreen is ^
Hey guys, while trying to set up Ansible I ran into this bug at src/agent/ansible/roles/env_flanneld/fastinitnode/tasks/apply.yml:76. It couldn't find the ssh keys on the remote server, adding this to the copy line seemed to fix it: remote_src:true
@JordanDearsley look at this image https://hub.docker.com/r/hyperledger/cello-ansible-agent/
it was built to help people avoid the ansible controller setting up
if you simply download that, then you can pretty much running it just like you have everything setup local.
it includes a ssh-agent inside the container, so that you do not have to deal with that headache.
also look at the first part of this doc on how to use it. https://github.com/hyperledger/cello/blob/master/docs/worker_ansible_docker.md
the image will be looking for the ssh key pair at /opt/agent/vars directory, so you can hookup that a directory there with all your other configuration files.
make sure that the key file name is fd and fd.pub, that is what the ssh-agent will be looking for.
also read the doc when you have question
of course, this channel can always provide some help
[ ](https://chat.hyperledger.org/channel/cello?msg=eaTfgLr7CnDLmZF7m) @tongli okay... im actually trying to run a performance test tool like PTE or hyperledger caliper to the cello network
[ ](https://chat.hyperledger.org/channel/cello?msg=SDKiZjufbAwsTmQ8t) @tongli Thanks! I'm trying to pull that image and I get this error when running: [WARNING]: log file at /opt/agent/run/ansible.log is not writeable and we cannot create it, aborting
[ ](https://chat.hyperledger.org/channel/cello?msg=7TicoDeJ4vd9fsJKs) @tongli Thanks! I tried what you suggested and I'm running into this error:
TASK [Create a VirtualBox VM] **************************************************
fatal: [fabric001]: FAILED! => {"changed": false, "cmd": "/usr/local/bin/VBoxManage clonevm fabricbase --name fabric001 --register", "msg": "[Errno 2] No such file or directory", "rc": 2}
fatal: [fabric002]: FAILED! => {"changed": false, "cmd": "/usr/local/bin/VBoxManage clonevm fabricbase --name fabric002 --register", "msg": "[Errno 2] No such file or directory", "rc": 2}
I've now installed it locally then mapped the directory, now I get
TASK [Create a VirtualBox VM] **************************************************
fatal: [fabric001]: FAILED! => {"changed": false, "cmd": "/usr/local/bin/VBoxManage clonevm fabricbase --name fabric001 --register", "msg": "[Errno 13] Permission denied", "rc": 13}
fatal: [fabric002]: FAILED! => {"changed": false, "cmd": "/usr/local/bin/VBoxManage clonevm fabricbase --name fabric002 --register", "msg": "[Errno 13] Permission denied", "rc": 13}
I've tried setting permissions and adding --privileged
@tongli I watched your youtube video and tried to recreate on an AWS instance, but I end up getting a permission denied error and the instances never actually start up. I created the ssh keygen in ~/.ssh/, fd and fd.pub. I put my aws ID and secret, then ran the same command you did. Any idea why it wouldn't work?
Has joined the channel.
Has joined the channel.
hello
Screen Shot 2018-05-21 at 12.22.10 AM.png
can someone please let me know where am doing mistake ?
Has joined the channel.
cello-user-dashboard | 21:18:25 0|index | s:
errors what am seeing fir the issue
Has joined the channel.
Has joined the channel.
@JordanDearsley if you put up the error that you saw, I probably can help a bit.
[ ](https://chat.hyperledger.org/channel/cello?msg=H4jEGYp5hYGrGRHom) @Gvedam , it means that we need to run from the same machine when the fabric is setup right?
@tongli , i have setup the network using Cello , now i am planning to integrate using Node SDK from other machine using the connection profile settings , but not able to connect to the peer because when i see the netstat -tulpn , i don't see anything running on 7051 , but the peer container is up and running , how do i connect from another machine to the system in which peer is deployed?
@kiranarshakota if you are using ansible agent to setup fabric network, assumption is that your apps will run on the same network.
and all fabric stuff running on the overlay network.
simply looking at the network on the node may not reveal what you are looking for.
if you do an inspection of container, you should see them having ip address like 172.xxx
which should not be the network segment where your servers are on since this is a virtual network.
Has joined the channel.
Hi all, I am new to cello. I am following readdocs. In that I am using ubantu 16.04 commands
[Service]
DOCKER_OPTS="$DOCKER_OPTS -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --api-cors-header='*' --default-ulimit=nofile=8192:16384 --default-ulimit=nproc=8192:16384"
EnvironmentFile=-/etc/default/docker
ExecStart=
ExecStart=/usr/bin/dockerd -H fd:// $DOCKER_OPTS // I would like to know what's this -H fd://$DOCKER_OPTS(while executing I am getting -H command not found)
Could someone pls help me on this.
Thank you.
[ ](https://chat.hyperledger.org/channel/cello?msg=LLkGG4fDRSinDjDD2) can some able to help ?
~/deploy/cello$ ifconfig eth0
eth0: error fetching interface information: Device not found
Could someone pls suggest how can I resolve the error
@phulei what version are u using?
Has joined the channel.
Has joined the channel.
@tongli Hi there, I try to run the ansible controller in a docker container to launch fabric on aws. I've configured the files, and the script runs successfully. However, no instances are actually running on aws during or after I run the script. Any Ideas? Here is my output:
https://pastebin.com/HxtjPTQ5
The dashboard is also inaccessible on all, so I'm assuming they are not running whatsoever
seems to me that you run the only first step which is to provision 3 nodes
then next 2 steps are initialize the node and setupfabric.
if you do not really want to do that, there is one short cut.
do basically this
```docker run --rm -v ~/Desktop/opt/agent/run:/opt/agent/run -v ~/Desktop/opt/agent/vars:/opt/agent/vars hyperledger/cello-ansible-agent ansible-playbook -e "mode=apply" aws.yml
that is just one step which accomplish all 3 things
My end goal is to have fabric running with composer playground on the network
after that is done, you should have a functioning fabric network.
right.
Oh I see, let me give it a try. thanks!
that one step does all that for you and composer playground should be working.
since you already run the step 1, you probably want to remove that first.
so the command to do that is this
Another quick question, is it possible to have both Kubernetes and playground on the same network? I noticed they both use port 8080 for their dashboards
```docker run --rm -v ~/Desktop/opt/agent/run:/opt/agent/run -v ~/Desktop/opt/agent/vars:/opt/agent/vars hyperledger/cello-ansible-agent ansible-playbook -e "mode=destroy env=aws cloud_type=aws" provcluster.yml```
notice the word destroy instead of apply?
Oh I see, thanks!
the above command cleans up everything you just did.
And about Kubernetes/playground?
then use the one command to do everything altogerher
```docker run --rm -v ~/Desktop/opt/agent/run:/opt/agent/run -v ~/Desktop/opt/agent/vars:/opt/agent/vars hyperledger/cello-ansible-agent ansible-playbook -e "mode=apply" aws.yml```
looks like that you have already got ssh key file in the right place, I think you should be ok.
if you see some problems, it will be always good to post here, so that I can take a look.
@tongli Turns out I was looking at the wrong region on AWS, had about 25 instances running when I checked, lol.
@JordanDearsley haha.
yeah, probably remove them all and change the vars/aws.yml file to use the right region or just look at the right region.
also it will be really useful, if you pick a ubuntu 16.04 image which already has python and docker ce installed.
it will save time. sometimes using cloud init, due to the network issues, the VMs may not get provisioned very well.
I normally just create my own ubuntu 16.04 image with whatever I need, then I use that image.
just a sort of trick if you want to try tit.
try it.
Okay, so now the instances run. but I get this error at [Create the certificate files]:
fatal: [52.14.97.95]: FAILED! => {"changed": false, "cmd": "/opt/gopath/bc1st/src/github.com/hyperledger/fabric/build/bin/cryptogen generate --config crypto-config.yml --output keyfiles", "msg": "[Errno 2] No such file or directory", "rc": 2}
right.
you need to specify where to find the cryptogen,
this is actually a good sign.
you just need to configure it.
in the vars/bc1st.yml file.
this is the value for repo.bin I think.
```https://nexus.hyperledger.org/content/repositories/releases/org/hyperledger/fabric/hyperledger-fabric-build/linux-amd64-ff6f6dbffed57efb9f7d8886b4a47949ce2d4396/hyperledger-fabric-build-linux-amd64-ff6f6dbffed57efb9f7d8886b4a47949ce2d4396.tar.gz```
Oh, yes of course. I skipped most of the config. I forgot I pulled a fresh version of the repo
that was part of a conversation earlier with someone else I believe.
same problem.
@tgehrke Ohai!
it probably will be good to have that value in the bc1st.yml as a default.
if someone put up a patch to just change that value, I will +2 to it. easy contribution.
fatal: [18.188.241.109]: FAILED! => {"changed": false, "msg": "Failure downloading https://nexus.hyperledger.org/content/repositories/releases/org/hyperledger/fabric/hyperledger-fabric-build/linux-amd64-ff6f6dbffed57efb9f7d8886b4a47949ce2d4396/hyperledger-fabric-build-linux-amd64-ff6f6dbffed57efb9f7d8886b4a47949ce2d4396.tar.gz, HTTP Error 404: Not Found"}
@tongli is that the correct link?
Would this still work? https://nexus.hyperledger.org/content/repositories/releases/org/hyperledger/fabric/hyperledger-fabric-build/linux-amd64-0037682/hyperledger-fabric-build-linux-amd64-0037682.tar.gz
Used this instead: https://nexus.hyperledger.org/content/repositories/releases/org/hyperledger/fabric/hyperledger-fabric-build/linux-amd64-0037682/hyperledger-fabric-build-linux-amd64-0037682.tar.gz
@JordanDearsley try this
https://nexus.hyperledger.org/content/repositories/releases/org/hyperledger/fabric/hyperledger-fabric/linux-amd64-1.1.0/hyperledger-fabric-linux-amd64-1.1.0.tar.gz
that is for release 1.1.0, it should work.
I do not know why I got the snapshot build link, sorry about that.
Okay, seems to be working!
good to hear.
@tongli Is it possible to have both the kubernetes and playground dashboards available?
@JordanDearsley haha, I did not have time to get it in k8s land yet.
it will take some time to do that
cello ansible agent also sets up k8s env and overlay network all that kind of stuff.
it is pretty complicated.
Okay no problem! We're still in early development so we can live without it
Also, thanks so much for your help with all this!
all this is for the users to be able to use it easily.
once you get a handle of this, you will like what you accomplished with just few commands.
It's quite amazing
you can easily change the vars/bc1st.yml to make whatever fabric network you want.
such as number of orgs, peers, how you want to name them etc.
we are in process of deploy onto an existing k8s env,
once that is done, if you have a k8s cluster, cello ansible agent will be able to use the same configuration file to deploy onto the cluster.
of course, if you want cello ansible agent to stand up a k8s env, it can do that as well.
any way, if you find this helpful, I hope that you can start help fixing bugs or issues for the ansible agent.
Has joined the channel.
Oh cool thanks! I'll open any issues I find on the repo as I work through this.
A couple things:
I noticed when I rebooted the master node, it didn't automatically connect back to the network on bootup. How should I reconnect it?
Also, how do I deploy a bna to an existing network through the command line? Do I use compose in the node? I noticed it wasn't installed. Or do I have to execute commands inside the docker container in the node?
I'm trying to avoid using playground for deploying bna files
I was trying to run worker-node I followed every step and was able to regenerate docker service script and restart the docker engine. When I was trying to run docker -H worker_node_ip:2375 info at Master I was getting device not found, is the docker deamon running?
I was running Master on one VM and worker on my physical ubantu 16.04. Could someone plz suggest me on this.
Has joined the channel.
Has joined the channel.
@baohua @hightall Please review the patch set. https://gerrit.hyperledger.org/r/#/c/22001/
docs have been updated to reflect this.
@here, we recently have added support to ansible agent so that you can deploy fabric network onto an existing k8s cluster.
how recently? Like within the past week or past day?
here is the patch set. https://gerrit.hyperledger.org/r/#/c/22001/ we are very excited about this addition.
oh I see
@vcoss still a patch set, need to merge.
before that, we have ansible agent sets up k8s, then deploy fabric network onto the k8s , but it can not deploy onto an existing k8s such as google container env.
with that patch set, you can choose either to stand up your own k8s using ansible agent or you can choose to stand up fabric network onto a third party created k8s cluster.
it really just takes one command to do that.
how do I patch?
git pull https://gerrit.hyperledger.org/r/cello refs/changes/01/22001/8
thanks
and this doc shows how
https://gerrit.hyperledger.org/r/#/c/22001/8/docs/worker_ansible_howto.md
```ansible-playbook -e "mode=apply env=bc1st deploy_type=k8s" setupfabric.yml```
that is the one line command to do it.
Its saying that https://gerrit.hyperledger.org/r/cello refs/changes/01/22001/8 is not a git directory
do this ```git pull https://gerrit.hyperledger.org/r/cello refs/changes/01/22001/8```
that will pull the patch set.
to where you have cello downloaded.
fatal: Not a git repository (or any of the parent directories): .git
'''fatal: Not a git repository (or any of the parent directories): .git'''
so you do not have cello clone yet.
weird I downloaded it last night
fatal: destination path 'cello' already exists and is not an empty directory.
cd cello
then try the git pull command above
ohh
Thank you
np
with this patch will I be able to upload a .bna directly into the fabric?
without playground
do I run `ansible-playbook -e "mode=apply env=bc1st deploy_type=k8s" setupfabric.yml` inside or outside of the node?
Has joined the channel.
because when I run `composer network install --card PeerAdmin@orga --archiveFile
because when I run `composer network install --card PeerAdmin@orga --archiveFile
@vcoss that patch has nothing to do with composer.
it allows you to upload fabrics to third parties
that patch set is to allow users to deploy fabric network onto existing k8s
to deploy fabric network onto 3rd party k8s cluster, yes.
is there a reason I timeout when I upload to the fabric tho?
I do not know what you were trying to do and in what kind of env.
im trying to upload a bna without playground. I can upload the .bna with playground, but it times out when I attempt the samething with console
upload an existing .bna to the fabric created by cello/ansible
using console commands
its on an aws, I log in and download composer and attempt to upload the .bna to the existing fabric
@tongli I think @vcoss and I are having the same issue, deploying a bna onto a fabric network on aws. I've tried installing composer-cli on the master node and running "composer network install"... The response is timeout from all the peers. Am I trying to deploy the BNA correctly?
For what we're working on it's necessary that we deploy via command line
@JordanDearsley in a meeting right now.
can look at it later.
@tongli great! thanks so much. Sorry for the trouble
Hello, when trying to create a chain, I get the following error: ERROR [agent.docker.docker_swarm] [docker_swarm.py:274 get_swarm_node_ip()] - 404 Client Error: Not Found ("No such container: d690aa9db35b467a887c5c7d6bd51dee_peer0.[23/1805]
ple.com")
it seems that it create the container but it cannot find it, despite the container is created:
("No such container: d690aa9db35b467a887c5c7d6bd51dee_peer0.org1.example.com")
It could be a difference between the created name and the expected name on the status of creation check ?
ERROR [agent.docker.docker_swarm] [docker_swarm.py:273 get_swarm_node_ip()] - Exception happens when detect container host!
tong pls dad
@tongli Any ideas on how to load an existing .bna to the master node's fabric?
I think before you run setup fabric, you just need to place your own .bna file in secrets/networks
where you already see one .bna file, you can place your own there.
then do the following
```export NETWORKNAME=thefilename of the .bna file```
this way, you specified network will be deployed instead of the carauction-network.
@vcoss
Thanks! How do you redeploy the .bna file after the network has started, is that even possible?
via the command line
yes
possible.
I think you should do the following
```ansible-playbook -i run/runhosts -e "mode=apply env=bc1st NETWORKNAME=whateveryournetwork" setupfabric.yml --skip-tags="fabricbuild,certssetup,fabricsetup"```
give that a try, it should work.
remember place the .bna file in the same directory that carauction.bna is.
@tongli for `export NETWORKNAME=thefilename of the .bna file` do I include the .bna or just the file name with out it?
I deleted carauction.bna and replaced it with mine however when I run it while exporting NETWORKNAME it replaces it with carauction in the fabric and my bna is nowhere to be found.
@baohua @hightall please take a look at the patch set .
Need to get merged to run system tests
@hightall will give the project update to TSC tonight, welcome to help review: https://wiki.hyperledger.org/groups/tsc/project-updates/cello-2018-may
@tongli merged
Has joined the channel.
Has left the channel.
Has joined the channel.
@baohua thanks very much! now we can use ansible agent to deploy onto any existing k8s with given kubeconfig file.
the fabric network can be anything one prefer, all just need to be defined in vars/bc1st.yml
that's cool! do you like to create some quick tutorial video to demo that feature?
yeah, that will be nice. In my to do list, so many things to do.
had a meeting earlier this morning, can not make to the tsc meeting.
@hightall just made a perfect project update on the tst meeting.
Has joined the channel.
I have great troubles installing Docker engine: 1.10.0~1.13.0 for Cello, Anyone could give me a pointer? And Why is cello using such an old version of Docker?
@tongli Hi there, I tried your response to @vcoss and it seems to work up to "Create core.yml file for every peer". Here is the error:
`TASK [Create core.yml file for every peer] *************************************
[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using
`result|version_compare` instead use `result is version_compare`. This feature
will be removed in version 2.9. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
failed: [18.207.106.137] (item={'org': u'orga', 'role': u'anchor', 'name': u'peer1st-orga'}) => {"changed": false, "item": {"name": "peer1st-orga", "org": "orga", "role": "anchor"}, "msg": "AnsibleFilterError: Version comparison: 'project_version' is undefined"}
failed: [18.207.106.137] (item={'org': u'orga', 'role': u'worker', 'name': u'peer2nd-orga'}) => {"changed": false, "item": {"name": "peer2nd-orga", "org": "orga", "role": "worker"}, "msg": "AnsibleFilterError: Version comparison: 'project_version' is undefined"}
failed: [18.207.106.137] (item={'org': u'orga', 'role': u'worker', 'name': u'peer3rd-orga'}) => {"changed": false, "item": {"name": "peer3rd-orga", "org": "orga", "role": "worker"}, "msg": "AnsibleFilterError: Version comparison: 'project_version' is undefined"}
failed: [18.207.106.137] (item={'org': u'orgb', 'role': u'anchor', 'name': u'peer1st-orgb'}) => {"changed": false, "item": {"name": "peer1st-orgb", "org": "orgb", "role": "anchor"}, "msg": "AnsibleFilterError: Version comparison: 'project_version' is undefined"}
failed: [18.207.106.137] (item={'org': u'orgb', 'role': u'worker', 'name': u'peer2nd-orgb'}) => {"changed": false, "item": {"name": "peer2nd-orgb", "org": "orgb", "role": "worker"}, "msg": "AnsibleFilterError: Version comparison: 'project_version' is undefined"}
failed: [18.207.106.137] (item={'org': u'orgb', 'role': u'worker', 'name': u'peer3rd-orgb'}) => {"changed": false, "item": {"name": "peer3rd-orgb", "org": "orgb", "role": "worker"}, "msg": "AnsibleFilterError: Version comparison: 'project_version' is undefined"}
to retry, use: --limit @/opt/agent/setupfabric.retry
'
@tongli Hi there, I tried your response to @vcoss and it seems to work up to "Create core.yml file for every peer". Here is the error:
`TASK [Create core.yml file for every peer] *************************************
[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using
`result|version_compare` instead use `result is version_compare`. This feature
will be removed in version 2.9. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
failed: [18.207.106.137] (item={'org': u'orga', 'role': u'anchor', 'name': u'peer1st-orga'}) => {"changed": false, "item": {"name": "peer1st-orga", "org": "orga", "role": "anchor"}, "msg": "AnsibleFilterError: Version comparison: 'project_version' is undefined"}
failed: [18.207.106.137] (item={'org': u'orga', 'role': u'worker', 'name': u'peer2nd-orga'}) => {"changed": false, "item": {"name": "peer2nd-orga", "org": "orga", "role": "worker"}, "msg": "AnsibleFilterError: Version comparison: 'project_version' is undefined"}
failed: [18.207.106.137] (item={'org': u'orga', 'role': u'worker', 'name': u'peer3rd-orga'}) => {"changed": false, "item": {"name": "peer3rd-orga", "org": "orga", "role": "worker"}, "msg": "AnsibleFilterError: Version comparison: 'project_version' is undefined"}
failed: [18.207.106.137] (item={'org': u'orgb', 'role': u'anchor', 'name': u'peer1st-orgb'}) => {"changed": false, "item": {"name": "peer1st-orgb", "org": "orgb", "role": "anchor"}, "msg": "AnsibleFilterError: Version comparison: 'project_version' is undefined"}
failed: [18.207.106.137] (item={'org': u'orgb', 'role': u'worker', 'name': u'peer2nd-orgb'}) => {"changed": false, "item": {"name": "peer2nd-orgb", "org": "orgb", "role": "worker"}, "msg": "AnsibleFilterError: Version comparison: 'project_version' is undefined"}
failed: [18.207.106.137] (item={'org': u'orgb', 'role': u'worker', 'name': u'peer3rd-orgb'}) => {"changed": false, "item": {"name": "peer3rd-orgb", "org": "orgb", "role": "worker"}, "msg": "AnsibleFilterError: Version comparison: 'project_version' is undefined"}
to retry, use: --limit @/opt/agent/setupfabric.retry
`
@tongli Hi there, I tried your response to @vcoss and it seems to work up to "Create core.yml file for every peer". Here is the error:
`TASK [Create core.yml file for every peer] *************************************
failed: [18.207.106.137] (item={'org': u'orga', 'role': u'anchor', 'name': u'peer1st-orga'}) => {"changed": false, "item": {"name": "peer1st-orga", "org": "orga", "role": "anchor"}, "msg": "AnsibleFilterError: Version comparison: 'project_version' is undefined"}
failed: [18.207.106.137] (item={'org': u'orga', 'role': u'worker', 'name': u'peer2nd-orga'}) => {"changed": false, "item": {"name": "peer2nd-orga", "org": "orga", "role": "worker"}, "msg": "AnsibleFilterError: Version comparison: 'project_version' is undefined"}
failed: [18.207.106.137] (item={'org': u'orga', 'role': u'worker', 'name': u'peer3rd-orga'}) => {"changed": false, "item": {"name": "peer3rd-orga", "org": "orga", "role": "worker"}, "msg": "AnsibleFilterError: Version comparison: 'project_version' is undefined"}
failed: [18.207.106.137] (item={'org': u'orgb', 'role': u'anchor', 'name': u'peer1st-orgb'}) => {"changed": false, "item": {"name": "peer1st-orgb", "org": "orgb", "role": "anchor"}, "msg": "AnsibleFilterError: Version comparison: 'project_version' is undefined"}
failed: [18.207.106.137] (item={'org': u'orgb', 'role': u'worker', 'name': u'peer2nd-orgb'}) => {"changed": false, "item": {"name": "peer2nd-orgb", "org": "orgb", "role": "worker"}, "msg": "AnsibleFilterError: Version comparison: 'project_version' is undefined"}
failed: [18.207.106.137] (item={'org': u'orgb', 'role': u'worker', 'name': u'peer3rd-orgb'}) => {"changed": false, "item": {"name": "peer3rd-orgb", "org": "orgb", "role": "worker"}, "msg": "AnsibleFilterError: Version comparison: 'project_version' is undefined"}
to retry, use: --limit @/opt/agent/setupfabric.retry
`
@JordanDearsley that means you can not skip the build probably.
Where do I specify the version? And how can I pull the latest from the bna?
add in -e "aaaa=stuff"
try that.
exactly that?
lol
like -e "project_version=1.1.0" I think in addition to what already is there.
ah I see, thanks! Didn't see the variable name in the error. Really appreciate your help with this. I'll give it a go
good luck.
TASK [Runtime install] *********************************************************
failed: [18.207.106.137] (item=orga) => {"changed": true, "cmd": ["docker", "run", "-v", "/home/ubuntu/.composer:/home/composer/.composer", "-v", "/opt/gopath/bc1st/fabric:/opt/gopath/bc1st/fabric", "hyperledger/composer-cli:next", "runtime", "install", "-c", "PeerAdmin@orga", "-n", "carauction-network"], "delta": "0:00:01.991509", "end": "2018-05-24 15:08:50.606265", "item": "orga", "msg": "non-zero return code", "rc": 1, "start": "2018-05-24 15:08:48.614756", "stderr": "? Installing runtime for business network carauction-network. This may take a minute...", "stderr_lines": ["? Installing runtime for business network carauction-network. This may take a minute..."], "stdout": "Error: Error trying install composer runtime. Error: The Composer runtime is already installed on all the peers\nCommand failed", "stdout_lines": ["Error: Error trying install composer runtime. Error: The Composer runtime is already installed on all the peers", "Command failed"]}
failed: [18.207.106.137] (item=orgb) => {"changed": true, "cmd": ["docker", "run", "-v", "/home/ubuntu/.composer:/home/composer/.composer", "-v", "/opt/gopath/bc1st/fabric:/opt/gopath/bc1st/fabric", "hyperledger/composer-cli:next", "runtime", "install", "-c", "PeerAdmin@orgb", "-n", "carauction-network"], "delta": "0:00:02.004709", "end": "2018-05-24 15:08:53.473012", "item": "orgb", "msg": "non-zero return code", "rc": 1, "start": "2018-05-24 15:08:51.468303", "stderr": "? Installing runtime for business network carauction-network. This may take a minute...", "stderr_lines": ["? Installing runtime for business network carauction-network. This may take a minute..."], "stdout": "Error: Error trying install composer runtime. Error: The Composer runtime is already installed on all the peers\nCommand failed", "stdout_lines": ["Error: Error trying install composer runtime. Error: The Composer runtime is already installed on all the peers", "Command failed"]}
Any way to skip a specific task?
Has joined the channel.
#training-and-education-wg includes volunteers available to edit, test and contribute to tutorials and learning materials. Feel free to post possible collaboration in the wg or @KellyCooper. We have many volunteers for beginning projects and some volunteers for advanced. One way we can support your efforts is as users/consumers to comment on clarity, strengths and gaps in your documentation/materials. Your request(s) will be viewed by the group every Monday and we can track/manage on our end. Thanks, Kelly
Okay so finally got pretty far, but ran into another error:
TASK [Start network] ***********************************************************
fatal: [34.200.233.50]: FAILED! => {"changed": true, "cmd": ["docker", "run", "-v", "/home/ubuntu/.composer:/home/composer/.composer", "-v", "/opt/gopath/bc1st/fabric:/opt/gopath/bc1st/fabric", "hyperledger/composer-cli:next", "network", "start", "-c", "PeerAdmin@orga", "-a", "/opt/gopath/bc1st/fabric/run/keyfiles/carauction-network.bna", "-A", "orgaAdmin", "-C", "/home/composer/.composer/orgaAdmin/admin-pub.pem", "-A", "orgbAdmin", "-C", "/home/composer/.composer/orgbAdmin/admin-pub.pem"], "delta": "0:00:01.124696", "end": "2018-05-24 16:18:03.115103", "msg": "non-zero return code", "rc": 1, "start": "2018-05-24 16:18:01.990407", "stderr": "", "stderr_lines": [], "stdout": "Starting business network from archive: /opt/gopath/bc1st/fabric/run/keyfiles/carauction-network.bna\nError: Archive file /opt/gopath/bc1st/fabric/run/keyfiles/carauction-network.bna does not exist.\nCommand failed", "stdout_lines": ["Starting business network from archive: /opt/gopath/bc1st/fabric/run/keyfiles/carauction-network.bna", "Error: Archive file /opt/gopath/bc1st/fabric/run/keyfiles/carauction-network.bna does not exist.", "Command failed"]}
My guess is that the environment variable is effective when running ansible locally, but the ansible plays ran on the AWS server don't have that environment variable set, and it defaults to carauction-network
Found the solution,
` docker run --rm -v ~/cello/src/agent/ansible/roles:/opt/agent/roles -v ~/cello/src/agent/ansible/run:/opt/agent/run -v ~/cello/src/agent/ansible/vars:/opt/agent/vars -v ~/cello/src/agent/ansible/secrets:/opt/agent/secrets hyperledger/cello-ansible-agent ansible-playbook -e "mode=apply NETWORKNAME=$bnaName project_version=$version" aws.yml
`
Found the solution,
``` docker run --rm -v ~/cello/src/agent/ansible/roles:/opt/agent/roles -v ~/cello/src/agent/ansible/run:/opt/agent/run -v ~/cello/src/agent/ansible/vars:/opt/agent/vars -v ~/cello/src/agent/ansible/secrets:/opt/agent/secrets hyperledger/cello-ansible-agent ansible-playbook -e "mode=apply NETWORKNAME=$bnaName project_version=$version" aws.yml
```
Found the solution,
``` docker run --rm -v ~/cello/src/agent/ansible/roles:/opt/agent/roles -v ~/cello/src/agent/ansible/run:/opt/agent/run -v ~/cello/src/agent/ansible/vars:/opt/agent/vars -v ~/cello/src/agent/ansible/secrets:/opt/agent/secrets hyperledger/cello-ansible-agent ansible-playbook -e "mode=apply project_version=$version" aws.yml
```
I hard coded the network name then mounted all the modified directories
Going to test updating the bna via command line now
@tongli I feel like ```ansible-playbook -i run/runhosts -e "mode=apply env=bc1st NETWORKNAME=whateveryournetwork" setupfabric.yml --skip-tags="fabricbuild,certssetup,fabricsetup"``` may still be a bit overkill just to update the chaincode. Are there any plays that make use of ```composer network upgrade -c peeradmin@hlfv1 -n NETWORK-NAME -V NETWORK-VERSION```?
@tongli I feel like ```ansible-playbook -i run/runhosts -e "mode=apply env=bc1st NETWORKNAME=whateveryournetwork" setupfabric.yml --skip-tags="fabricbuild,certssetup,fabricsetup"``` may still be a bit overkill just to update the chaincode. Are there any plays that make use of ```composer network upgrade -c peeradmin@hlfv1 -n NETWORK-NAME -V NETWORK-VERSION```?
@tongli I feel like
```ansible-playbook -i run/runhosts -e \"mode=apply env=bc1st NETWORKNAME=whateveryournetwork\" setupfabric.yml --skip-tags=\"fabricbuild,certssetup,fabricsetup"```
may still be a bit overkill just to update the chaincode. Are there any plays that make use of ```composer network upgrade -c peeradmin@hlfv1 -n NETWORK-NAME -V NETWORK-VERSION```?
@tongli I feel like
```ansible-playbook -i run/runhosts -e "mode=apply env=bc1st NETWORKNAME=whateveryournetwork" setupfabric.yml --skip-tags="fabricbuild,certssetup,fabricsetup"```
may still be a bit overkill just to update the chaincode. Are there any plays that make use of ```composer network upgrade -c peeradmin@hlfv1 -n NETWORK-NAME -V NETWORK-VERSION```?
@tongli I feel like
ansible-playbook -i run/runhosts -e "mode=apply env=bc1st NETWORKNAME=whateveryournetwork" setupfabric.yml --skip-tags="fabricbuild,certssetup,fabricsetup"
may still be a bit overkill just to update the chaincode. Are there any plays that make use of ```composer network upgrade -c peeradmin@hlfv1 -n NETWORK-NAME -V NETWORK-VERSION```?
@JordanDearsley you can contribute to make that happen certainly.
@tongli Great! I'll try it out
@tongli I've submitted a pull request!
pull request? I think it should be patch set.
where is the link to it?
https://github.com/hyperledger/cello/pull/6?
we only use the github for read-only purpose.
we can not merge a pull request.
How do I submit a patch set?
Sorry, did further testing and found some issues, will submit when done
I created an ansible play to upload the new bna, run composer network install, and run composer network upgrade. The install works fine, but when trying to upgrade the network, I get `"Response from attempted peer comms was an error: Error: 2 UNKNOWN: chaincode error (status: 500, message: cannot get package for chaincode (empty-business-network:0.0.6))", "Command failed"]}`
Any idea why I would get this error?
Here is the apply.yml that's getting run:
Here is the apply.yml that I wrote:
`---
- name: Setup and initialize variables
set_fact:
current_host: "{{ hostvars[inventory_hostname].inter_name }}"
peers: "{{ [] }}"
- name: Get peer container list
set_fact:
peers: |
{{ peers + [{'org':item.split('@')[1].split('.')[-1],
'name':item.split('@')[1] | replace('.', '-'),
'role':item.split('@')[0]}] }}
with_items: "{{ fabric.network[current_host].peers | default([]) }}"
- name: Setup few variables
set_fact:
fabricworkdir: "/opt/gopath/{{ env }}/fabric"
fabricpath: "/opt/gopath/{{ env }}/src/github.com/hyperledger/fabric"
gopath: "/opt/gopath/{{ env }}"
localhome: "/home/{{ fabric.ssh_user }}/.composer"
alladmin: ""
peerorgs: "{{ peers | map(attribute='org') | list | unique | sort }}"
networkname: "{{ lookup('env', 'NETWORKNAME') | default('') }}"
version: "{{ lookup('env', 'project_version') | default('') }}"
- name: Generate all admin options
set_fact:
alladmin: "{{ alladmin + '-A ' + item + 'Admin -C /home/composer/.composer/' + item + 'Admin/admin-pub.pem ' }}"
with_items: "{{ peerorgs }}"
tags: "generatealladmin"
- name: Upload bna file
copy:
src: "{{ item }}"
dest: "{{ fabricworkdir }}/run/keyfiles/"
with_fileglob:
- "{{ playbook_dir }}/../../secrets/networks/*.bna"
tags: "uploadbna"
- name: Upload bna file again
copy:
src: "{{ item }}"
dest: "{{ fabricworkdir }}/keyfiles/"
with_fileglob:
- "{{ playbook_dir }}/../../secrets/networks/*.bna"
tags: "uploadbna2"
- name: Install new bna on network
command: >-
docker run -v {{ localhome }}:/home/composer/.composer
-v {{ fabricworkdir }}:{{ fabricworkdir }}
hyperledger/composer-cli network install
-c PeerAdmin@{{ peerorgs[0] }}
-a {{ fabricworkdir }}/run/keyfiles/{{ networkname }}.bna
tags: "installbna"
- name: Upgrade network bna
command: >-
docker run -v {{ localhome }}:/home/composer/.composer
-v {{ fabricworkdir }}:{{ fabricworkdir }}
hyperledger/composer-cli network upgrade
-c PeerAdmin@{{ peerorgs[0] }} -n {{ networkname }}
-V {{ version }}
tags: "upgradenetwork"
`
```---
- name: Setup and initialize variables
set_fact:
current_host: "{{ hostvars[inventory_hostname].inter_name }}"
peers: "{{ [] }}"
- name: Get peer container list
set_fact:
peers: |
{{ peers + [{'org':item.split('@')[1].split('.')[-1],
'name':item.split('@')[1] | replace('.', '-'),
'role':item.split('@')[0]}] }}
with_items: "{{ fabric.network[current_host].peers | default([]) }}"
- name: Setup few variables
set_fact:
fabricworkdir: "/opt/gopath/{{ env }}/fabric"
fabricpath: "/opt/gopath/{{ env }}/src/github.com/hyperledger/fabric"
gopath: "/opt/gopath/{{ env }}"
localhome: "/home/{{ fabric.ssh_user }}/.composer"
alladmin: ""
peerorgs: "{{ peers | map(attribute='org') | list | unique | sort }}"
networkname: "{{ lookup('env', 'NETWORKNAME') | default('') }}"
version: "{{ lookup('env', 'project_version') | default('') }}"
- name: Generate all admin options
set_fact:
alladmin: "{{ alladmin + '-A ' + item + 'Admin -C /home/composer/.composer/' + item + 'Admin/admin-pub.pem ' }}"
with_items: "{{ peerorgs }}"
tags: "generatealladmin"
- name: Upload bna file
copy:
src: "{{ item }}"
dest: "{{ fabricworkdir }}/run/keyfiles/"
with_fileglob:
- "{{ playbook_dir }}/../../secrets/networks/*.bna"
tags: "uploadbna"
- name: Upload bna file again
copy:
src: "{{ item }}"
dest: "{{ fabricworkdir }}/keyfiles/"
with_fileglob:
- "{{ playbook_dir }}/../../secrets/networks/*.bna"
tags: "uploadbna2"
- name: Install new bna on network
command: >-
docker run -v {{ localhome }}:/home/composer/.composer
-v {{ fabricworkdir }}:{{ fabricworkdir }}
hyperledger/composer-cli network install
-c PeerAdmin@{{ peerorgs[0] }}
-a {{ fabricworkdir }}/run/keyfiles/{{ networkname }}.bna
tags: "installbna"
- name: Upgrade network bna
command: >-
docker run -v {{ localhome }}:/home/composer/.composer
-v {{ fabricworkdir }}:{{ fabricworkdir }}
hyperledger/composer-cli network upgrade
-c PeerAdmin@{{ peerorgs[0] }} -n {{ networkname }}
-V {{ version }}
tags: "upgradenetwork"
```
Has joined the channel.
Hello! I wonder about the Cello meeting today -- is there a special channel for it or is it going to be held here?
https://zoom.us/my/hyperledger.community
@miahfost great to see u today!
related to the fabric-utils projects, it will be nice you can help create an tracking issue at https://jira.hyperledger.org/projects/CE/
Thank you very much!
And add updates there :)
Makes sense, I'll do that.
I think our team needs to bring some more documentation as well so I'm working on that.
sure, and feel free to ping me or @tongli with questions, and feel free to use the maillist too: cello@lists.hyperledger.org
definitely, we treasure document as important as code!
Yes, I think I'll jump on the mailing list since the set of people who are here are not always the same set who are on the list.
Makes sense, modern software is so complex in its configuration that good clear documentation is vital! :-)
#miahfost, Jeremiah, ansible agent of cello seems do exactly what fabric util does if I heard you right.
That may be the case @tongli, it appears they're very similar.
ansible agent uses ansible to produce k8s deployment files according to a configuration file (which defines wanted fabric network topology),
From what I understand from our team however, there may be some things we do slightly differently.
@miahfost ok, happy to discuss and consolidate.
At the very least, I think our folks should look at the work that you've done so we can understand it.
Sometimes even our company suffers from "Not Invented Here" and that is not useful in open source
will be very glad to hear the differences.
But, on the other hand, if there is something useful in our code, we're happy to participate.
cello ansible agent can be run as a container as well.
@miahfost yeah, I am sure. will be happy to work together to improve
@tongli What is the URL to the code you've created? I'm sure you've posted it here a hundred times, but I didn't put it in my browser. :/
https://hub.docker.com/r/hyperledger/cello-ansible-agent
that is where you find the container if you want to give it a try
Cool, thanks
all the playbooks for ansible agent is here https://github.com/hyperledger/cello/tree/master/src/agent/ansible
as part of the cello project.
I use a file like this https://github.com/hyperledger/cello/blob/master/src/agent/ansible/vars/bc1st.yml to allow users to create their own fabric network
@tongli should we close this issue? https://jira.hyperledger.org/browse/CE-167
yes @baohua . that was done long time ago. let me close it
thanks
@miahfost users can create very big network or very small network, they can do that by changing that file or creating their own as long as they follow the same format.
the link to the ansible agent doc https://github.com/hyperledger/cello/blob/master/docs/worker_ansible_howto.md
the specific command to do this is just one command.
https://github.com/hyperledger/cello/blob/master/docs/worker_ansible_howto.md#using-existing-k8s-cluster
or something like this.
```ansible-playbook -e "mode=apply env=bc1st deploy_type=k8s" setupfabric.yml```
that commands set things up.
this command destroy all.
```ansible-playbook -e "mode=destroy env=bc1st deploy_type=k8s" setupfabric.yml```
Interesting. This is useful info.
@tongli Sorry to bother, any idea what would cause this error from the apply.yml I posted earlier? `"Response from attempted peer comms was an error: Error: 2 UNKNOWN: chaincode error (status: 500, message: cannot get package for chaincode (empty-business-network:0.0.6))", "Command failed"]}`
The error occurs when I run `composer network upgrade`
@JordanDearsley looks like the composer can not find where your .bna file.
Shouldn't `composer network install` install it in the correct place on all the peers? The installation task runs successfully
@tongli I've been following these docs: https://hyperledger.github.io/composer/latest/business-network/upgrading-bna
I would think it depends where you run that command.
I run the command on the ansible controller from my local docker container
I suggest that you do the following.
It moves the bna successfully
1. use ansible agent to setup the network which comes with carauction.bna installed and configured. you should be able to verify that.
and the playground should be up running as well. you should be able to run and test from a browser
2. once all these things are confirmed, you know that you have a running network.
3. to add more networks, you can log into the first node where composer is running.
during the process of step1, all necessary composer containers have been downloaded
including the composer command line tool.
I've managed steps one and two, confirmed that it's installed on playground correctly as well
Regarding step three, doesn't ansible take care of the login? I'm running on the builder node
then you can log into the first node, with composer command line tool container (do not remember exact name of the container image), you should be able to follow the right process to add more things onto.
@JordanDearsley what login?
once ansible agent is done, it does not do anything more.
it gives you the connection profiles about the network,
after that, it is all on you when you need to add more things onto.
Okay I'll try it out
Thanks!
yeah. good luck.
What directories would I need to mount when running the composer cli container?
take a look at the playbook which agent uses.
I do not remember off top of head.
Okay, thanks
https://github.com/hyperledger/cello/blob/master/src/agent/ansible/roles/deploy_compose/composersetup/tasks/apply.yml
should be in that file.
I think
maybe here https://github.com/hyperledger/cello/blob/master/src/agent/ansible/roles/deploy_compose/composersetup/tasks/apply.yml#L32
you can verify that.
yeah, I think you should follow the exact same steps in that playbook to add new network.
missing any step probably will not work.
I think you should be able to just run the playbook with new parameter.
I am not sure why it wont work. you may just need to skip standing up the playground container since it is already running.
Unfortunately, there's no plays that make use of `composer network upgrade`
I was hoping to contribute that to the project
yeah, that will be very nice.
Any idea why the code I posted earlier wouldn't work?
Specifically, running this on the builder node fails:
```- name: Upgrade network bna
command: >-
docker run -v {{ localhome }}:/home/composer/.composer
-v {{ fabricworkdir }}:{{ fabricworkdir }}
hyperledger/composer-cli network upgrade
-c PeerAdmin@{{ peerorgs[0] }} -n {{ networkname }}
-V {{ version }}
tags: "upgradenetwork"```
the bna is installed successfully on the network, it's just when I try to run the `upgrade` command, which tells all peer nodes to use the latest version, they can't seem to find the package.
@JordanDearsley I do not really know what problem might be. I wonder if you can manually do these steps to verify each step described by playbook, then you probably can get more info.
when you do these things manually, all things including your bna file should be available.
playbooks basically do exactly things but just automate it a bit.
Okay will do, thanks. I have a feeling I'll get the same result
On the builder node, where should the bna be stored?
prior to installation
if you get same errors, then you can take the error to composer channel, they may provide some help.
Good point, will do
in terms of where bna file, please look at the playbook.
how the nba file was copied to and where and how it is mounted.
Has joined the channel.
Hello, Iḿ getting these eroors when I try to create a Chain with the Operator-Dashboard:
Creating 0a3d4e041e2c47d4a387fd77aa0d90a0_cli ... error
cello-watchdog | [2018-05-25 16:20:50,759] INFO [__main__] [watchdog.py:125 watch_run()] - Watchdog run checks with period = 15 s
cello-watchdog | [2018-05-25 16:20:50,760] INFO [modules.host] [host.py:248 list()] - filter data {}
cello-watchdog | [2018-05-25 16:20:50,784] INFO [__main__] [watchdog.py:127 watch_run()] - Found 1 hosts
cello-watchdog | [2018-05-25 16:21:08,838] INFO [__main__] [watchdog.py:125 watch_run()] - Watchdog run checks with period = 15 s
cello-watchdog | [2018-05-25 16:21:08,839] INFO [modules.host] [host.py:248 list()] - filter data {}
Creating 0a3d4e041e2c47d4a387fd77aa0d90a0_explorer ... error
cello-watchdog | [2018-05-25 16:21:26,910] INFO [__main__] [watchdog.py:125 watch_run()] - Watchdog run checks with period = 15 s
cello-watchdog | [2018-05-25 16:21:26,911] INFO [modules.host] [host.py:248 list()] - filter data {}
cello-watchdog | [2018-05-25 16:21:26,929] INFO [__main__] [watchdog.py:127 watch_run()] - Found 1 hosts
cello-operator-dashboard |
cello-operator-dashboard | ERROR: for 0a3d4e041e2c47d4a387fd77aa0d90a0_cli Cannot start service cli: oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:359: container init caused \"rootfs_linux.go:54: mounting \\\"/opt/cello/fabric-1.0/solo/configtx.yaml\\\" to rootfs \\\"/var/lib/docker/overlay2/0b7660a66e27b270a892f4324868bdc5c2784c98cadc5fa95cb4ea6099eaa105/merged\\\" at \\\"/var/lib/docker/overlay2/0b7660a66e27b270a892f4324868bdc5c2784c98cadc5fa95cb4ea6099eaa105/merged/etc/hyperledger/fabric/configtx.yaml\\\" caused \\\"not a directory\\\"\""
cello-operator-dashboard | : Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
cello-operator-dashboard |
cello-operator-dashboard | ERROR: for 0a3d4e041e2c47d4a387fd77aa0d90a0_explorer Cannot start service explorer: oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:359: container init caused \"rootfs_linux.go:54: mounting \\\"/opt/cello/fabric-1.0/explorer-artifacts/config.json\\\" to rootfs \\\"/var/lib/docker/overlay2/45a00ee99f516848e66424a0c4ea665aa3642b480bee4420d4eb3e8443edd8b6/merged\\\" at \\\"/var/lib/docker/overlay2/45a00ee99f516848e66424a0c4ea665aa3642b480bee4420d4eb3e8443edd8b6/merged/blockchain-explorer/config.json\\\" caused \\\"not a directory\\\"\""
cello-operator-dashboard | : Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
cello-operator-dashboard |
cello-operator-dashboard | ERROR: for explorer Cannot start service explorer: oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:359: container init caused \"rootfs_linux.go:54: mounting \\\"/opt/cello/fabric-1.0/explorer-artifacts/config.json\\\" to rootfs \\\"/var/lib/docker/overlay2/45a00ee99f516848e66424a0c4ea665aa3642b480bee4420d4eb3e8443edd8b6/merged\\\" at \\\"/var/lib/docker/overlay2/45a00ee99f516848e66424a0c4ea665aa3642b480bee4420d4eb3e8443edd8b6/merged/blockchain-explorer/config.json\\\" caused \\\"not a directory\\\"\""
cello-operator-dashboard | : Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
cello-operator-dashboard |
cello-operator-dashboard | ERROR: for cli Cannot start service cli: oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:359: container init caused \"rootfs_linux.go:54: mounting \\\"/opt/cello/fabric-1.0/solo/configtx.yaml\\\" to rootfs \\\"/var/lib/docker/overlay2/0b7660a66e27b270a892f4324868bdc5c2784c98cadc5fa95cb4ea6099eaa105/merged\\\" at \\\"/var/lib/docker/overlay2/0b7660a66e27b270a892f4324868bdc5c2784c98cadc5fa95cb4ea6099eaa105/merged/etc/hyperledger/fabric/configtx.yaml\\\" caused \\\"not a directory\\\"\""
cello-operator-dashboard | : Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
Can anyone help me? Thanks in advance!
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=Jm3fWQXZEXQsnSpCp) @baohua am on v0.8.0 currently.
sorry for the delayed response
@tongli I've submitted the patch on Gerrit for updating a bna via ansible agent
Would you mind taking a look to see if I'm doing ti correctly? It's my first time contributing!
Would you mind taking a look to see if I'm doing it correctly? It's my first time contributing!
Would you mind taking a look to see if I'm doing it correctly?
https://gerrit.hyperledger.org/r/#/c/22373/
@DerHolzloeffel suppose u're using the docker host, then need to run the setup cmd at the worker nodes with following http://cello.readthedocs.io/en/latest/setup/
@JordanDearsley ci not passed, pls check it's log errors.
@patrickgardella : https://gerrit.hyperledger.org/r/#/c/20737/, would you like to help revise the pr according to the review? Thanks!
@baohua Thanks for the response. I did all the steps according to the tutorial. I figured the problem was within the setup_docker_worker_node.sh. The mount at the end of the script didnt work until i added the line MASTER_NODE_IP=... Then it worked.
[ ](https://chat.hyperledger.org/channel/cello?msg=G2e9njsAHDxq8n3Yw) @baohua I think it was due to the commit. I submitted a second patch, but how to I get it retested?>
[ ](https://chat.hyperledger.org/channel/cello?msg=G2e9njsAHDxq8n3Yw) @baohua I think it was due to the commit. I submitted a second patch, but how to I get it retested?
Has joined the channel.
HI ,I met a strange issue when I install cello using docker damen way. When I want to a a host in the cello dashboard it report the can;t not add I found the error log
cello-operator-dashboard
| [2018-05-28 14:27:18,717] ERROR [agent.docker.docker_sw arm] [docker_swarm.py:173 check_daemon()] - Exception in check_daemon Erro r while fetching server API version: HTTPConnectionPool(host='209.97.130.89', port =2357): Max retries exceeded with url: /version (Caused by NewConnectionError('
HI ,I met a strange issue when I install cello using docker way. When I want to a a host in the cello dashboard it report the I can;t add new host.
I found the error log
| [2018-05-28 14:27:18,717] ERROR [agent.docker.docker_sw arm] [docker_swarm.py:173 check_daemon()] - Exception in check_daemon Erro r while fetching server API version: HTTPConnectionPool(host='209.97.130.89', port =2357): Max retries exceeded with url: /version (Caused by NewConnectionError('
I have setup the docker dameon for the worker node,aslo I can use docker -H 209.97.130.8:2375 info get response.What is the possible issue I miss?
@baohua Resubmitted and verified https://gerrit.hyperledger.org/r/#/c/22373/
@guangyingyuan - can you try - http://cello.readthedocs.io/en/latest/setup_worker_docker/
Ubuntu 16.04 - this part
@baohua - can you please help me with my issues which am facing while invoking chiancode
https://jira.hyperledger.org/browse/CE-375
Hey there, wanted to know if there is a basefabric VDI image to use? If not, how do we configure the network settings on VirtualBox? I have the ansible script starting the instances, but nothing past "Wait until the servers are up and running"
@DerHolzloeffel great, would you like to help highlight that trick in the tutorial? https://cello.readthedocs.io/en/latest/CONTRIBUTING/
@guangyingyuan what is ur docker version? The related code is at https://github.com/hyperledger/cello/blob/master/src/agent/docker/docker_swarm.py#L278.
@JordanDearsley thanks, @tongli would you like to help review https://gerrit.hyperledger.org/r/#/c/22373/?
@phulei i feel that you u're using wrong version of sdk. Is it the latest code? @hightall ,can you help see https://jira.hyperledger.org/browse/CE-375?
@baohua The docker version should be Server Version: 18.03.1-ce,aslo I try that link http://cello.readthedocs.io/en/latest/setup_worker_docker/
Has joined the channel.
unable to add a host Request given this Error 400: http://ec2-35-171-156-102.compute-1.amazonaws.com:8080/api/host
The request was issued with an error. The server did not create or modify data.
When I was trying to add host in the dashboard I was facing this error: I have master running in one IP and worker on other IP. Request Error 400: http://192.168.1.196:8080/api/host
The request was issued with an error. The server did not create or modify data
Could some one please suggest me on this
When I checked the network it looked like
sudo docker network ls
NETWORK ID NAME DRIVER SCOPE
de89e89662e6 bridge bridge local
945b11ad23b9 host host local
33c32e0e4c15 none null local
Does cello creates a network ? if so why the network is not created here? Do I get some suggestions
Can I get some guidelines or links to start Cello from scratch.
Thank you in advance.
Has joined the channel.
Him
I have followed instructions from *https://github.com/hyperledger/cello* and setted up master node with out any error, now when I am trying to access the operator dashboard with myIP:8080/index nothing is coming up?
Any body can help?
@Vanitha not sure what your goal is. if you like to stand up a fabric network on multiple nodes, you probably can try the ansible agent which makes that effort a lot easier especially we have created anisible agent docker container which help you to get all the stuff you need.
@Vanitha The problem is that ansible agent sets up fabric network which is not integrated with cello dashboard.
so that you do not see the things visually.
you will have to use command lines tool to do things.
but if you really want to learn fabric, I think that is the way to go.
@tongli Can I get some links to start working on Cello. I also went through video on youtube I was trying to establish communication between master and worker node. I am in initial stage of learning.
you mean to start contributing to cello or start using ansible agent?
Cello
look into the cello wiki page please.
I guess first I need to have a clear picture on cello and then ca move to ansible
I guess first I need to have a clear picture on cello and then can move to ansible
ca?
https://wiki.hyperledger.org/projects/cello
wiki page
I followed the same link and while adding the host I was getting the error is the docker deamon running succesfully
in that case, it seems to me that you are already down the different path than using ansible agent.
That is where I was stuck. I have master on VM and worker running on my local Ubantu machine
@baohua and others probably can provide more help.
if you were using ansible agent, I will be able to help a bit more.
Okay I will start working with Ansible part too, but would like to know y this is not happening to me
Thank you @tongli
just let you know, you do not have to use any other part of cello if you use ansible agent.
it is almost like a complete independent project and it uses ansible playbooks to setup everything.
@tongli I have a .bna file and I have deployed that into fabric network can I connect Ansible part to that fabric network?
ansible agent is the tool to stand up fabric network. once the network is up running, it does not do anything more. its job is done.
if I connect my up end running Fabric part with the Ansible(that means I can have multiple organizations on multiple VM's right)?
not sure what you mean "my up end running"
It's running successfully without any errors
https://github.com/hyperledger/cello/blob/master/docs/worker_ansible_howto.md @Vanitha
Thank you @tongli
@tongli would you mind reviewing my patch if you get the chance? Thanks!
@JordanDearsley yes, I will. thanks.
@JordanDearsley looks ok.
this is simply do an update, not necessarily adding a new bna, right?
I was hoping that you can add some scripts to allow new bna to be deployed or improve what we have.
approved. need one more +2 though to merge it.
please ask Baohua to do that.
@tongli I believe that the update will replace the existing bna. So you could put a different bna file in secrets/networks and it should update the same. Do you mean creating a separate new business network on the same nodes? If I find the time I'll write that as well
@baohua Would you mind +2 to merge?
right.
It's just `composer install` and `composer start`, shouldn't be too bad
adding completely new bna. which I think playbook already does it, but may need a bit doc, and test.
like I mentioned before once the network is running, it already has carauction going.
Yeah it may, I'll have to check on that
if someone wants to add more bna
Your command from last week may work then
I am pretty sure it does that already, just need a bit more doc.
it should.
I'll test it out when I get the chance
Has joined the channel.
Has joined the channel.
Not sure I am getting this issue intermittently - TASK [Gathering Facts] *********************************************************
fatal: [cloud]: FAILED! => {"changed": false, "module_stderr": "sudo: a password is required\n", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 1}
to retry, use: --limit @/home/snap/hyperledger-stuff/cello-master/src/agent/ansible/awsk8s.retry
Has joined the channel.
Hi All, I am wanted to try cello to setup fabric network in a set of VMs that I have created in different machines. How to setup the fabric network on those VMs using cello?
Has joined the channel.
Hi All, I have been Setup a Master Node and setup Docker Host as worker node,After add host and add chain ,the chain will be auto remove, the cello-operator-dashboard error log like this:
cello-operator-dashboard | [2018-05-30 06:03:53 +0000] [10] [ERROR] Error handling request /socket.io/?EIO=3&transport=websocket&sid=2a0e2597a556440f8caee61675480593
cello-operator-dashboard | Traceback (most recent call last):
cello-operator-dashboard | File "/usr/local/lib/python3.5/dist-packages/gunicorn/workers/async.py", line 52, in handle
cello-operator-dashboard | self.handle_request(listener_name, req, client, addr)
cello-operator-dashboard | File "/usr/local/lib/python3.5/dist-packages/gunicorn/workers/async.py", line 112, in handle_request
cello-operator-dashboard | resp.close()
cello-operator-dashboard | File "/usr/local/lib/python3.5/dist-packages/gunicorn/http/wsgi.py", line 418, in close
cello-operator-dashboard | self.send_headers()
cello-operator-dashboard | File "/usr/local/lib/python3.5/dist-packages/gunicorn/http/wsgi.py", line 334, in send_headers
cello-operator-dashboard | tosend = self.default_headers()
Creating 45c304c38f98465388dbf2afe140febd_peer0_org1 ... done
cello-operator-dashboard | elif self.should_close():
cello-operator-dashboard | File "/usr/local/lib/python3.5/dist-packages/gunicorn/http/wsgi.py", line 238, in should_close
cello-operator-dashboard | if self.status_code < 200 or self.status_code in (204, 304):
cello-operator-dashboard | AttributeError: 'Response' object has no attribute 'status_code'
cello-operator-dashboard | [2018-05-30 06:03:54,267] INFO [resources.cluster_api] [cluster_api.py:314 cluster_list()] - {}
Creating 45c304c38f98465388dbf2afe140febd_cli ... error
, 'service_url': {}, 'api_url': '', 'service_ports': [], 'worker_api': 'tcp://192.168.1.49:2375', 'network_type': 'fabric-1.0', 'mapped_ports': {'peer1_org1_grpc': 7250, 'ca_org1_ecap': 7850, 'peer0_org2_event': 7550, 'peer0_org2_grpc': 7
450, 'peer1_org2_grpc': 7650, 'peer0_org1_event': 7150, 'peer1_org2_event': 7750, 'peer0_org1_grpc': 7050, 'orderer': 8050, 'ca_org2_ecap': 7950, 'dashboard': 8150, 'peer1_org1_event': 7350}, 'size': 4, 'apply_ts': None, 'name': 'test_cha
in_1', 'duration': 0.0, 'health': '', 'host_id': 'bdab76d474dd4fcb8705f3754dea87db', 'containers': [], 'id': '45c304c38f98465388dbf2afe140febd', 'create_ts': '2018-05-30T06:03:50.832000+00:00'}]
Creating 45c304c38f98465388dbf2afe140febd_explorer ...
Creating 45c304c38f98465388dbf2afe140febd_explorer ... error
cello-operator-dashboard |
cello-operator-dashboard | ERROR: for 45c304c38f98465388dbf2afe140febd_cli Cannot start service cli: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"root
fs_linux.go:58: mounting \\\"/opt/cello/fabric-1.0/solo/configtx.yaml\\\" to rootfs \\\"/var/lib/docker/overlay2/08aaa739bf0832eba4f66b31c455063fac186727e58f3973ae71a41b96775df4/merged\\\" at \\\"/var/lib/docker/overlay2/08aaa739bf0832eba
4f66b31c455063fac186727e58f3973ae71a41b96775df4/merged/etc/hyperledger/fabric/configtx.yaml\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path
exists and is the expected type
cello-operator-dashboard |
cello-operator-dashboard | ERROR: for 45c304c38f98465388dbf2afe140febd_explorer Cannot start service explorer: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init cau
sed \"rootfs_linux.go:58: mounting \\\"/opt/cello/fabric-1.0/explorer-artifacts/config.json\\\" to rootfs \\\"/var/lib/docker/overlay2/8b64eedc67ecb1dd09779a744f157863f981b86c00f479910ab1649bce90d1ef/merged\\\" at \\\"/var/lib/docker/over
lay2/8b64eedc67ecb1dd09779a744f157863f981b86c00f479910ab1649bce90d1ef/merged/blockchain-explorer/config.json\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the spe
cified host path exists and is the expected type
cello-operator-dashboard |
ok,thanks, I got it. nfs mount error,because my system miss /sbin/mount.nfs file, just install nfs-common like this $ sudo apt-get install nfs-common
@sunyoubo thanks for the report
I think this issue should be recorded into FAQ wiki
:grimacing: is my pleasure @Luke_Chen
when execute command: $ MASTER_NODE_IP=xx.xx.xx.xx make setup-worker, the log error like this :family_mwgb:
mount: wrong fs type, bad option, bad superblock on 192.168.1.49:/,
missing codepage or helper program, or other error
(for several filesystems (e.g. nfs, cifs) you might
need a /sbin/mount.
@Luke_Chen you mean recorde into here:https://wiki.hyperledger.org/doku.php?id=projects/cello/faq ? how can do it?
@sunyoubo login and describe your problem and then write down relative solution.
@Luke_Chen I update my words into this page(https://wiki.hyperledger.org/projects/cello/faq) or I report my issue into https://jira.hyperledger.org/projects/CE/issues/ ?
:grimacing:, I am sorry , I am newbie
@sunyoubo Haha it doesn't matter, I think the update of wiki page will be enough
@Luke_Chen OK,thank you.
you are welcome:grimacing:
[ ](https://chat.hyperledger.org/channel/cello?msg=HBEHLZoYrtZptBeBY) I just had to keep trying. It is working fine for me.
[ ](https://chat.hyperledger.org/channel/cello?msg=qSw7xKY6B7PSshDmL) @sunyoubo Thanks just fixed my issue
@Amir you are welcome:joy:
HI Heroes: when I add chaincode use dashboard,just select one .go file, but my chaincode has more than one file, and as well more than one packet, so how can I install my chaincode?
when I just select the main.go, the main can't import packet, so how can I let the chaincode container know all .go files? In other words, how can I mapping my chaincode into container?
[ ](https://chat.hyperledger.org/channel/cello?msg=hJdpBJefgMGDHfW6A) @rkrish82 follow this doc (https://github.com/hyperledger/cello/blob/master/docs/worker_ansible_howto.md#use-the-existing-servers), there is a section about "Using existing servers"
@baohua Would you mind taking a look to +2 on this? https://gerrit.hyperledger.org/r/#/c/22373/ Thanks!
@JordanDearsley merged, thanks and congrats for ur first contribution to cello! https://gerrit.hyperledger.org/r/#/c/22373/
Hi All, If I want to deploy fabric on physical servers then which configuration file should I use for ansible agent?
Has joined the channel.
Has joined the channel.
Has joined the channel.
Has joined the channel.
@rkrish82 you can if you want to
There is a section in the ansible agent doc specifically talk about that
@baohua Thanks!
@tongli Do you know of a way to pause an AWS network (stop its instances) without terminating it completely?
So it can be started back up
can anyone take a look at this error
cello-operator-dashboard | ERROR: for a2eb120d42664f179d04c6d531b0c18e_cli Cannot start service cli: OCI runtime create failed: container_linux.go:348: starting co
ntainer process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/opt/cello/fabric-1.0/solo/configtx.yaml\\\" to rootfs \\\"/v
ar/lib/docker/overlay2/20856e6825783b862d6ba57913bdce010dd92d6475471cc5bc29c1a09eeccc22/merged\\\" at \\\"/var/lib/docker/overlay2/20856e6825783b862d6ba57913bdce010
dd92d6475471cc5bc29c1a09eeccc22/merged/etc/hyperledger/fabric/configtx.yaml\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto
a file (or vice-versa)? Check if the specified host path exists and is the expected type
the services are keep on deleting for the above reason
@JordanDearsley no, I do not know.
[ ](https://chat.hyperledger.org/channel/cello?msg=PJmnDhG76DNN6bDiJ) @phulei try installing nfs-common in worker node(s) (sudo apt-get install nfs-common)
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=NJ7RwZyMjgGX4Xm7k) @Amir it didn't work for me as well same issue
ERRO[0665] containerd: start container error=oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:359: container init caused \"rootfs_linux.go:54: mounting \\\"/opt/cello/fabric-1.0/solo/configtx.yaml\\\" to rootfs \\\"/var/lib/docker/aufs/mnt/ff55206e4f51a5be6257e58cfdf66aa7e6104390f9b0a5a272a628565be6043b\\\" at \\\"/var/lib/docker/aufs/mnt/ff55206e4f51a5be6257e58cfdf66aa7e6104390f9b0a5a272a628565be6043b/etc/hyperledger/fabric/configtx.yaml\\\" caused \\\"not a directory\\\"\""
id=0e2096545b662a283d03057157a4cc5c84e99a44c3f8ee1848f4547f8df4fd1d
ERRO[0665] Create container failed with error: invalid header field value "oci runtime error: container_linux.go:247: starting container process caused \"process_linux.go:359: container init caused \\\"rootfs_linux.go:54: mounting \\\\\\\"/opt/cello/fabric-1.0/solo/configtx.yaml\\\\\\\" to rootfs \\\\\\\"/var/lib/docker/aufs/mnt/ff55206e4f51a5be6257e58cfdf66aa7e6104390f9b0a5a272a628565be6043b\\\\\\\" at \\\\\\\"/var/lib/docker/aufs/mnt/ff55206e4f51a5be6257e58cfdf66aa7e6104390f9b0a5a272a628565be6043b/etc/hyperledger/fabric/configtx.yaml\\\\\\\" caused \\\\\\\"not a directory\\\\\\\"\\\"\"\n"
Hi All, When I am trying setup fabric network using ansible agent on physical machine, anisble got stuck at TASK [Pull certificates from the builder machine] stage.
TASK [Pull certificates from the builder machine] ***********************************************************************************************************
task path: /home/test/cello/src/agent/ansible/roles/deploy_compose/fabricsetup/tasks/apply.yml:120
Using module file /usr/local/lib/python2.7/dist-packages/ansible/modules/commands/command.py
playbook awsk8s.yml doesnot create public ip 's automatically as shown in the video @tongli ,Are there any additional steps which are required which are not mentioned in the video ? and btw the awsk8s ansible files are different from one in the video any update on that would be a great help.thanks
@chandrakanthMamillapalli it has been creating public ips when vm gets created. I have not recently tried it on aws. It could be some package or api change
which I do not think is the case though.
*sudo systemctl restart docker.service*
Job for docker.service failed because the control process exited with error code. See "systemctl status docker.service" and "journalctl -xe" for details.
While running this command I was getting error like this please help me to resolve this
When trying to setup worker node(docker)
@Vanitha - when you are facing this
i think the config of the docker.service must be messed up
can you share what the vaule there ?
@Amir - still facing the issue ! cello-operator-dashboard | ERROR: for explorer Cannot start service explorer: OCI runtime create failed: container
_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58:
mounting \\\"/opt/cello/fabric-1.0/explorer-artifacts/config.json\\\" to rootfs \\\"/var/lib/docker/overlay2/9ccbd
06b71a41ac753e7b1cc916003042298759975c556adccd7fa9f807c543a/merged\\\" at \\\"/var/lib/docker/overlay2/9ccbd06b71a4
1ac753e7b1cc916003042298759975c556adccd7fa9f807c543a/merged/blockchain-explorer/config.json\\\" caused \\\"not a di
rectory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified ho
st path exists and is the expected type
cello-operator-dashboard |
cello-operator-dashboard | ERROR: for cli Cannot start service cli: OCI runtime create failed: container_linux.go:
348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting
\\\"/opt/cello/fabric-1.0/solo/configtx.yaml\\\" to rootfs \\\"/var/lib/docker/overlay2/f3f7021ced24193ffe7ca0405a2
bffc887356999f6f18ee21a1e5874e3c9e949/merged\\\" at \\\"/var/lib/docker/overlay2/f3f7021ced24193ffe7ca0405a2bffc887
356999f6f18ee21a1e5874e3c9e949/merged/etc/hyperledger/fabric/configtx.yaml\\\" caused \\\"not a directory\\\"\"": u
nknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists an
d is the expected type
@phulei `[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target docker.socket firewalld.service
Requires=docker.socket
[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
DOCKER_OPTS="$DOCKER_OPTS -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --api-cors-header='*' --default-ulimit=nofile=8192:16384 --default-ulimit=nproc=8192:16384"
EnvironmentFile=-/etc/default/docker
ExecStart=/usr/bin/dockerd -H fd:// $DOCKER_OPTS
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=1048576
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
`
I was trying to setup cello
ok
can you share systemctl status docker.service
@Vanitha
@phulei status is degraded
Screenshot from 2018-06-01 13-44-27.png
can you remove docker and reinstall
apt-get remove docker-ce ; apt-get install docker-ce
Sure @phulei
Screenshot from 2018-06-01 13-51-34.png
@phulei When checked for version :smiley: sudo docker-compose -v
docker-compose version 1.18.0, build 8dd22a9
@phulei When checked for version sudo docker-compose -v
docker-compose version 1.18.0, build 8dd22a9
that's fine
just remove docker
Removed
when reinstalling it was like above screenshot @phulei
hmm
where you are spinning the vm ? and can you recreate a fresh VM ?
It's on my physical machine
Can we do both master and worker setup on single VM? @phulei
i dont think so
[ ](https://chat.hyperledger.org/channel/cello?msg=A6iiepZpdPuRfH8Rf) doable actually
Okay
you need 2 VMS to run
Can't I run on physical ubantu machine(16.04)
I have master node on one VM
ok
Trying to setup worker on my physical machine
reinstall everything ! as i feel your docker set up messed
sure @phulei
Do I need to uninstall Docker or everything I mean Node and all?
@phulei
only on the worker node
as we need 2375 to listen on the worker node :- as master node want it as it gives instruction to spin the instances there
@Amir - you there ?
[ ](https://chat.hyperledger.org/channel/cello?msg=ku9gZMyx8dgX5b99J) @phulei Yup
still am seeing the those error !
any other inputs ?
Creating 8d1d84177707486590f13581e6f6dd38_cli ... error
cello-watchdog | [2018-06-01 18:15:27,055] INFO [__main__] [watchdog.py:125 watch_run()] - Watchdog run checks with period = 15 s
cello-watchdog | [2018-06-01 18:15:27,059] INFO [modules.host] [host.py:248 list()] - filter data {}
cello-watchdog | [2018-06-01 18:15:27,066] INFO [__main__] [watchdog.py:127 watch_run()] - Found 1 hosts
Creating 8d1d84177707486590f13581e6f6dd38_peer1_org2 ...
cello-operator-dashboard | Creating 8d1d84177707486590f13581e6f6dd38_peer0_org1 ...
Creating 8d1d84177707486590f13581e6f6dd38_explorer ... error
cello-operator-dashboard | Creating 8d1d84177707486590f13581e6f6dd38_peer1_org1 ...
Creating 8d1d84177707486590f13581e6f6dd38_explorer ...
cello-operator-dashboard | Creating 8d1d84177707486590f13581e6f6dd38_cli ...
cello-operator-dashboard |
cello-operator-dashboard | ERROR: for 8d1d84177707486590f13581e6f6dd38_cli Cannot start service cli: OCI runtime create failed: container_linux.go:348: starting co
ntainer process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/opt/cello/fabric-1.0/solo/configtx.yaml\\\" to rootfs \\\"/v
ar/lib/docker/overlay2/5ac57549bf0ef115ea9f60f6c988d0d384a52ad5c73d7d3aafe031c0afc6e8a9/merged\\\" at \\\"/var/lib/docker/overlay2/5ac57549bf0ef115ea9f60f6c988d0d38
4a52ad5c73d7d3aafe031c0afc6e8a9/merged/etc/hyperledger/fabric/configtx.yaml\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto
a file (or vice-versa)? Check if the specified host path exists and is the expected type
did you tried to clean and re start cello ?
on the worker node ?
ya
doing it
clean '/opt/cello'
then start he worker "MASTER_NODE_IP=xx.xx.xx.xx make setup-worker"
then start he worker "MASTER_NODE_IP=xx.xx.xx.xx make setup-worker" - i have to executed on the master node ?
this should go in worker node
yes in the worker node
ok done ; creating the chain
before that do you want me to restart master node ?
what is the final output of this ""MASTER_NODE_IP=xx.xx.xx.xx make setup-worker" ? does it say any error ?
no error
Downloading fabric images from DockerHub...with tag = 1.0.5... need a while
hyperledger/fabric-peer:x86_64-1.0.5 already exist locally
hyperledger/fabric-tools:x86_64-1.0.5 already exist locally
hyperledger/fabric-orderer:x86_64-1.0.5 already exist locally
hyperledger/fabric-ca:x86_64-1.0.5 already exist locally
hyperledger/fabric-ccenv:x86_64-1.0.5 already exist locally
===Re-tagging fabric images to *:1.0.5* tag
Downloading images for fabric explorer
mysql:5.7 already exist locally
yeasy/blockchain-explorer:0.1.0-preview already exist locally
Downloading and retag images for kafka/zookeeper separately, as they are still v1.0.4
hyperledger/fabric-kafka:x86_64-1.0.4 already exist locally
hyperledger/fabric-zookeeper:x86_64-1.0.4 already exist locally
Done, now worker node should have all required images, use 'docker images' to check
Copy required fabric 1.0 artifacts
Checking local artifacts path /opt/cello...
Local artifacts path /opt/cello not existed, creating one
Mount NFS Server 10.142.0.2
Setup ip forward rules
net.ipv4.ip_forward = 1
Setup done
run this before starting the chain "sysctl -w net.ipv4.ip_forward=1"
done
creating chain
@Amir :woo:
O
i got the problem ; it was not mounting the nfs serever
hmmm
and the command made sure to mount the nfs
as we are using the nfs to store all the confis
as we are using the nfs to store all the configs
Thank you !
is it the same error as the previous one ?
yes it is
is it working now ? can you create a chain ?
yes ! it is working and and chain status is healthy
:thumbsup:
please add this to the FAQ
I think it is added when it was reported
@chandrakanthMamillapalli - you need to do "MASTER_NODE_IP=xx.xx.xx.xx make setup-worker" to overcome from the nfs error
ok !
i was having one more problem where i was not able to invoke
let me chck that as well
I never used the interface to invoke, I do it from script
there was some issue with the invoking GUI, not sure if it is fixed
hmm
under `src/agent/docker/_compose_files/fabric-1.0/scripts` there are scripts
CE-375
ok
they are called when you instantiate a chain from docker worker
you can customize them to your need
ok , let me check
are you from india ?
@phulei I have completely uninstalled docker and docker-compose. When I was trying to install again it's throwing errors.
invoke-rc.d: initscript docker, action "start" failed.
● docker.service - Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
Active: activating (auto-restart) (Result: exit-code) since Fri 2018-06-01 14:58:44 EDT; 3ms ago
Docs: https://docs.docker.com
Process: 24706 ExecStart=/usr/bin/dockerd -H fd:// (code=exited, status=1/FAILURE)
Main PID: 24706 (code=exited, status=1/FAILURE)
Jun 01 14:58:44 miracle systemd[1]: Failed to start Docker Application Cont...e.
Jun 01 14:58:44 miracle systemd[1]: docker.service: Unit entered failed state.
Jun 01 14:58:44 miracle systemd[1]: docker.service: Failed with result 'exi...'.
Hint: Some lines were ellipsized, use -l to show in full.
dpkg: error processing package docker-ce (--configure):
subprocess installed post-installation script returned error exit status 1
Processing triggers for libc-bin (2.23-0ubuntu10) ...
Processing triggers for systemd (229-4ubuntu21.1) ...
Processing triggers for ureadahead (0.100.0-19) ...
Errors were encountered while processing:
docker-ce
E: Sub-process /usr/bin/dpkg returned an error code (1)
ok
share the docker.service file
`[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target docker.socket firewalld.service
Wants=network-online.target
Requires=docker.socket
[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
ExecStart=/usr/bin/dockerd -H fd://
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=1048576
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
# restart the docker process if it exits prematurely
Restart=on-failure
StartLimitBurst=3
StartLimitInterval=60s
[Install]
WantedBy=multi-user.target
`
@phulei
But when I checked version it's showing the version
remove all the content of that file
Like it is throwing error at *ExecStart=/usr/bin/dockerd -H fd://*
update it accordingly
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target docker.socket firewalld.service
Wants=network-online.target
Requires=docker.socket
[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
EnvironmentFile=-/etc/default/docker
ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --api-cors-header='*' --de
fault-ulimit=nofile=8192:16384 --default-ulimit=nproc=8192:16384
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=1048576
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
# restart the docker process if it exits prematurely
Restart=on-failure
StartLimitBurst=3
StartLimitInterval=60s
[Install]
WantedBy=multi-user.target
sudo systemctl daemon-reload
and docker restart
[ ](https://chat.hyperledger.org/channel/cello?msg=x8H9zqdQprLebYvrR) @phulei u mean to say docker.service?
Still same error
sudo systemctl restart docker.service
Job for docker.service failed because the control process exited with error code. See "systemctl status docker.service" and "journalctl -xe" for details.
[ ](https://chat.hyperledger.org/channel/cello?msg=4LpKXNXFGcqf94gQ4) @baohua @hightall - still facing and havw checkout master branch , please have a look .
[ ](https://chat.hyperledger.org/channel/cello?msg=4LpKXNXFGcqf94gQ4) @baohua @hightall - still facing the problem and have checkedout master branch , please have a look .
Has joined the channel.
Clipboard - June 2, 2018 5:20 PM
Clipboard - June 2, 2018 5:20 PM
@tongli
Clipboard - June 2, 2018 8:19 PM
Clipboard - June 2, 2018 8:19 PM
Clipboard - June 2, 2018 9:56 PM
did anyone face this issue ?
Screenshot 2018-06-02_23-52-21.png
containers stop abruptly and restarts again
could not able to Apply New Chain
@Luke_Chen I test the master branch code on the new empty environment in development mode, and it's fine. the step are: make docker; make build-admin-js; DEV=True make start
After login from Operator dashboard, I can't see any thing I started getting this in the log
error.png
I'm I missing some file in the template ? (I did pull the latest version)
Has joined the channel.
Hi I am new to Cello. Had some problems with installation on Mac. Could someone help me with it. I got the following error
WARNING: The WEBROOT variable is not set. Defaulting to a blank string.
WARNING: Some services (mongo) use the 'deploy' key, which will be ignored. Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm.
Starting cello-dashboard_mongo ... error
Starting cello_nfs_1 ...
Starting cello-operator-dashboard ... done
Starting cello-watchdog ... done
Starting cello-mongo ... done
Starting cello-engine ... done
ERROR: for cello-dashboard_mongo Cannot start service dashboard_mongo: error while creating mount source path '/opt/cello/dashboard_mongo': mkdir /opt/cello/dashboard_mongo: permission denied
ERROR: for dashboard_mongo Cannot start service dashboard_mongo: error while creating mount source path '/opt/cello/dashboard_mongo': mkdir /opt/cello/dashboard_mongo: permission denied
ERROR: Encountered errors while bringing up the project.
@suryasuresh06 - sudo make start - can you try
same error
ERROR: for cello-user-dashboard Cannot start service user-dashboard: b'Mounts denied: \r\nThe path /opt/cello/baas\r\nis not shared from OS X and is not known to Docker.
am getting this error
but unable to change preferences in docker to /opt/cello
but unable to change preferences in docker to /opt/cello
Ok , you are running on Mac , I see - go to docket preferences and provide mount point there - /opt:cello
Ok , you are running on Mac , I see - go to docket preferences and provide mount point there - /opt/cello
Has joined the channel.
I just cloned the beta version 8 of the repo and ran sudo make start and created the docker containers, unfortunately the url won't resolve in my browser on cello.ink:8080 or localhost:8080 and its the same for 8081, can anyone advise? Im using a ssh ubuntu 16 environment.. any help appreciated..
@hightall - could you please let me know how to install a package
@hightall - could you please let me know how to install "grpc " package
am following up - CE-375
[ ](https://chat.hyperledger.org/channel/cello?msg=4LpKXNXFGcqf94gQ4) @baohua - .
[ ](https://chat.hyperledger.org/channel/cello?msg=4LpKXNXFGcqf94gQ4) @hightall .
Hi All, Got following error while setting up fabric on multiple hosts - (item=orgb) => {"changed": true, "cmd": ["docker", "run", "-v", "/home/test/.composer:/home/composer/.composer", "-v", "/opt/gopath/vb1st/fabric:/opt/gopath/vb1st/fabric", "hyperledger/composer-cli:next", "runtime", "install", "-c", "PeerAdmin@orgb", "-n", "carauction-network"], "delta": "0:05:07.350317", "end": "2018-06-05 08:56:14.019511", "item": "orgb", "msg": "non-zero return code", "rc": 1, "start": "2018-06-05 08:51:06.669194", "stderr": "✖ Installing runtime for business network carauction-network. This may take a minute...", "stderr_lines": ["✖ Installing runtime for business network carauction-network. This may take a minute..."], "stdout": "Error: Error trying install composer runtime. Error: No valid responses from any peers.\nResponse from attempted peer comms was an error: Error: REQUEST_TIMEOUT\nResponse from attempted peer comms was an error: Error: REQUEST_TIMEOUT\n\nCommand failed", "stdout_lines": ["Error: Error trying install composer runtime. Error: No valid responses from any peers.", "Response from attempted peer comms was an error: Error: REQUEST_TIMEOUT", "Response from attempted peer comms was an error: Error: REQUEST_TIMEOUT", "", "Command failed"]}
This is setup ansible command - ansible-playbook -i run/runhosts -e "mode=apply env=vb1st" setupfabric.yml
@hightall OK Thanks.
[ ](https://chat.hyperledger.org/channel/cello?msg=GJAaQNHpKcrPub3qJ) Installation is failing for https://github.com/hyperledger/cello/blob/master/src/agent/ansible/roles/deploy_compose/composersetup/tasks/apply.yml#L60
docker run -v /home/test/.composer:/home/composer/.composer -v /opt/gopath/vb1st/fabric:/opt/gopath/vb1st/fabric hyperledger/composer-cli:next runtime install -c "PeerAdmin@orga" -n carauction-network
Installing runtime for business network carauction-network. This may take a minute...
Error: Error trying install composer runtime. Error: No valid responses from any peers.
Response from attempted peer comms was an error: Error: REQUEST_TIMEOUT
Response from attempted peer comms was an error: Error: REQUEST_TIMEOUT
Command failed
The installation fails for me with the same above error , when i check the peer logs it is not able to connect to couch db
when i check the couch db logs found the below error
[error] 2018-06-05T11:05:50.190825Z nonode@nohost emulator -------- Error in process <0.454.0> with exit value:
{database_does_not_exist,[{mem3_shards,load_shards_from_db,"_users",[{file,"src/mem3_shards.erl"},{line,403}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,378}]},{mem3_shards,load_shards_from_disk,2,[{file,"src/mem3_shards.erl"},{line,407}]},{mem3_shards,for_docid,3,[{file,"src/mem3_shards.erl"},{line,91}]},{fabric_doc_open,go,3,[{file,"src/fabric_doc_open.erl"},{line,38}]},{chttpd_auth_cache,ensure_auth_ddoc_exists,2,[{file,"src/chttpd_auth_cache.erl"},{line,187}]},{chttpd_auth_cache,listen_for_changes,1,[{file,"src/chttpd_auth_cache.erl"},{line,134}]}]}
Is there any resolution for this error?
Has joined the channel.
@hightall @baohua Please review this patch which fixed a bug for ansible agent. We are working to support deploy fabric onto any existing k8s cluster like google k8s engine and ibm container services.
this patch fixed couple errors. https://gerrit.hyperledger.org/r/22799
i am getting this error while creating genesis block
export FABRIC_CFG_PATH=/opt/gopath/bc1st/fabric && /opt/gopath/bc1st/src/github.com/hyperledger/fabric/build/bin/configtxgen -profile OrdererGenesis -outputBlock keyfiles/genesis.block -channelID orderersystemchannel && /opt/gopath/bc1st/src/github.com/hyperledger/fabric/build/bin/configtxgen -profile OrgChannel -outputCreateChannelTx keyfiles/firstchannel.tx -channelID firstchannel
2018-06-06 02:04:11.973 UTC [common/configtx/tool] main -> INFO 001 Loading configuration
2018-06-06 02:04:11.988 UTC [common/configtx/tool/localconfig] Load -> CRIT 002 Error unmarshaling config into struct: 21 error(s) decoding:
* '' has invalid keys: capabilities, channel
* 'Application' has invalid keys: Policies
* 'Application.Organizations[0]' has invalid keys: Policies
* 'Application.Organizations[1]' has invalid keys: Policies
* 'Orderer' has invalid keys: Capabilities, Policies
* 'Orderer.Organizations[0]' has invalid keys: Policies
* 'Orderer.Organizations[1]' has invalid keys: Policies
* 'Organizations[0]' has invalid keys: Policies
* 'Organizations[1]' has invalid keys: Policies
* 'Organizations[2]' has invalid keys: Policies
* 'Organizations[3]' has invalid keys: Policies
* 'Profiles[OrdererGenesis]' has invalid keys: Capabilities, Policies
* 'Profiles[OrdererGenesis].Consortiums[FabricConsortium].Organizations[0]' has invalid keys: Policies
* 'Profiles[OrdererGenesis].Consortiums[FabricConsortium].Organizations[1]' has invalid keys: Policies
* 'Profiles[OrdererGenesis].Orderer' has invalid keys: Capabilities, Policies
* 'Profiles[OrdererGenesis].Orderer.Organizations[0]' has invalid keys: Policies
* 'Profiles[OrdererGenesis].Orderer.Organizations[1]' has invalid keys: Policies
* 'Profiles[OrgChannel]' has invalid keys: Capabilities
* 'Profiles[OrgChannel].Application' has invalid keys: Policies
* 'Profiles[OrgChannel].Application.Organizations[0]' has invalid keys: Policies
* 'Profiles[OrgChannel].Application.Organizations[1]' has invalid keys: Policies
[ ](https://chat.hyperledger.org/channel/cello?msg=LZs92Mnqyry94dW3u) @kiranarshakota I am not choose the couchdb. peer_db is configured with default goleveldb. I am getting error while cli tries to connect to orderer.
{"log":"Error: failed to create deliver client: orderer client failed to connect to orderer2nd-orgc:7050: failed to create new connection: context deadline exceeded\n","stream":"stderr","time":"2018-06-06T03:55:50.88451472Z"}
facing this error while launching new chain through cello dashboard:
ERRO[2018-06-06T09:02:44.149205172Z] stream copy error: reading from a closed fifo
ERRO[2018-06-06T09:02:44.149273618Z] stream copy error: reading from a closed fifo
@hightall @baohua Please review this patch which fixed a bug for ansible agent. We are working to support deploy fabric onto any existing k8s cluster like google k8s engine and ibm container services.
Has joined the channel.
Screen Shot 2018-06-06 at 10.14.18 AM.png
Copying the text so it is more readable: Hi everyone, it looks like the most recent commit (85b39745e90a24a6ded4ce662e50e88954f9c0d6) to the Cello repo is messing something up with Composer. Running the most recent commit yields an error that looks like it has something to do with the generated configtx.yml file. (That recent commit does have modifications to the configtx.j2 file) Doing a git reset one commit back (to 9162c3dbdc47ebc270d3c07a4b932f2946504ba1) and re-running the playbooks is successful. Below is a screenshot of the error. Has anyone else had this problem?
@hightall What should I do If I want to update some forms in UI, I have noticed that it is not enough to just update the ```cello/src/static/dashboard/src/routes/Host/CreateHost/index.js``` file
@hightall What should I do If I want to update some forms in UI, I have noticed that it is not enough to just update the file
```cello/src/static/dashboard/src/routes/Host/CreateHost/index.js```
Hello. I'm using ansible + AWS to deploy Fabric network. Cello 0.8.0. Could you briefly explain to me how do you create docker overlay network? I'm familiar with creating an overlay network using etcd or flannel+etcd. But I see that you use flannel+etc+skydns+registrator. What is the purpose of skydns and registrator? I can't find any information about how to build docker network using this tools. The reason I asking you is that I want to improve this overlay network as it is not a fault tolerant.
BTW do you have any guide how I can contribute to the Cello project? I have some changes related to deploying CA cluster.
I've found the guide how to contribute
@krabradosty ansible agent uses flanneld to set up overlay network so that all fabric nodes can communicate with each other regardless which node they will be created on. It also sets up a dns server so that fabric nodes (peers, orderers,etc) can all reference names to reach each other.
@krabradosty not sure why you are saying it is not fault tolerant.
can you provide a bit more info why you make that conclusion? thx
@tongli Because of this:
```
# only have skydns container running on the first node
- name: Start skydns service
command: >-
docker run -d --name skydns.{{ node_name }} --net=host --restart always -p 53:53/udp
skynetservices/skydns:2.5.3a -machines {{ etcd_cluster }}
-nameservers {{ nameservers.stdout_lines | join(',') }}
-addr {{ node_ip }}:53 -local {{ node_name }}.{{ domain }}
when: node_ip == hostvars[groups['allnodes'][0]].private_ip
```
so you want to have multiple instances of dns?
I think yes
sure, you could start many
skydns uses etcd as backend
which is already by default use 3 nodes.
you can start one skydns per physical node if you want to.
I will try, thanks
there is nothing prevent you from doing that.
but you may have to change your docker configuration so the docker dns will point to your own dns
do not know if you can point to multiple dns servers
normally the dns load should be really light, that is the reason why only one was setup.
@tongli I have 4 nodes where fabric network is deployed. On the fabric001 I executed `./etcdctl ls /skydns/fabricnet` and got output:
```
/skydns/fabricnet/fabric004
/skydns/fabricnet/zookeeper2nd
/skydns/fabricnet/kafka2nd
/skydns/fabricnet/peer1st-main
/skydns/fabricnet/fabriccli
/skydns/fabricnet/prometheus
/skydns/fabricnet/fabric001
/skydns/fabricnet/ca1st-main
/skydns/fabricnet/zookeeper1st
/skydns/fabricnet/zookeeper3rd
/skydns/fabricnet/zookeeper4th
/skydns/fabricnet/kafka3rd
/skydns/fabricnet/kafka4th
/skydns/fabricnet/orderer1st-orderer
/skydns/fabricnet/fabric002
/skydns/fabricnet/peer3rd-main
/skydns/fabricnet/cadvisor
/skydns/fabricnet/orderer2nd-orderer
/skydns/fabricnet/orderer4th-orderer
/skydns/fabricnet/gallant_williams
/skydns/fabricnet/0
/skydns/fabricnet/fabric003
/skydns/fabricnet/orderer3rd-orderer
/skydns/fabricnet/peer2nd-main
/skydns/fabricnet/node-exporter
/skydns/fabricnet/kafka1st
```
Then I switched off my fabric002 VM for a while. I execute same command again:
```
/skydns/fabricnet/zookeeper4th
/skydns/fabricnet/kafka3rd
/skydns/fabricnet/kafka4th
/skydns/fabricnet/peer3rd-main
/skydns/fabricnet/fabric002
/skydns/fabricnet/ca1st-main
/skydns/fabricnet/cadvisor
/skydns/fabricnet/zookeeper3rd
/skydns/fabricnet/fabric003
/skydns/fabricnet/orderer4th-orderer
/skydns/fabricnet/0
/skydns/fabricnet/node-exporter
/skydns/fabricnet/orderer3rd-orderer
/skydns/fabricnet/fabric001
/skydns/fabricnet/fabric004
/skydns/fabricnet/prometheus
```
Entries for fabric002 containers are disappeared. I expected this. But what about fabric001 containers? It's strange. After I turned on fabric002, entries for both of nodes have been added to the etcd.
Also if I switched off all the nodes for a while, Fabric network will never get up by itself.
That is the fault tolerance I talked about. I'm going to investigate this problem. Will let you know.
@tongli I have 4 nodes where fabric network is deployed. On the fabric001 I executed `./etcdctl ls /skydns/fabricnet` and got output:
```
/skydns/fabricnet/fabric004
/skydns/fabricnet/zookeeper2nd
/skydns/fabricnet/kafka2nd
/skydns/fabricnet/peer1st-main
/skydns/fabricnet/fabriccli
/skydns/fabricnet/prometheus
/skydns/fabricnet/fabric001
/skydns/fabricnet/ca1st-main
/skydns/fabricnet/zookeeper1st
/skydns/fabricnet/zookeeper3rd
/skydns/fabricnet/zookeeper4th
/skydns/fabricnet/kafka3rd
/skydns/fabricnet/kafka4th
/skydns/fabricnet/orderer1st-orderer
/skydns/fabricnet/fabric002
/skydns/fabricnet/peer3rd-main
/skydns/fabricnet/cadvisor
/skydns/fabricnet/orderer2nd-orderer
/skydns/fabricnet/orderer4th-orderer
/skydns/fabricnet/gallant_williams
/skydns/fabricnet/0
/skydns/fabricnet/fabric003
/skydns/fabricnet/orderer3rd-orderer
/skydns/fabricnet/peer2nd-main
/skydns/fabricnet/node-exporter
/skydns/fabricnet/kafka1st
```
Then I switched off my fabric002 VM for a while. I execute the same command again:
```
/skydns/fabricnet/zookeeper4th
/skydns/fabricnet/kafka3rd
/skydns/fabricnet/kafka4th
/skydns/fabricnet/peer3rd-main
/skydns/fabricnet/fabric002
/skydns/fabricnet/ca1st-main
/skydns/fabricnet/cadvisor
/skydns/fabricnet/zookeeper3rd
/skydns/fabricnet/fabric003
/skydns/fabricnet/orderer4th-orderer
/skydns/fabricnet/0
/skydns/fabricnet/node-exporter
/skydns/fabricnet/orderer3rd-orderer
/skydns/fabricnet/fabric001
/skydns/fabricnet/fabric004
/skydns/fabricnet/prometheus
```
Entries for fabric002 containers are disappeared. I expected this. But what about fabric001 containers? It's strange. After I turned on fabric002, entries for both of nodes have been added to the etcd.
Also if I switched off all the nodes for a while, Fabric network will never get up by itself.
That is the fault tolerance I talked about. I'm going to investigate this problem. Will let you know.
the problem is that these fabric containers won't start automatically when machine they run on restart
you can certainly improve that.
by using restart policies
@tongli - can you please help me with this ticket - https://jira.hyperledger.org/browse/CE-375?
am kind of stuck on this part
I am trying to create a cluster (Chain) within a Host using hyperledger-cello, but got error during the process. When containers created below error is thrown in Master node log and containers stoped and removed immediately
ERRO[0358] containerd: start container error=oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:359: container init caused \"rootfs_linux.go:54: mounting \\\"/opt/cello/fabric-1.0/solo/configtx.yaml\\\" to rootfs \\\"/var/lib/docker/aufs/mnt/c2676fd25b73972d4844ccd931311ec90ce423e37559d3f249e778f08af94528\\\" at \\\"/var/lib/docker/aufs/mnt/c2676fd25b73972d4844ccd931311ec90ce423e37559d3f249e778f08af94528/etc/hyperledger/fabric/configtx.yaml\\\" caused \\\"not a directory\\\"\""
Has left the channel.
https://gerrit.hyperledger.org/r/#/c/22845/
Has joined the channel.
I've already done it. First of all.
@Luke_Chen add my comments, pls help check: https://gerrit.hyperledger.org/r/#/c/22845/, thanks!
hi guys, is there anything useful for getting cello run with swarm as backend? would like to setup a poc bcaas
Has joined the channel.
@vanitha
@dklesev yes, maybe can look at http://cello.readthedocs.io/en/latest/setup/
@here Who will go to attend the Amsterdam Hackfest? https://docs.google.com/document/d/1hDlWTKSBmXM6UQW5s9qRjFwO_eZv0LU8nppHqMwoIxM/edit
We will have the weekly meeting this Friday, feel free to add the agenda at https://wiki.hyperledger.org/projects/cello/meetings/20180607.
We will have the weekly meeting this Friday, feel free to add the agenda at https://wiki.hyperledger.org/projects/cello/meetings/20180608.
@tongli I found out the reason of why etcd doesn't have entries for containers of two nodes if only one node is switched off:
```
- name: Start registrator service
command: >-
docker run -d --name registrator.{{ node_name }}
--restart always
-v /var/run/docker.sock:/tmp/docker.sock
email4tong/registrator:v10 -internal -ttl 30 -ttl-refresh 20
-resync 18
skydns2://{{ hostvars[groups['etcdnodes'] | random].private_ip }}:2379/{{ domain }}
```
Registrator configured to send containers information to the one of the etcd nodes, not to the local.
@tongli I found out the reason of why etcd doesn't have entries for containers of two nodes if only one node is switched off:
```
- name: Start registrator service
command: >-
docker run -d --name registrator.{{ node_name }}
--restart always
-v /var/run/docker.sock:/tmp/docker.sock
email4tong/registrator:v10 -internal -ttl 30 -ttl-refresh 20
-resync 18
skydns2://{{ hostvars[groups['etcdnodes'] | random].private_ip }}:2379/{{ domain }}
```
Registrator is configured to send containers information to the one of the etcd nodes, not to the local.
@tongli I found out the reason of why etcd doesn't have entries for containers of two nodes if only one node is switched off:
```
- name: Start registrator service
command: >-
docker run -d --name registrator.{{ node_name }}
--restart always
-v /var/run/docker.sock:/tmp/docker.sock
email4tong/registrator:v10 -internal -ttl 30 -ttl-refresh 20
-resync 18
skydns2://{{ hostvars[groups['etcdnodes'] | random].private_ip }}:2379/{{ domain }}
```
Registrator is configured to send containers information to one of the etcd nodes, not to the local.
@krabradosty since the current setup container only one dns
if you like to improve this, you can create an jira issue and add a flag so that the behavior can be controlled.
Has joined the channel.
@baohua Thanks for +2 !
Has joined the channel.
Hi everyone, new to this, but i just installed Cello using the manual, however when i try to access the web UI using default credential, login doesnt take me to anywhere. Any help would be appreciated.
@LichenShen do u mean the operator dashboard? the default token is `admin:pass`
Has joined the channel.
When the user dashboard creates a channel, it gets all peers of the chain to join the channel. For this it must be accessing the private keys of the admins of each org right?
@baohua I am so sorry that I'll be unable to attend the meeting tonight, I will update the process of Kubernetes agent as soon as possbile
The definition of Chain Size given at http://hyperledger-cello.readthedocs.io/en/latest/terminology/ says that it is the number of nodes. Should it not be number of Peers in the Chain?
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/22845/
@baohua yes, is that token i use for the log in? is there something else i need to install before i install cell (ie fabric?)
oh, you'd better follow http://cello.readthedocs.io/en/latest/setup/
through my experience, reading documentation can save lots of time in following usage.
Today's meeting minutes is available now: https://wiki.hyperledger.org/projects/cello/meetings/20180608.
Has joined the channel.
I am having issues signing into the cello dashboard. I have successfully installed everything and all the docker containers are up and running. I am trying to use username: admin, password: pass, but nothing happens at myip:8080
Has joined the channel.
Has joined the channel.
Hi All, I'm trying to setup a network on a single ubuntu server via ansible scripts provided in this doc http://cello.readthedocs.io/en/latest/setup_worker_ansible_allinone/
I'm getting the error below:```TASK [Runtime install] **********************************************************************************************************************
failed: [10.0.2.15] (item=mad) => {"changed": true, "cmd": ["docker", "run", "-v", "/home/osboxes/.composer:/home/composer/.composer", "-v", "/opt/gopath/bc2nd/fabric:/opt/gopath/bc2nd/fabric", "hyperledger/composer-cli:next", "runtime", "install", "-c", "PeerAdmin@mad", "-n", "carauction-network"], "delta": "0:00:06.886005", "end": "2018-06-10 14:32:30.251394", "item": "mad", "msg": "non-zero return code", "rc": 1, "start": "2018-06-10 14:32:23.365389", "stderr": "✖ Installing runtime for business network carauction-network. This may take a minute...", "stderr_lines": ["✖ Installing runtime for business network carauction-network. This may take a minute..."], "stdout": "Error: Network configuration is missing this client's organization and certificate authority\nCommand failed", "stdout_lines": ["Error: Network configuration is missing this client's organization and certificate authority", "Command failed"]}
to retry, use: --limit @/home/osboxes/cello/src/agent/ansible/setupfabric.retry
```
Please advise what steps I was missing.
```
```
Hi All, Is cello ansible agent provided with proxy support if my VMs are behind proxy..? Please help on this
Has joined the channel.
Hello everyone, i have some questions regarding cello:
- Is it production ready?
- can you use cello to set up channels for hyperledger fabric?
Has joined the channel.
Has joined the channel.
Hi @baohua and @hightall can you verify the patchset https://gerrit.hyperledger.org/r/#/c/22915/ and do the needful
can anyone help with signing into cello dashboard?
@baohua @hightall Please take a look at this patch to allow specification of mem, cpu and storage allocation when deploy fabric onto any k8s cluster. https://gerrit.hyperledger.org/r/#/c/22895/
that is an bit improvement so that one ever deploys fabric can have a bit control how resources can be allocated.
@baohua , does the cello analytics work? I don't see fabric peer exporting any kind of metrics.
@baohua thanks for +2 to the patch
Is there a way to deploy the same fabric network on multiple VM's using Cello
i.e. (1 CA, 1 Order) on server 1 and (1 Peer) on server 2
@nasserfci ansible agent was developed to do just that.
I see ansible agent i think and correct me if am wrong that it is used to deploy fabric network in multiple server i want to divide fabric network in multiple server example to it install each container in server
[ ](https://chat.hyperledger.org/channel/cello?msg=jf5MFizhN2mfsiW3b) @NicholasBaro username:admin , password:pass
[ ](https://chat.hyperledger.org/channel/cello?msg=hahrjpMGjJheNzYOa0) @nasserfci yes it is what it does!
I am trying to create a cluster (Chain) within a Host using hyperledger-cello, but got error during the process. When containers created below error is thrown in Master node log and containers stoped and removed immediately
ERRO[0358] containerd: start container error=oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:359: container init caused \"rootfs_linux.go:54: mounting \\\"/opt/cello/fabric-1.0/solo/configtx.yaml\\\" to rootfs \\\"/var/lib/docker/aufs/mnt/c2676fd25b73972d4844ccd931311ec90ce423e37559d3f249e778f08af94528\\\" at \\\"/var/lib/docker/aufs/mnt/c2676fd25b73972d4844ccd931311ec90ce423e37559d3f249e778f08af94528/etc/hyperledger/fabric/configtx.yaml\\\" caused \\\"not a directory\\\"\""
acing this error while launching new chain through cello dashboard:
ERRO[2018-06-06T09:02:44.149205172Z] stream copy error: reading from a closed fifo
ERRO[2018-06-06T09:02:44.149273618Z] stream copy error: reading from a closed fifo
[error2]facing this error while launching new chain through cello dashboard:
ERRO[2018-06-06T09:02:44.149205172Z] stream copy error: reading from a closed fifo
ERRO[2018-06-06T09:02:44.149273618Z] stream copy error: reading from a closed fifo
[ ](https://chat.hyperledger.org/channel/cello?msg=jnq7LbXdFpZ5RtcC7) @NicholasBaro Did you resolve the issue? I'm getting the same behavior for the newly installed dashboard. It worked before
@AshishMishra 1 not really done, do you have some idea? basically we want to collect some metrics and analyze using ELK.
Has joined the channel.
Has joined the channel.
I am new to cello and I would like to know if anybody has already setup the master-node.
I am getting this error..
ERROR: Version in "./docker-compose.yml" is unsupported. You might be seeing this error because you're using the wrong Compose file version. Either specify a version of "2" (or "2.0")
If I change the version in yml file to "2.0", rest of the definitions do not work as they belong to later versions. I have checked my docker and docker-compose versions. both are latest.
docker version
Client:
Version: 18.04.0-ce
API version: 1.37
Go version: go1.9.4
Git commit: 3d479c0
Built: Tue Apr 10 18:20:32 2018
OS/Arch: linux/amd64
Experimental: false
Orchestrator: swarm
Server:
Engine:
Version: 18.04.0-ce
API version: 1.37 (minimum version 1.12)
Go version: go1.9.4
Git commit: 3d479c0
Built: Tue Apr 10 18:18:40 2018
OS/Arch: linux/amd64
Experimental: false
docker-compose version 1.8.0, build unknown
docker-py version: 1.9.0
CPython version: 2.7.12
OpenSSL version: OpenSSL 1.0.2g 1 Mar 2016
@SmartContract2018 just use `make setup-master`
@SmartContract2018 just use `make setup-master` from the latest git tag
Has joined the channel.
Hi there, Can master and worker node be separate virtual machines?
ofc
Hello, is there any example regarding events (in chaincode and listening in the sdk)
Hi All,
when trying to access the worker node and adding IP 127.0.0.1:2375 I am getting error 400
The request was issued with an error. The server did not create or modify data.
Error.png
master.png
[ ](https://chat.hyperledger.org/channel/cello?msg=qr423vFyeH3un28Ee) @infowolfe Thanks. That is what I am using and make is successful. Next step is to run 'make start' and that is when I get the error I described before. Any help is appreciated.
@all The Cello project meeting will be held at 12:30 (UTC)/20:30 (UTC+8) this Friday. Zoom address: https://zoom.us/my/hyperledger.community
Has joined the channel.
Hi everyone. I'm trying to start a chain with cello. I managed to add a distant docker host, but when i start a chain i have this error : Exception when compose start=404 Client Error: Not Found ("manifest for hyperledger/fabric-ca:1.0.5 not found") . Looks like there is a wrong version somewhere, it should be x86-84-1.0.5 ? i'm using tag v0.8.0
@here Today's meeting minutes is available now, thanks! https://wiki.hyperledger.org/projects/cello/meetings/20180615.
Has joined the channel.
Has joined the channel.
@SmartContract2018 Are you still having the docker error? I think you might need to upgrade you docker compose. The latest version is 1.21.2.
Running `sudo curl -L https://github.com/docker/compose/releases/download/1.21.2/docker-compose-$(uname -s)-$(uname -m) -o LOCATION_OF_YOUR_DOCKER_COMPOSE`
Running `which docker-compose` will give the location. I hope it helps
@LichenShen Do you still have your issue? I am also new to this and I ran into the same issue while running the setup the first time in DEV (`DEV=True make start`). In the logs (`DEV=True make logs`) I found this `jinja2.exceptions.TemplateNotFound: 404.html`. I notice that I can log properly when using `make start` (without `DEV=True`) and start using cello
@LichenShen Do you still have your issue? I am also new to this and I ran into the same issue while running the setup the first time in DEV (`DEV=True make start`). In the logs (`DEV=True make logs`) I found this `jinja2.exceptions.TemplateNotFound: 404.html`. I notice that I can log properly when using `make start` (without `DEV=True`) and start using cello
Has joined the channel.
does cello supports network without docker or docker-swarm or kubernets? I mean each peer or orderer is on a single machine
Does cello supports this type of network setip?
Does cello supports this type of network setup?
I mean running peer binary in each system where peer is required and running orderer binary in each system where orderer is required
anyone knows the term `BareMetal` used in `cello` documentation?
Anyhelp would be appreciated
Hi All,`localhost:8080` keeps loading after `make start` command
Hi All, \`localhost:8080` keeps loading after `make start` command
Hi All, `localhost:8080` keeps loading after `make start` command
Has joined the channel.
`operator-dashboard` on `localhost:8080` keeps loading forever. Anyone knows this issue?
Hey everyone, does Cello support authentication for the rest server?
Has joined the channel.
Has joined the channel.
Hi all, I'm trying to follow the cello documentation and everything looks OK till I want to "apply new chain" in the operator dashboard. It says "Apply chain test fails". Any idea why?
Has joined the channel.
Hi there, I see the "vm" and "baremetal" options in the Cello wiki -- what is the difference between them, and what is the implementation status of each?
@pankajcheema try to run make build-admin-js first
Hi all, Does anybody managed to install the recent Cello version into AWS via ansible scripts? Is it stable at all? I'm getting the error below. Can anybody help clarify ?
`TASK [Start network] ************************************************************************************************************************************************
fatal: [18.221.22.130]: FAILED! => {"changed": true, "cmd": ["docker", "run", "-v", "/home/ubuntu/.composer:/home/composer/.composer", "-v", "/opt/gopath/bc1st/fabric:/opt/gopath/bc1st/fabric", "hyperledger/composer-cli:next", "network", "start", "-c", "PeerAdmin@orga", "-a", "/opt/gopath/bc1st/fabric/run/keyfiles/carauction-network.bna", "-A", "orgaAdmin", "-C", "/home/composer/.composer/orgaAdmin/admin-pub.pem", "-A", "orgbAdmin", "-C", "/home/composer/.composer/orgbAdmin/admin-pub.pem"], "delta": "0:00:02.298221", "end": "2018-06-18 21:21:20.944139", "msg": "non-zero return code", "rc": 1, "start": "2018-06-18 21:21:18.645918", "stderr": "✖ Starting business network definition. This may take a minute...", "stderr_lines": ["✖ Starting business network definition. This may take a minute..."], "stdout": "Starting business network from archive: /opt/gopath/bc1st/fabric/run/keyfiles/carauction-network.bna\nBusiness network definition:\n\tIdentifier: carauction-network@0.2.3-20180314171154\n\tDescription: Car Auction Business Network\n\nProcessing these Network Admins: \n\tuserName: orgaAdmin\n\tuserName: orgbAdmin\n\nError: Error trying to instantiate composer runtime. Error: Error: 2 UNKNOWN: access denied: channel [firstchannel] creator org [orga]\nCommand failed", "stdout_lines": ["Starting business network from archive: /opt/gopath/bc1st/fabric/run/keyfiles/carauction-network.bna", "Business network definition:", "\tIdentifier: carauction-network@0.2.3-20180314171154", "\tDescription: Car Auction Business Network", "", "Processing these Network Admins: ", "\tuserName: orgaAdmin", "\tuserName: orgbAdmin", "", "Error: Error trying to instantiate composer runtime. Error: Error: 2 UNKNOWN: access denied: channel [firstchannel] creator org [orga]", "Command failed"]}`
Hi all, Does anybody managed to install the recent Cello version into AWS via ansible scripts? Is it stable at all? I'm getting the error below. Can anybody help clarify ?
```
```
Hi all, Does anybody managed to install the recent Cello version into AWS via ansible scripts? Is it stable at all? I'm getting the error below. Can anybody help clarify ?
```TASK [Start network] ************************************************************************************************************************************************
fatal: [18.221.22.130]: FAILED! => {"changed": true, "cmd": ["docker", "run", "-v", "/home/ubuntu/.composer:/home/composer/.composer", "-v", "/opt/gopath/bc1st/fabric:/opt/gopath/bc1st/fabric", "hyperledger/composer-cli:next", "network", "start", "-c", "PeerAdmin@orga", "-a", "/opt/gopath/bc1st/fabric/run/keyfiles/carauction-network.bna", "-A", "orgaAdmin", "-C", "/home/composer/.composer/orgaAdmin/admin-pub.pem", "-A", "orgbAdmin", "-C", "/home/composer/.composer/orgbAdmin/admin-pub.pem"], "delta": "0:00:02.298221", "end": "2018-06-18 21:21:20.944139", "msg": "non-zero return code", "rc": 1, "start": "2018-06-18 21:21:18.645918", "stderr": "✖ Starting business network definition. This may take a minute...", "stderr_lines": ["✖ Starting business network definition. This may take a minute..."], "stdout": "Starting business network from archive: /opt/gopath/bc1st/fabric/run/keyfiles/carauction-network.bna\nBusiness network definition:\n\tIdentifier: carauction-network@0.2.3-20180314171154\n\tDescription: Car Auction Business Network\n\nProcessing these Network Admins: \n\tuserName: orgaAdmin\n\tuserName: orgbAdmin\n\nError: Error trying to instantiate composer runtime. Error: Error: 2 UNKNOWN: access denied: channel [firstchannel] creator org [orga]\nCommand failed", "stdout_lines": ["Starting business network from archive: /opt/gopath/bc1st/fabric/run/keyfiles/carauction-network.bna", "Business network definition:", "\tIdentifier: carauction-network@0.2.3-20180314171154", "\tDescription: Car Auction Business Network", "", "Processing these Network Admins: ", "\tuserName: orgaAdmin", "\tuserName: orgbAdmin", "", "Error: Error trying to instantiate composer runtime. Error: Error: 2 UNKNOWN: access denied: channel [firstchannel] creator org [orga]", "Command failed"]}
```
Hi all, Does anybody managed to install the recent Cello version into AWS via ansible scripts? Is it stable at all? I'm getting the error below. Can anybody help clarify ?
```TASK [Start network] ************************************************************************************************************************************************
fatal: [xx.xx.xx.xx]: FAILED! => {"changed": true, "cmd": ["docker", "run", "-v", "/home/ubuntu/.composer:/home/composer/.composer", "-v", "/opt/gopath/bc1st/fabric:/opt/gopath/bc1st/fabric", "hyperledger/composer-cli:next", "network", "start", "-c", "PeerAdmin@orga", "-a", "/opt/gopath/bc1st/fabric/run/keyfiles/carauction-network.bna", "-A", "orgaAdmin", "-C", "/home/composer/.composer/orgaAdmin/admin-pub.pem", "-A", "orgbAdmin", "-C", "/home/composer/.composer/orgbAdmin/admin-pub.pem"], "delta": "0:00:02.298221", "end": "2018-06-18 21:21:20.944139", "msg": "non-zero return code", "rc": 1, "start": "2018-06-18 21:21:18.645918", "stderr": "✖ Starting business network definition. This may take a minute...", "stderr_lines": ["✖ Starting business network definition. This may take a minute..."], "stdout": "Starting business network from archive: /opt/gopath/bc1st/fabric/run/keyfiles/carauction-network.bna\nBusiness network definition:\n\tIdentifier: carauction-network@0.2.3-20180314171154\n\tDescription: Car Auction Business Network\n\nProcessing these Network Admins: \n\tuserName: orgaAdmin\n\tuserName: orgbAdmin\n\nError: Error trying to instantiate composer runtime. Error: Error: 2 UNKNOWN: access denied: channel [firstchannel] creator org [orga]\nCommand failed", "stdout_lines": ["Starting business network from archive: /opt/gopath/bc1st/fabric/run/keyfiles/carauction-network.bna", "Business network definition:", "\tIdentifier: carauction-network@0.2.3-20180314171154", "\tDescription: Car Auction Business Network", "", "Processing these Network Admins: ", "\tuserName: orgaAdmin", "\tuserName: orgbAdmin", "", "Error: Error trying to instantiate composer runtime. Error: Error: 2 UNKNOWN: access denied: channel [firstchannel] creator org [orga]", "Command failed"]}
```
@Luke_Chen Thanks for suggestion. I am running `make build-admin-js` and will let you know whether it works or not,
@Luke_Chen Thanks for suggestion. I am running `make build-admin-js` and will let you know whether it works or not.
@Luke_Chen I tried `make build-admin-js` but still `localhost:8080` taking forever to load
Has joined the channel.
Please find the issue details on https://stackoverflow.com/questions/50921415/hyperledger-cello-operator-dashboard-keeps-loading
Is there any expert of cell available?
I am facing an issue from last 2 days
@baohua @hightall
https://gerrit.hyperledger.org/r/#/c/23303/
@hightall
Are you there?
yeah
Please see this if you have some time https://stackoverflow.com/questions/50921415/hyperledger-cello-operator-dashboard-keeps-loading
I am facing issue from last 2 days
@pankajcheema you use the latest version?
yes
cello.png
I'm fixing on version 0.8.0 image failed issue, after fixed this you can try v0.8.0 image for testing.
I cloned from master
Is it a bug?
yes, I think maybe not compiled js files in latest image.
Ok
How much time it will take?
Sorry but I was furious from last 2 days
:(
maybe 1 or 2 days:relaxed:
Ok
Thank you very much @hightall
you're welcome
need some more moderators like you
Very much appreciated
@hightall one more question
is cello stable?
I mean is there any release version for cello that is stable?
https://gerrit.hyperledger.org/r/#/c/23309/
Hi all, I am facing an issue with cello while trying to apply a new chain. I searched online and found the exact same issue in stackoverflow, which is not answered, yet. I'm wondering is someone could have a look at it. Thanks.
Here is the link: https://stackoverflow.com/questions/49005095/not-able-to-create-hyperledger-fabric-network-using-cello
@baohua @hightall please take a look at this patch which is to reduce some duplicate code and also performance improvement on chaincode container creation. the main work is for k8s env. Please review https://gerrit.hyperledger.org/r/#/c/23157/
Thanks.
@hightall @tongli - after your suggestion , i have pulled up the v0.8.0 image but still am facing the same issue
@phulei i am not familar with that part of the cello.
cello-user-dashboard | 17:10:24 0|index | error: [client-utils.js]: sendPeersProposal - Promise is rejected: Error: 2 UNKNOWN: could not find chaincode with nam
e 'c42cbe481c1c4a049d2f865210a3e36e-5b2937d6afce220029a6a47c' - make sure the chaincode c42cbe481c1c4a049d2f865210a3e36e-5b2937d6afce220029a6a47c has been successfu
lly instantiated and try again
cello-user-dashboard | 17:10:24 0|index | at new createStatusError (/node_modules/grpc/src/client.js:64:15)
cello-user-dashboard | 17:10:24 0|index | at /node_modules/grpc/src/client.js:583:15
cello-user-dashboard | 17:10:24 0|index | [2018-06-19 17:10:24.229] [ERROR] invoke-chaincode - transaction proposal was bad
cello-user-dashboard | 17:10:24 0|index | [2018-06-19 17:10:24.230] [ERROR] invoke-chaincode - Failed to send Proposal or receive valid response. Response null
or status is not 200. exiting...
cello-user-dashboard | 17:10:24 0|index | [2018-06-19 17:10:24.230] [ERROR] invoke-chaincode - Failed to order the transaction. Error code: undefined
I have been developing ansible agent which stand up fabric onto existing k8s env.
oh ok !
@hightall - updated the ticket with the details
Not sure how they will mesh with the Cello project, but at AID:Tech, @nicolapaoli and I are open-sourcing some Helm Charts to deploy Fabric on Kubernetes, stay tuned, and feel free to contribute to them once they are approved:
https://github.com/kubernetes/charts/pull/6181
https://github.com/kubernetes/charts/pull/6165
https://github.com/kubernetes/charts/pull/6163
https://github.com/kubernetes/charts/pull/6144
[ ](https://chat.hyperledger.org/channel/cello?msg=wqsmtN3EKex9fG7nC) anyone know about this?
@pankajcheema we have fix this problem, please use version 0.8.0 image to run.
in master node
1. git clone -b v0.8.0 --depth 1 https://github.com/hyperledger/cello.git
2. VERSION=0.8.0 make dockerhub-pull
3. make start
in worker node
1. git clone -b v0.8.0 --depth 1 https://github.com/hyperledger/cello.git
2. make setup-worker
Has joined the channel.
This is my first post.. not sure if this is right forum.. Did somebody try running IBM marbles demo(chain codes) in mutilple worker nodes ..if yes, can you please share the steps
Thank you very much @hightall. I will try it and will let you know. Thanks again :)
@hightall one more question. Is Cello stable?
I mean cello build is in alpha, beta or release?
version 0.8.0 is release version.
ok
Thanks for confirming
https://gerrit.hyperledger.org/r/#/c/23365/
is there documentation for initializing k8s worker with cello?
@infowolfe I am working on it
is there an initial version?
If you really want to try the k8s agent, refer to this article for setting up a minikube first https://github.com/kubernetes/minikube
I've already got a working kubernetes cluster
i only need to know how cello interacts with k8s
alright, I send you an unfinished one.
just wait a minute.
:thumbsup:
thanks much
@infowolfe can I send it to you later, since I can't connect to my server right now, apologize for that.
np
https://gerrit.hyperledger.org/r/#/c/23369/
Has joined the channel.
Hi All, I am trying to run worker node and master node on separate Snyology clouds, please advise me, I can successfully put up the master and worker dashboard but unable to add host IP???
@baohua @hightall Responded to your comment. Please review https://gerrit.hyperledger.org/r/#/c/23157/ thanks.
+2!
@baohua thanks a lot
@hightall - can you please take a look at the errors - https://jira.hyperledger.org/projects/CE/issues/CE-375
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=8Wp2x7gfqtHRawssK) @alexvicegrab FWIW, an equivalent helm chart has been done here https://github.com/IBM-Blockchain/ibm-container-service
Hi @harsha, thank you for mentioning that repo.
I'm aware of the monolithic Helm chart created on that repository, but that Chart has a number of things I sought to improve upon:
1) it's not being maintained by anyone (at least for the last 5 months)
2) it's monolithic, and uses a single shared volume to store all the crypto material, making it unsuitable to a production deployment
3) it uses a single CA, rather than a hierarchy of root to intermediate CAs, which also leads to security issues
4) it does not enable a Kafka based ordering service
5) to implement a multiple number of peers you need to add extra deployments/services - I felt it would be better to have a helm chart per node, to enable flexible deployments.
To be fair, my own Helm charts themselves works in progress and would benefit from extra work to make them as versatile as possible, but I still feel they may provide a useful basis to others working on making deployment and orchestration easier.
Hi @harsha, thank you for mentioning that repo.
I'm aware of the monolithic Helm chart created on that repository, but that Chart has a number of things I sought to improve upon:
1) it's not being maintained by anyone (at least for the last 4 months)
2) it's monolithic, and uses a single shared volume to store all the crypto material, making it unsuitable to a production deployment
3) it uses a single CA, rather than a hierarchy of root to intermediate CAs, which also leads to security issues
4) it does not enable a Kafka based ordering service
5) to implement a multiple number of peers you need to add extra deployments/services - I felt it would be better to have a helm chart per node, to enable flexible deployments.
My set of charts is therefor not `equivalent`, since they are more modular. To be fair, my own Helm charts themselves works in progress and would benefit from extra work to make them as versatile as possible, but I still feel they may provide a useful basis to others working on making deployment and orchestration easier.
Hi @harsha, thank you for mentioning that repo.
I'm aware of the monolithic Helm chart created on that repository, but that Chart has a number of things I sought to improve upon:
1) it's not being maintained by anyone (at least for the last 4 months)
2) it's monolithic, and uses a single shared volume to store all the crypto material, making it unsuitable to a production deployment
3) it uses a single CA, rather than a hierarchy of root to intermediate CAs, which also leads to security issues
4) it does not enable a Kafka based ordering service
5) to implement a multiple number of peers you need to add extra deployments/services - I felt it would be better to have a helm chart per node, to enable flexible deployments.
My set of charts is therefore not `equivalent`, since they are more modular. To be fair, my own Helm charts themselves works in progress and would benefit from extra work to make them as versatile as possible, but I still feel they may provide a useful basis to others working on making deployment and orchestration easier.
Hi @harsha, thank you for mentioning that repo.
I'm aware of the monolithic Helm chart created on that repository, but that Chart has a number of things I sought to improve upon:
1) it's not being maintained by anyone (at least for the last 4 months)
2) it's monolithic, and uses a single shared volume to store all the crypto material, making it unsuitable to a production deployment
3) it uses a single CA, rather than a hierarchy of root to intermediate CAs, which also leads to security issues
4) it does not enable a Kafka based ordering service
5) to implement a multiple number of peers you need to add extra deployments/services - I felt it would be better to have a helm chart per node, to enable flexible deployments.
My set of charts is therefore not `equivalent`, since they are more modular. To be fair, my own Helm charts themselves works in progress and would benefit from extra work to make them as versatile as possible, but I still feel they may provide a useful basis to others working on making deployment and orchestration easier. We also plan to maintain them for the foreseeable future.
Hi @harsha, thank you for mentioning that repo.
I'm aware of the monolithic Helm chart created on that repository, but that Chart has a number of things I sought to improve upon:
1) it's not being maintained by anyone (at least for the last 4 months)
2) it's monolithic, and uses a single shared volume to store all the crypto material, making it unsuitable to a production deployment
3) it uses a single CA, rather than a hierarchy of root to intermediate CAs, which also leads to security issues
4) it does not enable a Kafka based ordering service
5) to implement a multiple number of peers you need to add extra deployments/services - I felt it would be better to have a helm chart per node, to enable flexible deployments.
My set of charts is therefore not `equivalent`, since they are more modular, permit working with a Kafka orderer set, enable intermediate CAs, etc.
To be fair, my own Helm charts themselves works in progress and would benefit from extra work to make them as versatile as possible, but I still feel they may provide a useful basis to others working on making deployment and orchestration easier. We also plan to maintain them for the foreseeable future.
Hi @harsha, thank you for mentioning that repo.
I'm aware of the monolithic Helm chart created on that repository, but that Chart has a number of things I sought to improve upon:
1) it's not being maintained by anyone (at least for the last 4 months)
2) it's monolithic, and uses a single shared volume to store all the crypto material, making it unsuitable to a production deployment
3) it uses a single CA, rather than a hierarchy of root to intermediate CAs, which also leads to security issues
4) it does not enable a Kafka based ordering service
5) to implement a multiple number of peers you need to add extra deployments/services - I felt it would be better to have a helm chart per node, to enable flexible deployments.
My set of charts is therefore not `equivalent`, since they are more modular, permit working with a Kafka orderer set, enable intermediate CAs, etc.
To be fair, my own Helm charts themselves are works in progress and would benefit from extra work to make them as versatile as possible, but I still feel they may provide a useful basis to others working on making deployment and orchestration easier. We also plan to maintain them for the foreseeable future.
Hi @harsha, thank you for mentioning that repo.
I'm aware of the monolithic Helm chart created on that repository, but that Chart has a number of things I sought to improve upon:
1) it's not being maintained by anyone (at least for the last 4 months)
2) it's monolithic, and uses a single shared volume to store all the crypto material, making it unsuitable to a production deployment
3) it uses a single CA, rather than a hierarchy of root to intermediate CAs, which also leads to security issues
4) it does not enable a Kafka based ordering service
5) to implement a multiple number of peers you need to add extra deployments/services - I felt it would be better to have a helm chart per node, to enable flexible deployments.
My set of charts is therefore not `equivalent`, since they are more modular, permit working with a Kafka orderer set, enable intermediate CAs, etc.
To be fair, my own Helm charts themselves are works in progress and would benefit from extra contributions to make them as versatile as possible, but I still feel they may provide a useful basis to others working on making deployment and orchestration easier. We also plan to maintain them for the foreseeable future.
Hi @harsha, thank you for mentioning that repo.
I'm aware of the monolithic Helm chart created on that repository, but that Chart has a number of things I sought to improve upon:
1) it's not being maintained by anyone (at least for the last 4 months)
2) it's monolithic, and uses a single shared volume to store all the crypto material, making it unsuitable to a production deployment
3) it uses a single CA, rather than a hierarchy of root to intermediate CAs, which also leads to security issues
4) it does not enable a Kafka based ordering service
5) to implement a multiple number of peers you need to add extra deployments/services - I felt it would be better to have a helm chart per node, to enable flexible deployments.
I'd argue, hence, that my set of charts is therefore not `equivalent`, since they are more modular, permit working with a Kafka orderer set, enable intermediate CAs, etc.
To be fair, my own Helm charts themselves are works in progress and would benefit from extra contributions to make them as versatile as possible, but I still feel they may provide a useful basis to others working on making deployment and orchestration easier. We also plan to maintain them for the foreseeable future.
Hi @harsha, thank you for mentioning that repo.
I'm aware of the monolithic Helm chart created on that repository, but that Chart has a number of things I sought to improve upon:
1) it's not being maintained by anyone (at least for the last 4 months)
2) it's monolithic, and uses a single shared volume to store all the crypto material, making it unsuitable to a production deployment
3) it uses a single CA, rather than a hierarchy of root to intermediate CAs, which also leads to security issues
4) it does not enable a Kafka based ordering service
5) to implement a multiple number of peers you need to add extra deployments/services - I felt it would be better to have a helm chart per node, to enable flexible deployments.
I'd argue, hence, that my set of charts isnot `equivalent`, since they are more modular, permit working with a Kafka orderer set, enable intermediate CAs, etc.
To be fair, my own Helm charts themselves are works in progress and would benefit from extra contributions to make them as versatile as possible, but I still feel they may provide a useful basis to others working on making deployment and orchestration easier. We also plan to maintain them for the foreseeable future.
Hi @harsha, thank you for mentioning that repo.
I'm aware of the monolithic Helm chart created on that repository, but that Chart has a number of things I sought to improve upon:
1) it's not being maintained by anyone (at least for the last 4 months)
2) it's monolithic, and uses a single shared volume to store all the crypto material, making it unsuitable to a production deployment
3) it uses a single CA, rather than a hierarchy of root to intermediate CAs, which also leads to security issues
4) it does not enable a Kafka based ordering service
5) to implement a multiple number of peers you need to add extra deployments/services - I felt it would be better to have a helm chart per node, to enable flexible deployments.
I'd argue, hence, that my set of charts is not `equivalent`, since they are more modular, permit working with a Kafka orderer set, enable intermediate CAs, etc.
To be fair, my own Helm charts themselves are works in progress and would benefit from extra contributions to make them as versatile as possible, but I still feel they may provide a useful basis to others working on making deployment and orchestration easier. We also plan to maintain them for the foreseeable future.
Hi @harsha, thank you for mentioning that repo.
I'm aware of the monolithic Helm chart created on that repository, but that Chart has a number of things I sought to improve upon:
1) it's not being maintained by anyone (at least for the last 4 months)
2) it's monolithic, and uses a single shared volume to store all the crypto material, making it unsuitable to a production deployment
3) it uses a single CA, rather than a hierarchy of root to intermediate CAs, which also leads to security issues
4) it does not enable a Kafka based ordering service
5) to implement a multiple number of peers you need to add extra deployments/services - I felt it would be better to have a helm chart per node, to enable flexible deployments.
I'd argue, hence, that my set of charts is not `equivalent`, since our charts are more modular, permit working with a Kafka orderer set, enable intermediate CAs, etc.
To be fair, my own Helm charts themselves are works in progress and would benefit from extra contributions to make them as versatile as possible, but I still feel they may provide a useful basis to others working on making deployment and orchestration easier. We also plan to maintain them for the foreseeable future.
Hi @harsha, thank you for mentioning that repo.
I'm aware of the monolithic Helm chart created on that repository, but that Chart has a number of things I sought to improve upon:
1) it's not being maintained by anyone (at least for the last 4 months)
2) it's monolithic, and uses a single shared volume to store all the crypto material, making it unsuitable to a production deployment
3) it uses a single CA, rather than a hierarchy of root to intermediate CAs, which also leads to security issues
4) it does not enable a Kafka based ordering service
5) to implement a multiple number of peers you need to add extra deployments/services - I felt it would be better to have a helm chart per node, to enable flexible deployments.
I'd argue, hence, that my set of charts is not `equivalent`, since our charts are more modular, permit working with a Kafka orderer set, enable intermediate CAs, etc.
To be fair, our own Helm charts themselves are works in progress and would benefit from extra contributions to make them as versatile as possible, but I still feel they may provide a useful basis to others working on making deployment and orchestration easier. We also plan to maintain them for the foreseeable future.
@baohua @hightall I found a bug in the ansible agent which does not do dns service correctly when container is trying to access external site. I created a bug and fixed it in this patch https://gerrit.hyperledger.org/r/#/c/23419/. Please take a look. This patch set also made some improvement so that chaincode container images can be downloaded before a chaincode vm is to be created which in many cases will timeout. With this patch, that problem will not happen. Thanks.
The link again here https://gerrit.hyperledger.org/r/#/c/23419/
@phulei Please remove the move in parameter only have a, b, 100
@hightall @baohua looks like we are having some build issues.
my patch set produced the following error which has nothing to do with the ansible agent. Looks that the error came from dashboard build.
Here is the error
```Creating celloverifyx8664_check-user-dashboard_1 ... done
Creating celloverifyx8664_check-js_1 ... done
Attaching to celloverifyx8664_check-user-dashboard_1, celloverifyx8664_check-js_1
check-user-dashboard_1 |
check-user-dashboard_1 | > user-dashboard@1.0.0 lint /var/www
check-user-dashboard_1 | > eslint --ignore-pattern 'app/assets/**' .
check-user-dashboard_1 |
check-js_1 |
check-js_1 | > cello-operator@0.8.0 lint /var/www/dashboard
check-js_1 | > eslint --ext .js src mock tests && npm run lint:style
check-js_1 |
check-user-dashboard_1 |
check-user-dashboard_1 | > user-dashboard@1.0.0 lint:ui /var/www
check-user-dashboard_1 | > eslint app/assets -c .ui-eslintrc
check-user-dashboard_1 |
check-js_1 |
check-js_1 | /var/www/dashboard/src/routes/Host/CreateHost/index.js
check-js_1 | 500:18 error Expected indentation of 16 space characters but found 17 react/jsx-indent
check-js_1 | 501:19 error Expected indentation of 19 space characters but found 18 react/jsx-indent
check-js_1 | 516:19 error Expected indentation of 19 space characters but found 18 react/jsx-indent
check-js_1 | 517:20 error Expected indentation of 20 space characters but found 19 react/jsx-indent-props
check-js_1 | 518:20 error Expected indentation of 20 space characters but found 19 react/jsx-indent-props
check-js_1 | 520:20 error Expected indentation of 20 space characters but found 19 react/jsx-indent
check-js_1 |
check-js_1 | ✖ 6 problems (6 errors, 0 warnings)
check-js_1 | 6 errors, 0 warnings potentially fixable with the `--fix` option.
check-js_1 |
check-js_1 | npm ERR! code ELIFECYCLE
check-js_1 | npm ERR! errno 1
check-js_1 | npm ERR! cello-operator@0.8.0 lint: `eslint --ext .js src mock tests && npm run lint:style`
check-js_1 | npm ERR! Exit status 1
check-js_1 | npm ERR!
check-js_1 | npm ERR! Failed at the cello-operator@0.8.0 lint script.
check-js_1 | npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
check-js_1 |
check-js_1 | npm ERR! A complete log of this run can be found in:
check-js_1 | npm ERR! /root/.npm/_logs/2018-06-21T03_10_56_246Z-debug.log
celloverifyx8664_check-js_1 exited with code 1
check-user-dashboard_1 |
check-user-dashboard_1 | /var/www/app/assets/src/components/Login/index.js
check-user-dashboard_1 | 11:3 error defaultProps should be placed after childContextTypes react/sort-comp
check-user-dashboard_1 |
check-user-dashboard_1 | ✖ 1 problem (1 error, 0 warnings)```
This is js format check for operator dashboard, I think it's because luke's patch, but why it's didn't report error when his patch uploaded.
I will check it locally
Before I commit the patch, I run the fix for js code format.
@hightall yeah, that is right, I do not know why it was ok when it got merged.
really weird.
anyway, I will fix the js code format locally, and push another patch, then you merge and repush your patch. @tongli
yeah, I can rebase.
not a problem. Thanks for taking care of this.
talk to you friday night.
ok
[ ](https://chat.hyperledger.org/channel/cello?msg=GnyAkxm4xcX3nP3b6) Thank you very much @hightall. Operator dashboard now working
hi all.. I'm trying to setup master and worker nodes on 2 separate VMs each running on different machines. I am able to launch both dashboards, add host IP and create chains; however, the apply blockchain part fails.
I noticed that all my containers for the peers keep restarting and the CAs report TLS handshake errors. I also observed that the container denoted fabric-tools is empty.
I noticed that all my containers for the peers keep restarting and the CAs report TLS handshake errors. I also observed that the container denoted fabric-tools has no tools inside.
I noticed that all my containers for the peers keep restarting and the CAs report TLS handshake errors. I also observed that the container denoted fabric-tools has no tools inside. I believe the Hyperledger Fabric tools (cryptogen, configtxgen etc) are missing in the created images.
Could someone point me in the right direction on how to fix this issue ? I believe the Hyperledger Fabric tools (cryptogen, configtxgen etc) are missing in the created images. Has anyone else faced and resolved this issue; if so please help with the fix.
Has anyone else faced and resolved this issue; if so please help with the fix.
Hi Cello Experts
Anyone know what is `capacity` when adding a host?
and what is `chain size` when creating a cluster
@hightall r we good on the build? Any patch set to fix the issue?
https://gerrit.hyperledger.org/r/#/c/23419/
@frustrated-chain I have made some changes in the above patch set to address the problems that you had
the error that you had few days ago won't happen for a k8s cluster when dns policy was set appropriately.
when you use cello to also setup k8s, the dns policy was not set in some cases.
you are the one hit that issue.
that issue caused the chaincode container not being able to pull images from outside.
the patch that I just put up solve all sort of the issues for you and also some other improvement.
it also exposes the peer and orderer service endpoint if you choose to.
this is a patch set you really want to use. hope it lands soon.
since you are doing this on aws
you can easily do all the tasks with just one command
``` ansible-playbook -e "mode=apply" awsk8s.yml```
giving that you have a clean account.
of course, you need to make sure that your vars/aws.yml file is correct.
and your vars/bc1st.yml is the latest
I would suggest that you remove everything that you have and give it a try
This is great, thank you @tongli.
This is great, thank you @tongli
Life will be much easier with kubernetes. I will do ansible-playbook -e "mode=apply" awsk8s.yml
Meanwhile, I did ansible-playbook -e "mode=apply" aws.yml with our chaincode.
And I have a question, after building the Fabric network with the ansible agent (with custom chaincode installed) without kubernetes, how can I make REST calls? I could not see the port 5000 binding!
if you already have an existing k8s cluster,
you just need to place your kubeconfig file in vars/kubeconfig. then run the following command
```ansible-playbook -e "mode=apply env=bc1st deploy_type=k8s" setupfabric.yml```
replace word apply with destroy to get rid of every thing
when you use aws.yml, it setup everything using docker and overlay network. it has nothing to do with k8s
all the endpoints won't be available to the outside of world
unless you do proxy services which is a whole different ball game
the assumption is that in that case, your applications will run on the same network that fabric runs.
so you expose your application endpoint instead of fabric endpoint.
it was done that way with a lot of considerations.
with k8s , it is a bit different, since k8s already has the proxy for you to use
you are using aws already, I strong suggest you use eks to get a k8s system up and running, then simply using the k8s deploy command.
that part I have not tested myself.
I have done it on ibm k8s and google gke
I mean I have not tested it against eks.
but I have tested on ibm k8s and gke
what you are doing now, is using cello and aws setting k8s yourself.
it will be a bit different from using AKS.
sorry EKS
but all commands will be exactly the same once you have a k8s cluster, does not matter how or who set it up
Alright, got it. I haven’t used EKS but I will try to learn!
I don't have a k8s cluster and I am going to deploy a fresh install with fresh instances. Just to get your confirmation, that the following steps are correct, I am going to:
1) change the chaincode in the fabricsetup/templates/firstcode.go with ours
2) ansible-playbook -e "mode=apply" awsk8s.yml
3) ansible-playbook -e "mode=apply env=bc1st deploy_type=k8s" setupfabric.yml
and live happily ever after!
Alright, got it @tongli . I haven’t used EKS but I will try to learn!
I don't have a k8s cluster and I am going to deploy a fresh install with fresh instances. Just to get your confirmation, that the following steps are correct, I am going to:
1) change the chaincode in the fabricsetup/templates/firstcode.go with ours
2) ansible-playbook -e "mode=apply" awsk8s.yml
3) ansible-playbook -e "mode=apply env=bc1st deploy_type=k8s" setupfabric.yml
and live happily ever after!
you do not need 3)
2) does it all
thank you!!
you need to get that patch on your ansible controller first
of course, will do that first.
git pull https://gerrit.hyperledger.org/r/cello refs/changes/19/23419/3
https://gerrit.hyperledger.org/r/#/c/23495/
Hi All
Anyone know how to reset cello settings?
How to clear all information?
@hightall Why cello keeps creating chain automatically?
Is it a bug?
and operator dashboard is loading very very slow
even it is on localhost
loading.png
```Creating 299b681732d24ed8b857b8a4156c82d2_orderer ... done
72.18.0.1:2375', 'PEER_NETWORKID': '299b681732d24ed8b857b8a4156c82d2', 'NETWORK_TYPES': 'fabric-1.0', 'COMPOSE_FILE': 'fabric-solo-4.yaml', 'CLUSTER_LOG_LEVEL': 'DEBUG', 'COMPOSE_PROJECT_PATH': '/opt/cello/fabric-1.0', 'COMPOSE_PROJECT_NAME': '299b681732d24ed8b857b8a4156c82d2', 'CLUSTER_NETWORK': 'cello_net_solo', 'DOCKER_HOST': 'tcp://172.18.0.1:2375', 'VM_DOCKER_HOSTCONFIG_NETWORKMODE': 'cello_net_solo'}
Creating 299b681732d24ed8b857b8a4156c82d2_peer1_org1 ... done
0
cello-watchdog | Creating 299b681732d24ed8b857b8a4156c82d2_ca_org2 ...
cello-watchdog | Creating 299b681732d24ed8b857b8a4156c82d2_ca_org1 ...
cello-watchdog | Creating 299b681732d24ed8b857b8a4156c82d2_orderer ...
cello-watchdog | Creating 299b681732d24ed8b857b8a4156c82d2_mysql ...
Creating 299b681732d24ed8b857b8a4156c82d2_peer1_org2 ...
cello-watchdog | Creating 299b681732d24ed8b857b8a4156c82d2_peer1_org1 ...
cello-watchdog | Creating 299b681732d24ed8b857b8a4156c82d2_peer0_org1 ...
cello-watchdog | Creating 299b681732d24ed8b857b8a4156c82d2_peer0_org2 ...
Creating 299b681732d24ed8b857b8a4156c82d2_cli ...
cello-watchdog | Creating 299b681732d24ed8b857b8a4156c82d2_explorer ...
```
it keeps creating peers and orderers even if chain is not created
Anyone?
@hightall could you please look into this if you have some time?
@pankajcheema do you create the host with keep filled option?
you mean `Autofill`?
yes
if you select this, after you create the host, watchdog will keep the host full filled with capacity.
Ok ok
it was checked
@hightall what `schedulable checkbox` means?
mean whether this host can be scheduled when use want to apply in user dashboard
I could not find description for some of the actions
like `schedulable` , `capacity`, `chain size`
chain.png
Does chain size means total number of peers?
@hightall
and does capacity means max number of chains that can be created in a host?
yes, 2org4peers
Great
[ ](https://chat.hyperledger.org/channel/cello?msg=5ih69zcFdvcdd4sa5) what about this?
@hightall
yes
Thank you very much @hightall
@hightall I am getting error on the default cello setup when creating a chain.
The logs shows
```cello-operator-dashboard | ERROR: for c3e3edb1dcd74a1c8bd592070ecf0b48_cli Cannot start service cli: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/opt/cello/fabric-1.0/solo/configtx.yaml\\\" to rootfs \\\"/var/lib/docker/overlay2/a64eeceadd6376347ef1c939e9f105883c7780222191b1f754728eb08269a7d8/merged\\\" at \\\"/var/lib/docker/overlay2/a64eeceadd6376347ef1c939e9f105883c7780222191b1f754728eb08269a7d8/merged/etc/hyperledger/fabric/configtx.yaml\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
cello-operator-dashboard |
cello-operator-dashboard | ERROR: for c3e3edb1dcd74a1c8bd592070ecf0b48_explorer Cannot start service explorer: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/opt/cello/fabric-1.0/explorer-artifacts/config.json\\\" to rootfs \\\"/var/lib/docker/overlay2/1c43db97964250c6a17209af42d6388b537d498a9dfce34f519911e06cc4baaf/merged\\\" at \\\"/var/lib/docker/overlay2/1c43db97964250c6a17209af42d6388b537d498a9dfce34f519911e06cc4baaf/merged/blockchain-explorer/config.json\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
cello-operator-dashboard |
cello-operator-dashboard | ERROR: for explorer Cannot start service explorer: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/opt/cello/fabric-1.0/explorer-artifacts/config.json\\\" to rootfs \\\"/var/lib/docker/overlay2/1c43db97964250c6a17209af42d6388b537d498a9dfce34f519911e06cc4baaf/merged\\\" at \\\"/var/lib/docker/overlay2/1c43db97964250c6a17209af42d6388b537d498a9dfce34f519911e06cc4baaf/merged/blockchain-explorer/config.json\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
cello-operator-dashboard |
cello-operator-dashboard | ERROR: for cli Cannot start service cli: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/opt/cello/fabric-1.0/solo/configtx.yaml\\\" to rootfs \\\"/var/lib/docker/overlay2/a64eeceadd6376347ef1c939e9f105883c7780222191b1f754728eb08269a7d8/merged\\\" at \\\"/var/lib/docker/overlay2/a64eeceadd6376347ef1c939e9f105883c7780222191b1f754728eb08269a7d8/merged/etc/hyperledger/fabric/configtx.yaml\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
```
@pankajcheema looks like that worker node doesn't have the /opt/cello directory, did you setup the worker node successfully?
@hightall I have only one system
so I am not able to setup worker node
It says failed to create container
the worker node & master node must separate
Can It be on same system
I mean Can I setup worker and master on same system?
no, because when you reset the host, will clean all docker in this host, include master service
ok
https://gerrit.hyperledger.org/r/#/c/23507/
@hightall have you spent much of any time working on the kubernetes host type?
@infowolfe haven't, will spend some time on it after refine user dashboard.
oh, ok
Has joined the channel.
Has joined the channel.
Hi All, I was trying to deploy the fabric using default cello settings on aws. I am getting below error. Tried to debug but not getting able to understand why permission denied error is through every time. TASK [Start network] ***************************************************************************************************************************************************************************************
fatal: [13.229.141.129]: FAILED! => {"changed": true, "cmd": ["docker", "run", "-v", "/home/ubuntu/.composer:/home/composer/.composer", "-v", "/opt/gopath/bc1st/fabric:/opt/gopath/bc1st/fabric", "--rm", "hyperledger/composer-cli:next", "network", "start", "-c", "PeerAdmin@hindalco", "-a", "/opt/gopath/bc1st/fabric/run/keyfiles/carauction-network.bna", "-A", "hindalcoAdmin", "-C", "/home/composer/.composer/hindalcoAdmin/admin-pub.pem"], "delta": "0:00:02.203133", "end": "2018-06-21 15:51:07.794971", "msg": "non-zero return code", "rc": 1, "start": "2018-06-21 15:51:05.591838", "stderr": "✖ Starting business network definition. This may take a minute...", "stderr_lines": ["✖ Starting business network definition. This may take a minute..."], "stdout": "Starting business network from archive: /opt/gopath/bc1st/fabric/run/keyfiles/carauction-network.bna\nBusiness network definition:\n\tIdentifier: carauction-network@0.2.3-20180314171154\n\tDescription: Car Auction Business Network\n\nProcessing these Network Admins: \n\tuserName: hindalcoAdmin\n\nError: Error trying to instantiate composer runtime. Error: Error: 2 UNKNOWN: access denied: channel [firstchannel] creator org [hindalco]\nCommand failed", "stdout_lines": ["Starting business network from archive: /opt/gopath/bc1st/fabric/run/keyfiles/carauction-network.bna", "Business network definition:", "\tIdentifier: carauction-network@0.2.3-20180314171154", "\tDescription: Car Auction Business Network", "", "Processing these Network Admins: ", "\tuserName: hindalcoAdmin", "", "Error: Error trying to instantiate composer runtime. Error: Error: 2 UNKNOWN: access denied: channel [firstchannel] creator org [hindalco]", "Command failed"]}
to retry, use: --limit @/home/gauran/Gaurang/hyperledger/cello/src/agent/ansible/aws.retry
all I know from manual deployment, this error comes when admin certificates is miss matched with what peer stores.
I am trying to add a host from operator dashboard but whatever IP (in daemon URL)i provide it gives me an error? please advise which specific URL it is looking for. I am currently using operator and user on separate machines
@baohua, can you give me the link to the release update? I like to look it over and prepare for the amsterdam update.
@hightall are there any non-dashboard methods for getting the host(s) configured within the dashboard?
@baohua, never mind, I found it https://github.com/hyperledger/cello/releases
https://gerrit.hyperledger.org/r/#/c/23523/
if anybody's around, how do you know what path to export for nfs server address? (wrt kubernetes)
@baohua @hightall please take a look at this patch to fix ansible agent docker container image issue. https://gerrit.hyperledger.org/r/#/c/23537/
the problem was that ansible agent container image does not allow sudo, but during the setup process, often user has to be changed to sudo to do things. this is a problem, that patch fixed the issue and also removed an unused package.
Thanks for taking a look at.
@tongli do you know if anybody other than hightail is working on kubernetes agents?
I am not really sure.
hm, well, k8s host creation seems broken on master _regardless_ of input
hm, well, k8s host creation seems broken on master _regardless_ of input validity
/msg @hightail is there a good time to talk to you 1 on 1?
Hi all, Does anybody managed to install the recent Cello version into AWS via ansible scripts? Is it stable at all? I'm getting the error below. Can anybody help clarify ?
```TASK [Start network] ************************************************************************************************************************************************
fatal: [xx.xx.xx.xx]: FAILED! => {"changed": true, "cmd": ["docker", "run", "-v", "/home/ubuntu/.composer:/home/composer/.composer", "-v", "/opt/gopath/bc1st/fabric:/opt/gopath/bc1st/fabric", "hyperledger/composer-cli:next", "network", "start", "-c", "PeerAdmin@orga", "-a", "/opt/gopath/bc1st/fabric/run/keyfiles/carauction-network.bna", "-A", "orgaAdmin", "-C", "/home/composer/.composer/orgaAdmin/admin-pub.pem", "-A", "orgbAdmin", "-C", "/home/composer/.composer/orgbAdmin/admin-pub.pem"], "delta": "0:00:02.298221", "end": "2018-06-18 21:21:20.944139", "msg": "non-zero return code", "rc": 1, "start": "2018-06-18 21:21:18.645918", "stderr": "✖ Starting business network definition. This may take a minute...", "stderr_lines": ["✖ Starting business network definition. This may take a minute..."], "stdout": "Starting business network from archive: /opt/gopath/bc1st/fabric/run/keyfiles/carauction-network.bna\nBusiness network definition:\n\tIdentifier: carauction-network@0.2.3-20180314171154\n\tDescription: Car Auction Business Network\n\nProcessing these Network Admins: \n\tuserName: orgaAdmin\n\tuserName: orgbAdmin\n\nError: Error trying to instantiate composer runtime. Error: Error: 2 UNKNOWN: access denied: channel [firstchannel] creator org [orga]\nCommand failed", "stdout_lines": ["Starting business network from archive: /opt/gopath/bc1st/fabric/run/keyfiles/carauction-network.bna", "Business network definition:", "\tIdentifier: carauction-network@0.2.3-20180314171154", "\tDescription: Car Auction Business Network", "", "Processing these Network Admins: ", "\tuserName: orgaAdmin", "\tuserName: orgbAdmin", "", "Error: Error trying to instantiate composer runtime. Error: Error: 2 UNKNOWN: access denied: channel [firstchannel] creator org [orga]", "Command failed"]}
```
What is the most stable tag of Cello as of today? I just need to get it installed and demo the concept
Has joined the channel.
Has joined the channel.
Has joined the channel.
When I use the cello system, I can not add one peer or one orderer or one organization dynamically after the fabric network built.
So, I suggest adding this feature. Is this feature included in the development plan?
If the rules permit, we are willing to participate in the development of this function.
Managing ca is also very important,Should it also be supported in cello?
When I use the cello system, I can not add one peer or one orderer or one organization dynamically after the fabric network built.
So, I suggest adding this feature. Is this feature included in the development plan?
If the rules permit, we are willing to participate in the development of this function.
Managing ca is also very important,Should it also be supported in cello?
https://jira.hyperledger.org/browse/CE-391
@luckydogchina we're considering to support importing a part of existing network (e.g., peers of the same org) into cello by a config file. Do you think this is what u need? Feel free to start the work by some design proposal. Thanks!
[ ](https://chat.hyperledger.org/channel/cello?msg=HqBWHBhQMrhj8B4He) @baohua Ok, We are now developing projects based on the fabric project, a fabric network management system is very important. At present, we are analyzing and arranging requirements, and at the same time we are doing some program design. We will complete it as soon as possible, and then we will share the design plan and hope to be helpful.
[ ](https://chat.hyperledger.org/channel/cello?msg=HqBWHBhQMrhj8B4He) @baohua Ok, We are now developing projects based on the fabric project, a fabric network management system is very important. At present, we are analyzing and arranging requirements, and doing one program design. We will complete it as soon as possible, and then we will share the design plan and hope to be helpful.
yes, that will be helpful! thanks!
Has joined the channel.
Has joined the channel.
@baohua Is there a url of the public design document of the cello project currently.
@baohua Is there a url of the public design document of the cello project currently?
Has joined the channel.
Because , We want to extend the functionality of the cello based on the current project so that repetitive and unnecessary conflicts can be avoided
https://wiki.hyperledger.org/projects/cello/, here u may find some
thanks
@hightall .. I have now setup a worker node on a separate system. but still peers keeps restarting
https://gerrit.hyperledger.org/r/#/c/23589/
If I want to use docker swarm as a host, do I need ansible?
From the operator dashboard when we create a chain of type SOLO, what is the compose file that is used ? Is it "src/agent/docker/_compose_files/fabric-1.0/fabric-solo-4.yaml"?
Hi Cello Experts
I am getting this error ```a now has Error: 2 UNKNOWN: could not find chaincode with name 'a3b7065c10224befba78488d490e0f28-5b30d5a111b67a00260b8d84' - make sure the chaincode a3b7065c10224befba78488d490e0f28-5b30d5a111b67a00260b8d84 has been successfully instantiated and try again after the move``` while querying from user dashboard
query.png
I am even not able to use `invoke`
I can see the chaincode containers in my `worker node`
VirtualBox_Ubuntu_25_06_2018_17_23_44.png
in `dashboard` logs I am getting this error : ```cello-user-dashboard | 11:54:32 0|index | error: [client-utils.js]: sendPeersProposal - Promise is rejected: Error: 2 UNKNOWN: could not find chaincode with name 'a3b7065c10224befba78488d490e0f28-5b30d5a111b67a00260b8d84' - make sure the chaincode a3b7065c10224befba78488d490e0f28-5b30d5a111b67a00260b8d84 has been successfully instantiated and try again
cello-user-dashboard | 11:54:32 0|index | at new createStatusError (/node_modules/grpc/src/client.js:64:15)
cello-user-dashboard | 11:54:32 0|index | at /node_modules/grpc/src/client.js:583:15
cello-user-dashboard | 11:54:32 0|index | error: [client-utils.js]: sendPeersProposal - Promise is rejected: Error: 2 UNKNOWN: could not find chaincode with name 'a3b7065c10224befba78488d490e0f28-5b30d5a111b67a00260b8d84' - make sure the chaincode a3b7065c10224befba78488d490e0f28-5b30d5a111b67a00260b8d84 has been successfully instantiated and try again
cello-user-dashboard | 11:54:32 0|index | at new createStatusError (/node_modules/grpc/src/client.js:64:15)
```
seems like it trying to search chaincode with the name a3b7065c10224befba78488d490e0f28-5b30d5a111b67a00260b8d84
@hightall
I am trying to deploy a fabric sample chaincode written in GoLang
peer0.png
@baohua @hightall need your help review this patch https://gerrit.hyperledger.org/r/#/c/23537/ to fix the ansible agent bug which is missing sudo package.
is `make dockerhub` supposed to work?
@infowolfe It should work. but I normally just go to that directory , use docker build command
```DOCKERHUB_IMAGES = baseimage engine mongo nginx operator-dashboard user-dashboard watchdog ansible-agent```
``` infowolfe ~ cello ls dockerhub/latest/
ansible-agent baseimage engine mongo operator-dashboard user-dashboard watchdog```
the `nginx` image doesn't exist
the `nginx` image doesn't exist in gerrit
@hightall
Are you there?
@hightall could you please look into this. https://chat.hyperledger.org/channel/cello?msg=2nanke8faMPE5PNHX
@tongli Can you please look into this if you have some spare time.
@pankajcheema it's a bug, I will fix it in master branch.
Thank you very much @hightall
@hightall I have a question. Is it possible to use my own network in cello?
Like is there any way to configure your own network?
you mean import your fabric network into cello? @pankajcheema
yes yes
You got it
will support
currently no support
Ok
@hightall Can I use cello for my physical network? Like My network is running on core linux ubuntu (no docker, no docker swarm). It is just running with the help of `peer` and `orderer` executable
The network is running on 13 physical systems
interconnected with each other
Can I use cello for that?
sorry, I can't understand your mean.
@pankajcheema if you have few physical servers and wanting fabric to be up and running, you can use ansible agent.
@pankajcheema take a look at this doc https://github.com/hyperledger/cello/blob/master/docs/worker_ansible_howto.md
I mean Can I use cello if my network is not setup on docker, docker swarm, kubernetes, vsphere.. I am just using physical servers and `peers`, `orderers` are running using their executables
Ok So ansible is for that purpose only?
I mean especially for this purpose?
@pankajcheema yes, that is what I am saying. even if your physical servers are only having base clean ubuntu 16.04 running, ansible agent was created to help getting fabric up and running.
Wowo
Wow
it will even install docker and all necessary software for you. please take a look at that doc
Great
I will try that
@hightall please let me know when you fix the bug. So that I can try it again :)
Thanks again moderators :)
Have a great day ahead :)
ok
@pankajcheema read the entire doc and pay close attention to this section. https://github.com/hyperledger/cello/blob/master/docs/worker_ansible_howto.md#using-existing-servers
@hightall please take a quick look at my patch to fix the ansible agent docker container issue
@tongli sure
it is here https://gerrit.hyperledger.org/r/#/c/23537/
Thanks @tongli I will jump into it once I learn everything on docker. Right now I can not use query and invoke feature as @hightall accepted it as a bug. Once fixed , I will try it
@tongli +2
@hightall thanks a lot bro.
@hightall Cello is using fabric v1.0.5 and v1.1 is latest one
Am I right?
currently only support 1.0.5
ok
So Will it support 1.1?
hi all, I am facing this error:
E0625 21:54:31.229363635 13930 ssl_transport_security.cc:989] Handshake failed with fatal error SSL_ERROR_SSL: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed.
error: "Error: 14 UNAVAILABLE: Connect Failed\n
using Cello v0.8.0
can anyone help
Has joined the channel.
@hightall cello doesn't support swarm completely, does it?
What is the compose file used for creating a SOLO chain? Is it "src/agent/docker/_compose_files/fabric-1.0/fabric-solo-4.yaml"?
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/23657/
+2!
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/23705/
Hi All, I am getting error while running " ansible-playbook -i run/runhosts -e "mode=apply env=bc2nd" setupfabric.yml"
failed: [192.168.1.89] (item=ordererOrganizations) => {
"changed": true,
"cmd": "if [ -d 'ordererOrganizations' ]; then mv -f ordererOrganizations/* . && rm -rf ordererOrganizations; fi",
"delta": "0:00:00.002174",
"end": "2018-06-27 08:40:44.908000",
"invocation": {
"module_args": {
"_raw_params": "if [ -d 'ordererOrganizations' ]; then mv -f ordererOrganizations/* . && rm -rf ordererOrganizations; fi",
"_uses_shell": true,
"chdir": "/opt/gopath/bc2nd/fabric/keyfiles",
"creates": null,
"executable": null,
"removes": null,
"stdin": null,
"warn": true
}
},
"item": "ordererOrganizations",
"msg": "non-zero return code",
"rc": 1,
"start": "2018-06-27 08:40:44.905826",
"stderr": "mv: cannot move 'ordererOrganizations/orga' to './orga': Directory not empty",
"stderr_lines": [
"mv: cannot move 'ordererOrganizations/orga' to './orga': Directory not empty"
],
"stdout": "",
"stdout_lines": []
}
This is generated crypto-config.yml
---
OrdererOrgs:
- Name: orga
Domain: orga
Specs:
- Hostname: orderer1st-orga
PeerOrgs:
- Name: orga
Domain: orga
EnableNodeOUs: true
Specs:
- Hostname: peer1st-orga
- Hostname: peer2nd-orga
~
@pankajcheema you can try master branch code, after this patch merged https://gerrit.hyperledger.org/r/23715
@hightall How will I know the patch is merged. I have never used gerrit
the step is 1. master node: make start
2. worker node: MASTER_NODE_IP=xx.xx.xx.xx make setup-worker
you can see this review status, or you can download the patch by yourself, and patch it into the master branch
~~Ok
remember pull the latest code, and patch this patch
I will check the status
ok
and will pull the master code once patched
@Luke_Chen are you there?
https://gerrit.hyperledger.org/r/#/c/23715/
@hightall is it patched?
no, in reviewing
ok
Hi while using user-dashboard I am getting error `Apply Chain test fail` can anybody advise me the reason?
@Tfabric did you create the chain pool in operator dashboard? If there is no available chain, you can't apply chain
@hightall my operator dashboard is on other system and when I am providing Daemon URL it is giving me error
you need set the docker configuration to expose the 2375 port,so that master can visit docker api.
apologies for being difficult please advise me how can I set docker configuration
@hightall also: if I want to host my chains on Synology server and access them from system. Is it possible for me to do this?
error whilst adding host.png
I am getting this error whilst adding a host. I am using MacOS and my docker ports are configured accordingly. Can anybody advise me why I am still getting this error?
@Tfabric you check the dashboard logs?
@infowolfe how can I check logs?
@infowolfe got logs
`TTPConnectionPool(host='192.168.0.1', port=2375): Max retries exceeded with url: /version (Caused by ConnectTimeoutError(
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/23773/
https://gerrit.hyperledger.org/r/#/c/23775/
@hightall in cello is there any restriction orderers and peers are to be in different organization?
I am getting error while running " ansible-playbook -i run/runhosts -e "mode=apply env=bc2nd" setupfabric.yml"
failed: [192.168.1.89] (item=ordererOrganizations) => {
"changed": true,
"cmd": "if [ -d 'ordererOrganizations' ]; then mv -f ordererOrganizations/* . && rm -rf ordererOrganizations; fi",
"delta": "0:00:00.002174",
"end": "2018-06-27 08:40:44.908000",
"invocation": {
"module_args": {
"_raw_params": "if [ -d 'ordererOrganizations' ]; then mv -f ordererOrganizations/* . && rm -rf ordererOrganizations; fi",
"_uses_shell": true,
"chdir": "/opt/gopath/bc2nd/fabric/keyfiles",
"creates": null,
"executable": null,
"removes": null,
"stdin": null,
"warn": true
}
},
"item": "ordererOrganizations",
"msg": "non-zero return code",
"rc": 1,
"start": "2018-06-27 08:40:44.905826",
"stderr": "mv: cannot move 'ordererOrganizations/orga' to './orga': Directory not empty",
"stderr_lines": [
"mv: cannot move 'ordererOrganizations/orga' to './orga': Directory not empty"
],
"stdout": "",
"stdout_lines": []
}
This is generated crypto-config.yml
---
OrdererOrgs:
- Name: orga
Domain: orga
Specs:
- Hostname: orderer1st-orga
PeerOrgs:
- Name: orga
Domain: orga
EnableNodeOUs: true
Specs:
- Hostname: peer1st-orga
- Hostname: peer2nd-orga
~
Here orderer and peers are same organization orga
@rkrish82 maybe this question you can ask @tongli
@tongli Please help me to fix there, I am stuck since last days in this..
@tongli Please help me to fix this error, I am stuck since last days in this..
@rkrish82 yes. Please put the orderers in a different org. They will have their own certs anyway.
@rkrish82 the error seems to be in the step that agent wants to put all the certs together. So try to put the orderers in an org which has no peers like what the sample configuration files such as bc1st, bc2nd etc
I am at hackfeat, may not be able to respond to you very quickly.
Ok thanks Tangli. But it has to be orderers and peers can be in one org right? Why this restriction?
Also I am not really sure your env.
Not they do not have to be in the same org.
My suggestion is try that first. If it works, then it means there is a bug. I suspect you may have some other issues.
if orderers and peers in same org then i face this issue, otherwise there is no error
https://gerrit.hyperledger.org/r/#/c/23779/
Has joined the channel.
@hightall please advise is it possible to run operator dashboard from Synology server. It does not have Ubuntu but a proprietary version of Linux
@rkrish82 probably a bug. Go ahead open a bug report, then we can fix it.
User User_1 added by jwagantall.
Now, Is it supported that creating a new channel in the exist chain?
Now, Is it supported that creating a new channel in the exist chain?
or, is it developing?
Now, Is it supported that creating a new channel in the exist chain?
or, is it developing?
Now, Is it supported that creating a new channel in the exist chain?
or, is it developing?
sorry, I don't run `make setup-worker`
Has joined the channel.
Has joined the channel.
Clipboard - 2018年7月2日中午11点46分
Is this a bug in master project?
Is this a bug in master project?
when I create a new chain in solo type.
docker info:Containers: 7
Running: 7
Paused: 0
Stopped: 0
Images: 160
Server Version: 17.05.0-ce
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 370
Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9048e5e50717ea4497b757314bad98ea3763c145
runc version: 9c2d8d184e5da67c95d601382adf14862e4f2228
init version: 949e6fa
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 4.4.0-124-generic
Operating System: Ubuntu 16.04.4 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 7.68GiB
Name: whty0-To-be-filled-by-O-E-M
ID: GFAD:JTW6:6ZAE:CM4X:3JIE:76CV:W37J:LNB3:CPRO:FZAL:WOQ4:JB2X
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
127.0.0.0/8
Registry Mirrors:
http://e14ac66d.m.daocloud.io/
Live Restore Enabled: false
docker info:Containers: 7
Running: 7
Paused: 0
Stopped: 0
Images: 160
Server Version: 17.05.0-ce
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 370
Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9048e5e50717ea4497b757314bad98ea3763c145
runc version: 9c2d8d184e5da67c95d601382adf14862e4f2228
init version: 949e6fa
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 4.4.0-124-generic
Operating System: Ubuntu 16.04.4 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 7.68GiB
Name: whty0-To-be-filled-by-O-E-M
ID: GFAD:JTW6:6ZAE:CM4X:3JIE:76CV:W37J:LNB3:CPRO:FZAL:WOQ4:JB2X
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
127.0.0.0/8
Registry Mirrors:
http://e14ac66d.m.daocloud.io/
Live Restore Enabled: false
Is this a bug? It has been happened when I create a new chain based docker-cello-dashboard.
https://gerrit.hyperledger.org/r/#/c/23933/
Has joined the channel.
+2!
Has joined the channel.
Anybody know if its possible to deploy Iroha using Cello?
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/23943/
People, is there a proper(er!!) way update the chaincode on a Fabric that is built with *"mode=apply" awsk8s.yml* ?
People, is there a proper(er!!) way to update the chaincode on a Fabric that is built with *"mode=apply" awsk8s.yml* ?
Has joined the channel.
congrats and thanks to @ShuoWang for his first code contribution! https://gerrit.hyperledger.org/r/23793
congrats and thanks to @ShuoWang for his first code contribution to Cello! https://gerrit.hyperledger.org/r/23793
Has joined the channel.
https://jira.hyperledger.org/browse/CE-406 dynamic crypto files generation design, feel free to put your suggestion.
[ ](https://chat.hyperledger.org/channel/cello?msg=qN4zJy6xF9sm26uyf) @hightall , from the title, it seems interesting. but i can not know its exact purpose.
@qsmen currently the crpto config file is static, so for all deployed fabric network use same crypto files, it's not safe and not a good design, so we want to generate the crypto config file dynamic.
Clipboard - 2018年7月3日下午3点14分
@luckydogchina so what's your question?
what is the function of the Object store?
what is the function of the Object store? the crypto config files will be customized by the user?
in fabric service, there will be a initial service which will generate the crypto config files, then copy the files into the minio object store service in a bucket, and minio service mount a local path same with user dashboard service, and the directory is named with chain id, so user dashboard can find the crypto files through the chain id, and call fabric api with the crypto config files, and user dashboard can call minio api with sdk to generate tmp url for download crypto files for user, so user can use the crypto files call api by themselves.
use can download the crypto files and use them .
even user can upload crypto files
upload crypto files is used for import existed fabric chain.
so for this function, we will need a initialize container for generate crypto config files before fabric network containers started.
[ ](https://chat.hyperledger.org/channel/cello?msg=jrAMERB6AcNoocu3T) @hightall include configtx.yaml ?
@luckydogchina that's a problem, I forgot how to generate this file, and what's needed for generating this file.
this is detail for dynamic generation for crypto config
[ ](https://chat.hyperledger.org/channel/cello?msg=yEqXHF6cMEYi2vTwb) @hightall Thanks.
[ ](https://chat.hyperledger.org/channel/cello?msg=wSN2yBo9tHm62GWEA) @hightall ,the crypto config file, you mean the crypto-config.yaml? to prevent depolyed fabric netwrok using the same crytpo files, some random elements can be added when implementing cryptogen command, then different cyrpto files will be generated for different deployment.
[ ](https://chat.hyperledger.org/channel/cello?msg=wSN2yBo9tHm62GWEA) @hightall ,the crypto config file, you mean the crypto-config.yaml? to prevent depolyed fabric netwrok from using the same crytpo files, some random elements can be added when implementing cryptogen command, then different cyrpto files will be generated for different deployment.
@qsmen yes, use cyrpto-config.yaml with scripts byfn.sh to generate crypto config files and genesis block file
ok, thank you.
It is good that the crypto-config.yaml is formed dynamicaly in the new design. It is better if the following problem can be solved. If all crypto files are not generated distributedly, but are still generated by one entity and then copied to different entitiesm like peer, order or client app user, it is still not safe.
It is good that the crypto-config.yaml is formed dynamicaly in the new design. It is better if the following problem can be solved. If all crypto files are not generated distributedly, but are still generated by one entity and then copied to different entities like peer, order or client app user, it is still not safe.
yeah @qsmen
In fabric-ca channel, many people are concerned about this problem. using fabric-ca server/client to generate crypto file, form msp and then form config.yaml is discussed very much
@hightall , do you plan to include a solution to the problem in your new design?
@hightall , do you plan to give a solution to the problem in your new design?
@qsmen yes, I think this is a big problem for dynamic crypto config generation, I haven't tried to use the crypto config files generated by script byfn.sh, we need to test it. maybe will have many more problem.
great! I will keep eyes on this.
Is it possible to have multiple master-nodes using Cello?
Also is there a limit of 4 chains per node?
I has created a topic : https://lists.hyperledger.org/g/cello/message/128, welcome to discuss.
@baohua @hightall I found a bug/improvement for the ansible agent. and fixed it in this patch set. https://gerrit.hyperledger.org/r/#/c/23983/ please review when you have few minutes. Thanks.
I've created a StackOverflow question, if anybody is able to help, you'd be saving me a lot of headaches. https://stackoverflow.com/questions/51155391/hyperledger-cello-multiple-master-nodes
Has joined the channel.
Screenshot from 2018-07-03 14-54-31.png
Thank you.
@luckydogchina usually, some jira issue or design spec will be better to collect comments :)
@tongli +2!
ok
I move the topic to cello jira https://jira.hyperledger.org/browse/CE-408
I has moved the topic to cello jira https://jira.hyperledger.org/browse/CE-408
https://gerrit.hyperledger.org/r/#/c/24029/
[ ](https://chat.hyperledger.org/channel/cello?msg=ekrt8PLKRrJcYJHuC) @Vanitha If you didnt figure it out username admin, password pass, then click the [EN] button in the top right
nosmartcontract.jpg
I have a question I git clone the master code and use docker host way to install cello.But I don;t find smart contract in the left menu.In the provide tuorial ,it has smart contract in the left menu.What I am missing?
[ ](https://chat.hyperledger.org/channel/cello?msg=BbESsvDd2mxwp7YaD) I have a question I git clone the master code and use docker host way to install cello.But I don;t find smart contract in the left menu.In the provide tuorial ,it has smart contract in the left menu.What I am missing? I I saw the document, there has two url one for operator and another for user dashboard. operation dashboard don;t has chaincode management feature.
I saw the document, there has two url one for operator and another for user dashboard. operation dashboard don;t has chaincode management feature.
Has joined the channel.
@here Dear all, we will have project meeting at 8:30PM--9:00PM (UTC+8) this Friday evening , welcome to join, and please help update the agenda at: https://wiki.hyperledger.org/projects/cello/meetings/20180706. Thanks!
What do we need to prepare in advance? Just install zoom?
bluepoint.jpg
Also I have a follow up question, can we upgrade the chaincode in the cello?
@nemo 8 Is user name and pwd are same for Operator dashboard and user dashboard
I think for user dashboard we should be able to login with the user we have created in operator dashboard
Please correct me if I am wrong
[ ](https://chat.hyperledger.org/channel/cello?msg=AK2w3kzXcouP4DxqS) Anyone?
Has left the channel.
I have integrated multi-orgs on single physical machine now I would like to integrate with Cello, I have master and worker nodes on two different physical machines and have added host and also added a new chain, but how can I integrate that with the network I already have
Thank you
https://gerrit.hyperledger.org/r/#/c/24149/
https://gerrit.hyperledger.org/r/#/c/24199/
Has joined the channel.
Hi All,
The cello dashboard, menu and panels depicted in the documentation at https://cello.readthedocs.io are significantly different than the ones I am seeing after installing master and worker nodes.
Also, chain is not getting deployed. Always get some kind of server.
Experts, please shed some light on this?
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/24233/
+2!
I've merged that patch.
Has joined the channel.
Has joined the channel.
Has joined the channel.
@tongli Is cello uses docker swarm for fabric setup in physical servers? I am not able establish connection with kafka to zookeeper. Getting this error {"log":"org.I0Itec.zkclient.exception.ZkException: Unable to connect to zookeeper1st:2181, {"log":"Caused by: java.net.UnknownHostException: zookeeper1st: Temporary failure in name resolution
@tongli Is cello uses docker swarm for fabric setup in physical servers? I am not able establish connection with kafka to zookeeper. Getting this error {"log":"org.I0Itec.zkclient.exception.ZkException: Unable to connect to zookeeper1st:2181, {"log":"Caused by: java.net.UnknownHostException: zookeeper1st: Temporary failure in name resolution...
@tongli Is cello uses docker swarm for fabric setup in physical servers? I am not able establish connection with kafka to zookeeper. Getting this error {"log":"org.I0Itec.zkclient.exception.ZkException: Unable to connect to zookeeper1st:2181, {"log":"Caused by: java.net.UnknownHostException: zookeeper1st: Temporary failure in name resolution...
@tongli Is cello uses docker swarm for fabric setup in physical servers?
HI, I have installed chaincode in Hyperledger cello but unable to create chain. it is giving me error, Please help!
@rkrish82 no cello ansible agent does not use docker swarm. it sets up its own overlay network to communicate with other containers on the network regardless physical or virtual servers
Has joined the channel.
@baohua @hightall I replaced two deprecated tasks in ansible agent due to ansible version upgrades. please take a look at this change. https://gerrit.hyperledger.org/r/#/c/24355/ thanks
+2!
Has joined the channel.
hello
Clipboard - July 13, 2018 5:58 PM
and always reproduced by Invoke smart contract with facar sample.
user dashboard always crashed while invoke smart contract (facar - creat car), error msg is : Failed to send transaction and get notifications within the timeout period.
Has joined the channel.
Has joined the channel.
Hi all ... Is there a specific reason why master node cannot be in k8s ?
Hi all ... Is there a specific reason why master node cannot be in k8s ?
@ageekymonk nobody limits that. master services utilized docker to run, hence it's OK to run them in k8s.
Thanks @baohua
Nominate Lukechen (@LordGoodman) as new project maintainer as his continuous contributions to CELLO: https://gerrit.hyperledger.org/r/24429. @hightall @tongli
@Luke_Chen
Hi all, I tried to deploy cello-master using helm in my minikube. The volume gets created but the minio is not able to initialize.
minio-container-error
looks like the storage class that we are using is `cello-minio` but it is not defined.
I think there is a missing storage class definition in helm cello-master
@ageekymonk let me take a look.
@ageekymonk let me check first
Has joined the channel.
Hi guys
i've noticed that wiki page tells that Iroha is supported but I haven't found anything in the docs
Do you have any support or plans to support Iroha?
Has joined the channel.
I keep getting
Name To resolve: peer0.org1.example.com.
Where would I address these
Has joined the channel.
Has joined the channel.
Hi everyone - We are pinning to a particular tag 'x86_64-1.1.0-preview' for kafka and couchdb docker images that are getting pulled from docker hub. Looks like they aren't there anymore. Were they taken down for a reason?
Hi everyone - We are pinning to a particular tag 'x86_64-1.1.0-preview' for kafka and couchdb docker images that are getting pulled from docker hub. Looks like they aren't there anymore. Were they taken down for a reason?
Hi everyone - We are pinning to a particular tag 'x86_64-1.1.0-preview' for kafka and couchdb docker images that are getting pulled from docker hub. Looks like they aren't there anymore. Were they taken down for a reason?
```
failed: [34.237.2.7] (item={u'flag': [u'zookeeper2nd'], u'name': u'fabric-zookeeper:x86_64-1.1.0-preview'}) => {"changed": true, "cmd": ["docker", "pull", "hyperledger/fabric-zookeeper:x86_64-1.1.0-preview"], "delta": "0:00:00.150682", "end": "2018-07-17 23:39:20.680736", "failed": true, "item": {"flag": ["zookeeper2nd"], "name": "fabric-zookeeper:x86_64-1.1.0-preview"}, "msg": "non-zero return code", "rc": 1, "start": "2018-07-17 23:39:20.530054", "stderr": "Error response from daemon: manifest for hyperledger/fabric-zookeeper:x86_64-1.1.0-preview not found", "stderr_lines": ["Error response from daemon: manifest for hyperledger/fabric-zookeeper:x86_64-1.1.0-preview not found"], "stdout": "", "stdout_lines": []}
failed: [34.206.1.198] (item={u'flag': [u'zookeeper3rd'], u'name': u'fabric-zookeeper:x86_64-1.1.0-preview'}) => {"changed": true, "cmd": ["docker", "pull", "hyperledger/fabric-zookeeper:x86_64-1.1.0-preview"], "delta": "0:00:00.144442", "end": "2018-07-17 23:39:20.839092", "failed": true, "item": {"flag": ["zookeeper3rd"], "name": "fabric-zookeeper:x86_64-1.1.0-preview"}, "msg": "non-zero return code", "rc": 1, "start": "2018-07-17 23:39:20.694650", "stderr": "Error response from daemon: manifest for hyperledger/fabric-zookeeper:x86_64-1.1.0-preview not found", "stderr_lines": ["Error response from daemon: manifest for hyperledger/fabric-zookeeper:x86_64-1.1.0-preview not found"], "stdout": "", "stdout_lines": []}
```
Has joined the channel.
Has joined the channel.
Hi there !
Is it possible to add peers and organisation dynamically using cello and/or is it under consideration?
Has joined the channel.
@shrey.techie no, you need to do that manually now, cello has no automatic support on this yet.
@here This week's meeting will be canceled, pls feel free to discuss offline and enjoy the week!
@shrey.techie ansible agent has plan to do that soon. The idea is simple
@shrey.techie ansible agent can be used to deploy one or many org peers independently.
Once they are done, we need another tool to allow any orgs who already have their own ca and peers to join an existing channel
That second piece has been discussed. Will start development very soon.
Once that piece is done, an existing fabric network can grow easily.
Which can run anywhere.
When I try to run transactions on my cello, I get a repeated error
Name To resolve: peer0.org1.example.com.
Can I, or where do I find this org settings to change it?
Has joined the channel.
http://141.198.3.66:8081/api/chain/5b4f45afebe400003b98997c?id=5b4f45afebe400003b98997c&type=summary
An error occurred on the server. Please check the server.
From Server debug log
DEBU[2018-07-18T08:54:53.752693966-05:00] [resolver] lookup name peer0.org1.example.com. present without IPv6 address
DEBU[2018-07-18T08:54:53.768580225-05:00] Name To resolve: peer0.org1.example.com.
http://xxx.xxx.xxx.xxx:8081/api/chain/5b4f45afebe400003b98997c?id=5b4f45afebe400003b98997c&type=summary
An error occurred on the server. Please check the server.
From Server debug log
DEBU[2018-07-18T08:54:53.752693966-05:00] [resolver] lookup name peer0.org1.example.com. present without IPv6 address
DEBU[2018-07-18T08:54:53.768580225-05:00] Name To resolve: peer0.org1.example.com.
This occurs after I click info from within my chain list
Hi - I am wondering if there is currently a way that the cello project aligns itself with the different releases of fabric. How do I know what version of cello would support changes with release-1.2 vs. release-1.0, etc.? Is there any plan to sync the release versions in the different projects?
@tongli @baohua @hightall ^^^
@latitiah I think it is a good idea and cello should support and make that change.
Though cello ansible agent does support current and previous releases of fabric.
@latitiah u can find all supported releases through the dashboard, currently, it's 1.0 and 1.1. 1.2 is on the road!
@tongli That makes sense. I am beginning to see some divergence such that tagging now would make things a bit easier to note what cello version supports what fabric version
@baohua Is each cello build tagged accordingly?
@tongli That makes sense. I am beginning to see some divergence such that tagging now would make things a bit easier to note what cello version supports what fabric version
@baohua Is each cello build tagged accordingly?
@baohua Is there any plan to support dynamically add orgs and peers in the future?
and one more question regarding the centralized deploy V.S distributed deploy, does Cello support distributed deploy worker node? now just using Docker to deploy worker node on a single physical server, but actually most deploys need be distributed on several physical server. .
Has joined the channel.
Has joined the channel.
@tongli Failed to create a kubernetes client: Error loading config file "/kube-config": read /kube-config: is a directory
@tongli Failed to create a kubernetes client: Error loading config file "/kube-config": read /kube-config: is a directory
This exception is thrown when I create k8s-dns.
Hi I am new to cello and just curious to know whether the organization artifacts are being generated separately of the different IP addresses for different nodes or its getting generated at a place and then gets distributed?
Has joined the channel.
Has joined the channel.
@WadeLu yes, there's in plan now. Certainly you can have distributed work nodes. try our latest k8s agent feature, which manages fabric on k8s cluster.
@Luke_Chen Did you get a chance to look at helm for cello-master ?
ok, thanks @baohua that's great feature! And I will try k8s
Hi Cello! my work lately has been focused on using kubernetes/helm as the deployment coordination tool for hyperledger networks.
I'd like to talk with you about how Cello could be used to help make kubernetes/helm easier to apply to hyperledger.
basically, most companies and cloud providers are moving to kubernetes/helm and I was wondering how Cello can be modified to meet that demand.
have you had any talks in this direction?
Hi all
When I try to manually create a chain I get an error like this one
ERROR: for cli Cannot start service cli: oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:359: container init caused \"rootfs_linux.go:54: mounting \\\"/opt/cello/fabric-1.0/solo/configtx.yaml\\\" to rootfs \\\"/var/lib/docker/overlay2/c3777abc7c25b03bef868702df86c798a969f3cac05f73b2208b8c553c4c51fd/merged\\\" at \\\"/var/lib/docker/overlay2/c3777abc7c25b03bef868702df86c798a969f3cac05f73b2208b8c553c4c51fd/merged/etc/hyperledger/fabric/configtx.yaml\\\" caused \\\"not a directory\\\"\""
Wondering what causes Docker to create a directory instead of a file for “configtx.yaml” ?
Any ideas ?
Has joined the channel.
Has joined the channel.
Hi everyone, I am new to cello, I want to quickstart into cello and understand its capabilities, Can anyone pls suggest me a quick tutorial!
Has joined the channel.
Hi,
I've noticed that wiki page tells that Iroha is supported but I haven't found anything in the docs
Do you have any support or plans to support Iroha?
Has joined the channel.
Has joined the channel.
@ageekymonk Not yet, will take a look today later.
Thanks @Luke_Chen
Has joined the channel.
Has joined the channel.
@dhuseby great to hear that. Would you be able to attend our weekly meeting (12:30 UTC every Friday)?
https://gerrit.hyperledger.org/r/#/c/24653/
@baohua Now, the cello can run with k8s?
@luckydogchina yes
please check out the tutorial
[ ](https://chat.hyperledger.org/channel/cello?msg=eKPMpfcbDfrA49W7K) @Luke_Chen ??
[ ](https://chat.hyperledger.org/channel/cello?msg=eKPMpfcbDfrA49W7K) @Luke_Chen ?? which version ? 0.8 ? master?
master
[ ](https://chat.hyperledger.org/channel/cello?msg=Dc3XDCze4dg7BMZ27) @Luke_Chen Thanks
[ ](https://chat.hyperledger.org/channel/cello?msg=Dc3XDCze4dg7BMZ27) @Luke_Chen master images had been pushed?
[ ](https://chat.hyperledger.org/channel/cello?msg=eKPMpfcbDfrA49W7K) @Luke_Chen Did cello support deploy multi hosts with k8s?
@giniz what did you mean multiple hosts?
@giniz Separated Fabric nodes into different machines?
Has joined the channel.
Since upgrading to Fabric 1.1, when running a deploy, I keep getting:
```
Error: got unexpected status: SERVICE_UNAVAILABLE -- will not enqueue, consenter for this channel hasn't started yet\n
```
Has anyone seen this issue, OR can i see a sample working bc1st.yaml file someone is leveraging?
Fixed it, and its a rough one. Boto's newest release was breaking some of my aws connections. I Pinned boto to an older release from 4 days ago and now i can get back to work 😅
[ ](https://chat.hyperledger.org/channel/cello?msg=SyLrHcgZBRFmzoQSD) @Luke_Chen yes
[ ](https://chat.hyperledger.org/channel/cello?msg=BMe3mGpKE49yw3pv2) @Russ.corsha kafka cluster not ready when orderer receives request
@Luke_Chen
@Luke_Chen I pull the last cello images , and `make start`, but when I add k8s in host-ui, It is failure .
Clipboard - July 25, 2018 10:18 AM
@luckydogchina better use the cert/key credential type
and disable the SSL cert
I am working on SSL cert bug fixing
Certificate Content == public cert?
yes
Clipboard - July 25, 2018 10:36 AM
server
server.crt?
It should be server.crt
ok, let me try it
Clipboard - July 25, 2018 10:38 AM
maybe, My k8s has some problems?
@Luke_Chen
Are you using the latest docker images ?
if not, please kill all the Cello containers and use `DEV=true make start` command to restart cello
@Luke_Chen the server.crt is k8s-tls-cert
@Luke_Chen the server.crt is k8s-tls-cert and not authorization-cert
ok:joy:
you should use the authorization one
Clipboard - July 25, 2018 11:37 AM
@Luke_Chen cello send a request cycle.
@Luke_Chen cello send a request cycle.And add host operation is blocked.
Has joined the channel.
@luckydogchina what's the problem?
@baohua thanks for closing some issues which have been finished.
Has joined the channel.
Has joined the channel.
sure~
@tongli Is cello provides a way to setup fabric network by running fabric components as a service/process? Is there any advantage to go for docker based?
Has joined the channel.
@rkrish82 the advantages are many, pls refer to docker.com. In short, for all next generation apps that can run inside large scale env, containerization is a better way.
@hightall would u like to help look at https://jira.hyperledger.org/browse/CE-420?
https://gerrit.hyperledger.org/r/#/c/24795/
[ ](https://chat.hyperledger.org/channel/cello?msg=JkmH2RWzvLLFaSmHG) @baohua The problem always happens in the docker environment. I think it is caused by docker volumes that mounted from one same path.
Has joined the channel.
```
2018-07-27 19:08:42.794 UTC [chaincode] processStream -> ERRO 048 Error handling chaincode support stream: rpc error: code = Canceled desc = context canceled
2018-07-27 19:09:12.762 UTC [chaincode] ExecuteChaincode -> ERRO 049 timeout expired while executing transaction
github.com/hyperledger/fabric/core/chaincode.(*ChaincodeSupport).Execute
/opt/gopath/src/github.com/hyperledger/fabric/core/chaincode/chaincode_support.go:813
github.com/hyperledger/fabric/core/chaincode.Execute
.
.skipping stdout
.
failed to execute transaction
error executing chaincode
2018-07-27 19:09:12.762 UTC [endorser] simulateProposal -> ERRO 04a [firstchannel][b2019bbb] failed to invoke chaincode name:"firstchaincode" , error: timeout expired while executing transaction
github.com/hyperledger/fabric/core/chaincode.(*ChaincodeSupport).Execute
/opt/gopath/src/github.com/hyperledger/fabric/core/chaincode/chaincode_support.go:813
github.com/hyperledger/fabric/core/chaincode.Execute
...
2018-07-27 19:31:08.398 UTC [chaincode] processStream -> ERRO 056 Error handling chaincode support stream: rpc error: code = Canceled desc = context canceled
2018-07-27 19:31:38.358 UTC [chaincode] ExecuteChaincode -> ERRO 057 timeout expired while executing transaction
github.com/hyperledger/fabric/core/chaincode.(*ChaincodeSupport).Execute
```
Hey guys, running into an error getting chaincode started. Im timing out but im not sure why. Im running on t2.xlarges with 1 peer and 1 order (which should be super overkill)
```
2018-07-27 19:08:42.794 UTC [chaincode] processStream -> ERRO 048 Error handling chaincode support stream: rpc error: code = Canceled desc = context canceled
2018-07-27 19:09:12.762 UTC [chaincode] ExecuteChaincode -> ERRO 049 timeout expired while executing transaction
github.com/hyperledger/fabric/core/chaincode.(*ChaincodeSupport).Execute
/opt/gopath/src/github.com/hyperledger/fabric/core/chaincode/chaincode_support.go:813
github.com/hyperledger/fabric/core/chaincode.Execute
.
.skipping stdout
.
failed to execute transaction
error executing chaincode
2018-07-27 19:09:12.762 UTC [endorser] simulateProposal -> ERRO 04a [firstchannel][b2019bbb] failed to invoke chaincode name:"firstchaincode" , error: timeout expired while executing transaction
github.com/hyperledger/fabric/core/chaincode.(*ChaincodeSupport).Execute
/opt/gopath/src/github.com/hyperledger/fabric/core/chaincode/chaincode_support.go:813
github.com/hyperledger/fabric/core/chaincode.Execute
...
2018-07-27 19:31:08.398 UTC [chaincode] processStream -> ERRO 056 Error handling chaincode support stream: rpc error: code = Canceled desc = context canceled
2018-07-27 19:31:38.358 UTC [chaincode] ExecuteChaincode -> ERRO 057 timeout expired while executing transaction
github.com/hyperledger/fabric/core/chaincode.(*ChaincodeSupport).Execute
```
Is there a way in fabric to get peers to load chaincode on initial startup as opposed to the first chaincode request?
Is there a way in cello to get peers to load chaincode on initial startup as opposed to the first chaincode request?
@Russ.corsha after instantiate, you can do chain code query against each peer to make sure each peer has the image and ready to go.,
@tongli thanks!
@baohua there was a bug introduced in an earlier patch which was merged. I fixed it in this patch. Please take a look https://gerrit.hyperledger.org/r/24881
@Luke_Chen @hightall please take a look as well https://gerrit.hyperledger.org/r/24881
merged, thanks!
Has joined the channel.
Thanks
Has joined the channel.
I'm working thru the Tutorial to learn Cello. Questions:
I'm working thru the Tutorial to learn Cello. Questions:
1.) After "Adding" a Chain (a Fabric) in the Operator Dashboard, I go on to "Start" the fabric as well (using the "More" dropdown on RHS) even though that's not mentioned in the tutorial. Please tell me if I'm making a mistake.
2.) In the Operator Dashboard, I "Added" both a Fabric 1.0 and a Fabric 1.1. However, in the User Dashboard when I go to Apply a Chain, I only get an option for a Fabric 1.0. Is there some way to get the other network listed? I assume I'm using the "Basic" Dashboards since I don't supply any options on the "make start" command.
I'm working thru the Tutorial to learn Cello. Questions:
1.) After "Adding" a Chain (a Fabric) in the Operator Dashboard, I go on to "Start" the fabric as well (using the "More" dropdown on RHS) even though that's not mentioned in the tutorial. Please tell me if I'm making a mistake.
2.) In the Operator Dashboard, I "Added" both a Fabric 1.0 and a Fabric 1.1. However, in the User Dashboard when I go to Apply a Chain, I only get an option for a Fabric 1.0. Is there some way to get the other network listed? I assume I'm using the "Basic" Dashboards since I don't supply any options on the "make start" command.
3.) After Installing the example02 chaincode and confirming a status of Installed, I don't see any options for instantiating the chaincode. The option to Invoke/Query against the chaincode app is disabled.
Thanks for any help - Kip.
@kipharris 1) by default, the chain will start after adding. 2) pls feel free to open a jira bug; 3) is there any error in the log?
Has joined the channel.
@baohua I opened CE-427 for item (2). I cannot reproduce my item (3). Thanks for the help.
This post explains how I worked through a problem that I've seen reported here in three other posts in last few months. The problem is characterized by an error of the following sort:
"Cannot start service cli: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/opt/cello/fabric-1.0/solo/configtx.yaml\\\" to rootfs \\\"/var/lib/docker/overlay2/6ed...b1d/merged\\\" at \\\"/var/lib/docker/overlay2/6ed6...b1d/merged/etc/hyperledger/fabric/configtx.yaml\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type"
When I encountered this, I found that configtx.yaml had changed from a file into being an empty directory. Very strange. Although I don't have insight into root cause, I have noticed this happening after I stop and later restart either the Master, the Worker, or maybe both. It may be that I am not doing something that I should be doing to shutdown the worker "cleanly", so that it can be restarted without losing state. For reference, my worker is a Docker host. I am running my fabrics on an internal IBM cloud platform ("Fyre") where we are encouraged to suspend the VM's when we are not using them.
I have found that by removing all images on both Master and Worker, and then re-pulling them, it clears the problem. I suspect I only need to reset the state on the NFS file system, but haven't explored the issue further.
Guidance from the experts on gracefully suspending and resuming a running system may be helpful. I'm using the 'make {start | stop}' on the Master, but don't know what I should be doing on the Worker. Thanks.
I want to create a Fabric 1.1 or 1.2 network that spans multiple hosts. I'm comfortable using the Cello Docker worker, and wondering whether I can achieve my objective by creating multiple Docker workers. My motivation is that I haven't yet been able to create a multi-host k8s environment on my available computing resources. If I can get the job done using just Docker workers, then I could defer the task of getting a k8s platform setup.
@hightall could you help have a look at https://jira.hyperledger.org/browse/CE-427?
Has joined the channel.
Has left the channel.
Has joined the channel.
Screenshot 2018-08-01 at 11.21.58 PM.png
Has joined the channel.
Hi Experts,i am trying to install cello based on Fabric 1.0.5,but met error of operater dashboard when installing, is there any suggestion thanks
error log
cello-operator-dashboard | npm ERR! path /app/package.json
cello-operator-dashboard | npm ERR! code ENOENT
cello-operator-dashboard | npm ERR! errno -2
cello-operator-dashboard | npm ERR! syscall open
cello-operator-dashboard | npm ERR! enoent ENOENT: no such file or directory, open '/app/package.json'
cello-operator-dashboard | npm ERR! enoent This is related to npm not being able to find a file.
cello-operator-dashboard | npm ERR! enoent
i am using root user
cello.JPG
[ ](https://chat.hyperledger.org/channel/cello?msg=G3CYNtmvbEGyyfzmp) @owen.liu I am having the same issue as well.
Screenshot 2018-08-02 at 3.02.52 PM.png
Has joined the channel.
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/25097/, welcome to review
Has joined the channel.
for anyone struggling with getting the operator dashboard up - https://jira.hyperledger.org/browse/CE-371?focusedCommentId=48410&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-48410
also, the way to have the containers up and running locally would be - ```make docker
make start```
Has joined the channel.
@tongli @baohua How make peer as a endorsing peer in cello? What changes are required peer argument?
They should be endorsing peers by default.
All peers should be able to endorse by default.
Then it means that even anchor peers can do endorsement job?
Has joined the channel.
@hightall In out latest code, after I ran `make setup-master` and `make start` command, the operator dashboard came out whit this error ``` npm ERR! path /app/package.json
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/app/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-06T12_51_31_353Z-debug.log
npm ERR! path /app/package.json
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/app/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-06T12_51_35_982Z-debug.log
npm ERR! path /app/package.json
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/app/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-06T12_51_38_470Z-debug.log
npm ERR! path /app/package.json
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/app/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-06T12_51_41_050Z-debug.log
npm ERR! path /app/package.json
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/app/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-06T12_51_42_867Z-debug.log
npm ERR! path /app/package.json
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/app/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-06T12_51_45_480Z-debug.log
npm ERR! path /app/package.json
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/app/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-06T12_51_49_716Z-debug.log```
@hightall In the latest code, after I ran `make setup-master` and `make start` command, the operator dashboard came out whit this error ``` npm ERR! path /app/package.json
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/app/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-06T12_51_31_353Z-debug.log
npm ERR! path /app/package.json
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/app/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-06T12_51_35_982Z-debug.log
npm ERR! path /app/package.json
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/app/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-06T12_51_38_470Z-debug.log
npm ERR! path /app/package.json
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/app/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-06T12_51_41_050Z-debug.log
npm ERR! path /app/package.json
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/app/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-06T12_51_42_867Z-debug.log
npm ERR! path /app/package.json
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/app/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-06T12_51_45_480Z-debug.log
npm ERR! path /app/package.json
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/app/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-06T12_51_49_716Z-debug.log```
@hightall In the latest code, after I ran `make setup-master` and `make start` command, the operator dashboard came out with this error ``` npm ERR! path /app/package.json
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/app/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-06T12_51_31_353Z-debug.log
npm ERR! path /app/package.json
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/app/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-06T12_51_35_982Z-debug.log
npm ERR! path /app/package.json
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/app/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-06T12_51_38_470Z-debug.log
npm ERR! path /app/package.json
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/app/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-06T12_51_41_050Z-debug.log
npm ERR! path /app/package.json
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/app/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-06T12_51_42_867Z-debug.log
npm ERR! path /app/package.json
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/app/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-06T12_51_45_480Z-debug.log
npm ERR! path /app/package.json
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/app/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-06T12_51_49_716Z-debug.log```
do you have any idea?
Has joined the channel.
Hi try commenting out this line in docker-compose.yml: command: bash -c "ln -sf /node_modules . && npm run start"
It is in line 60
I'm stall in in hyperledger cello-user-dashboard, when I click in Info on a chain I got Error 500 with a meesage that says: 请求错误 500: http://10.254.254.203:8081/api/chain/5b684da9e318e80048dda06d?id=5b684da9e318e80048dda06d&type=summary
服务器发生错误,请检查服务器。
Any clue about how to continue?
https://gerrit.hyperledger.org/r/#/c/25167/
@Luke_Chen @janmpo pls try this fix.
Has joined the channel.
Thanks for your support
It works!
Clipboard - August 7, 2018 2:50 PM
I clone the master cello
Has joined the channel.
Then : bash setup.sh
make start
the cello-dashboard logs show the wrongs
version : master commit ffb5608125cfac8ff35d582ae5aa24f6086ea13a
cello version : master commit ffb5608125cfac8ff35d582ae5aa24f6086ea13a
If I run `MODE=dev make start`, it is ok
Clipboard - August 7, 2018 3:04 PM
Clipboard - August 7, 2018 3:04 PM
Clipboard - August 7, 2018 3:04 PM
Clipboard - August 7, 2018 3:04 PM
Clipboard - August 7, 2018 3:05 PM
Clipboard - August 7, 2018 3:05 PM
Clipboard - August 7, 2018 3:05 PM
https://gerrit.hyperledger.org/r/#/c/25167/ has been merged! Thanks and congrats @yzhivkov for his first merging.:slight_smile:
https://gerrit.hyperledger.org/r/#/c/25167/ has been merged! Thanks and congrats @yzhivkov for his first merging. :slight_smile:
Cello error.png
Has joined the channel.
Screenshot 2018-08-07 at 2.24.46 PM.png
Getting this error in cello-operator-dashboard container after using make start. Any suggestion?
@jk.ananth That problem has been fixed by this patch https://gerrit.hyperledger.org/r/#/c/25167/
which is just merged
try out the latest code!
hi everyone!
I'm trying to get a fixed-size cluster running in k8s, but I am a newcomer to Fabric/Cello. If I kubectl apply all the files in https://github.com/hyperledger/cello/tree/master/src/agent/k8s/setup-scripts/manifests/fabric-1.0/local, will that work?
Has joined the channel.
Hi please review the user guide of Kubernetes agent https://drive.google.com/file/d/1ufLXf8EVZyps1w5lWu_ua-aXwKJqi3vQ/view?usp=sharing
Has joined the channel.
have cello integrated with composer?
@Luke_Chen pls add the link to the project wiki
@giniz not yet. Do you like the feature? Would you like to offer a hand?
Has joined the channel.
Has joined the channel.
Clipboard - August 8, 2018 2:27 PM
@Luke_Chen would you like to help have a look?
@ameijer Hi, Are you using the NFS server that cello provided?
If yes, than I think there is some thing wrong with your network, cause NFS server and Kubernestes cluster can not communicate with each other.
[ ](https://chat.hyperledger.org/channel/cello?msg=JSHfPxvAb4cFubSAa) @Luke_Chen @;
[ ](https://chat.hyperledger.org/channel/cello?msg=JSHfPxvAb4cFubSAa) @Luke_Chen @Luke_Chen tried with the latest code. Still got the same error in logs
ngoose/lib/mongoose.js:77:7"]
cello-user-dashboard | 2018-08-09 06:49:58,027 INFO 85 [egg-mongoose] mongodb://dashboard_mongo:27017/user_dashboard connected successfully
cello-user-dashboard | 2018-08-09 06:49:58,029 INFO 85 [egg-mongoose] instance[0] start successfully
cello-user-dashboard | 2018-08-09 06:49:58,029 INFO 85 [egg:core:ready_stat] end ready task /node_modules/egg-mongoose/lib/mongoose.js:77:7, remain ["/node_modules/egg-watcher/app.js:15:7"]
cello-user-dashboard | 2018-08-09 06:49:58,033 INFO 85 [egg-watcher:app] watcher start success
cello-user-dashboard | 2018-08-09 06:49:58,033 INFO 85 [egg:core:ready_stat] end ready task /node_modules/egg-watcher/app.js:15:7, remain []
cello-user-dashboard | 2018-08-09 06:49:58,037 INFO 80 [egg-mongoose] mongodb://dashboard_mongo:27017/user_dashboard connected successfully
cello-user-dashboard | 2018-08-09 06:49:58,039 INFO 80 [egg-mongoose] instance[0] start successfully
cello-user-dashboard | 2018-08-09 06:49:58,044 INFO 80 [egg-watcher:app] watcher start success
cello-user-dashboard | 2018-08-09 06:49:58,044 INFO 80 [egg:core:ready_stat] end ready task /node_modules/egg-mongoose/lib/mongoose.js:77:7, remain ["/node_modules/egg-watcher/app.js:15:7"]
cello-user-dashboard | 2018-08-09 06:49:58,044 INFO 80 [egg:core:ready_stat] end ready task /node_modules/egg-watcher/app.js:15:7, remain []
cello-user-dashboard | 2018-08-09 06:49:58,048 INFO 64 [master] app_worker#2:85 started at 8081, remain 1 (1340ms)
cello-user-dashboard | 2018-08-09 06:49:58,056 INFO 64 [master] app_worker#1:80 started at 8081, remain 0 (1348ms)
cello-user-dashboard | 2018-08-09 06:49:58,057 INFO 64 [master] egg started on http://127.0.0.1:8081 (1908ms)
cello-mongo exited with code 1
cello-mongo | chown: changing ownership of '/data/db': Operation not permitted
cello-mongo | chown: changing ownership of '/data/db': Operation not permitted
cello-mongo | chown: changing ownership of '/data/db': Operation not permitted
cello-mongo | chown: changing ownership of '/data/db': Operation not permitted
cello-mongo | chown: changing ownership of '/data/db': Operation not permitted
cello-mongo | chown: changing ownership of '/data/db': Operation not permitted
cello-mongo | chown: changing ownership of '/data/db': Operation not permitted
cello-mongo | chown: changing ownership of '/data/db': Operation not permitted
@jk.ananth could you please describe your environment ?
@Luke_Chen I am using Mac OS Mojave
Docker 18.03.0-ce-mac60
ok, I have not tried this yet.
Which environment do u use?
ubuntu 16.0.4
Has joined the channel.
Has joined the channel.
Has joined the channel.
Hi @Luke_Chen , i have configured cello in ubuntu machine (16.04 ) , (Docker version 18.06.0-ce, build 0ffa825) and (docker-compose version 1.13.0, build 1719ceb) . based on the document i am setting Master Node Setup and Setup worker docker. After that i could able to login operator dashboard UI. while adding the host for creating fabric cluster. i am getting error( Request Error 400: http://localhost:8080/api/host The request was issued with an error. The server did not create or modify data.) please guide me. what i have missed.
so looking into it.
Hi @Luke_Chen , i have configured cello in ubuntu machine (16.04 ) , (Docker version 18.06.0-ce, build 0ffa825) and (docker-compose version 1.13.0, build 1719ceb) . based on the document i am setting Master Node Setup and Setup worker docker. After that i could able to login operator dashboard UI. while adding the host for creating fabric cluster. i am getting error( Request Error 400: http://localhost:8080/api/host The request was issued with an error. The server did not create or modify data.) please guide me. what i have missed.
@jeevas could paste some logs here, especially from the cello-operator-dashboard
[ ](https://chat.hyperledger.org/channel/cello?msg=4L3sevnaywKt6ZrJE) @Luke_Chen cello-operator-dashboard | [2018-08-09 13:18:16,101] ERROR [agent.docker.docker_swarm] [docker_swarm.py:174 check_daemon()] - Exception in check_daemon Error while fetching server API version: HTTPConnectionPool(host='127.0.0.1', port=2375): Max retries exceeded with url: /version (Caused by NewConnectionError('
I add some remarks in the online document named `Enterprise BaaS based on Cello` @baohua
I add some remarks in the online document named `Enterprise BaaS based on Cello` @baohua
https://docs.google.com/document/d/1HfaWl8jXH75wua-hZCXgrpS-AQlYtTqSwlII1_1pi4s/edit#heading=h.x88ilguqci1l
Has cello supported fabric1.2 network? I created chain failed with fabric 1.2 network
level=error msg="Handler for GET /v1.25/networks/5d7973fb436940ffb6cd4991da26d37f_default returned error: network 5d7973fb436940ffb6cd4991da26d37f_default not found"
level=error msg="containerd: start container" error="oci runtime error: container_linux.go:247: starting container process caused \"process_linux.go:359: container init caused \\\"rootfs_linux.go:54: mounting \\\\\\\"/opt/cello/fabric-1.2/solo/configtx.yaml\\\\\\\" to rootfs \\\\\\\"/var/lib/docker/aufs/mnt/8b228edc51af8f79e4361a736cc5679852b4c9ad3980430e9897cf00025ba795\\\\\\\" at \\\\\\\"/var/lib/docker/aufs/mnt/8b228edc51af8f79e4361a736cc5679852b4c9ad3980430e9897cf00025ba795/etc/hyperledger/fabric/configtx.yaml\\\\\\\" caused \\\\\\\"not a directory\\\\\\\"\\\"\"\n" id=6e5170a078c94219dad2986d1257fc799965adb3ddfe030e155887c79310e92d
level=error msg="Create container failed with error: oci runtime error: container_linux.go:247: starting container process caused \"process_linux.go:359: container init caused \\\"rootfs_linux.go:54: mounting \\\\\\\"/opt/cello/fabric-1.2/solo/configtx.yaml\\\\\\\" to rootfs \\\\\\\"/var/lib/docker/aufs/mnt/8b228edc51af8f79e4361a736cc5679852b4c9ad3980430e9897cf00025ba795\\\\\\\" at \\\\\\\"/var/lib/docker/aufs/mnt/8b228edc51af8f79e4361a736cc5679852b4c9ad3980430e9897cf00025ba795/etc/hyperledger/fabric/configtx.yaml\\\\\\\" caused \\\\\\\"not a directory\\\\\\\"\\\"\"\n"
level=error msg="Handler for POST /v1.25/containers/6e5170a078c94219dad2986d1257fc799965adb3ddfe030e155887c79310e92d/start returned error: oci runtime error: container_linux.go:247: starting container process caused \"process_linux.go:359: container init caused \\\"rootfs_linux.go:54: mounting \\\\\\\"/opt/cello/fabric-1.2/solo/configtx.yaml\\\\\\\" to rootfs \\\\\\\"/var/lib/docker/aufs/mnt/8b228edc51af8f79e4361a736cc5679852b4c9ad3980430e9897cf00025ba795\\\\\\\" at \\\\\\\"/var/lib/docker/aufs/mnt/8b228edc51af8f79e4361a736cc5679852b4c9ad3980430e9897cf00025ba795/etc/hyperledger/fabric/configtx.yaml\\\\\\\" caused \\\\\\\"not a directory\\\\\\\"\\\"\"\n: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type"
this error is because of files that docker mounted not exist
this error is because of files that docker mounted not exist
@newthinker
this error is because of files that docker mounted not exist
@newthinker you can check the files , for example `configtx.yaml`, in the local path.
this error is because of files that docker mounted not exist
@newthinker you can check if the files , for example `configtx.yaml`, exist in the local path.
they are directories not files in the direcotry '/opt/cello/fabric-1.2/'
Clipboard - August 10, 2018 11:18 AM
so you can not setup up successfully
Because nfs is not installed
the operator dashboard is ok now but the user dashboard isn't work
[ ](https://chat.hyperledger.org/channel/cello?msg=TH8Mkh3Za8SBqJGmN) @Luke_Chen It worked when I manually changes the permission of folder /opt/cello/mongo
[ ](https://chat.hyperledger.org/channel/cello?msg=TH8Mkh3Za8SBqJGmN) @Luke_Chen It worked when I manually changed the permission of folder /opt/cello/mongo
@jk.ananth Thanks for the notification, I will try that later
[ ](https://chat.hyperledger.org/channel/cello?msg=2tMiKPWK4DPagQtEg) @newthinker yes, it is neccesary that installing `nfs` in worker-nodes
@luckydogchina operator dashboard is ok now but user dashboard also has problem and can't create chain
and also can't see the chain created in the operator dashboard
[ ](https://chat.hyperledger.org/channel/cello?msg=mXphQseYdm9yikt2R) @newthinker you can check if `/opt/cello/mongo` and `/opt/cello/bass` is exist.
[ ](https://chat.hyperledger.org/channel/cello?msg=mXphQseYdm9yikt2R) @newthinker you can check if `/opt/cello/mongo` and `/opt/cello/bass` is exist.
or cd $cello && make logs
The two directories are existed in master node not worker node
@luckydogchina
[ ](https://chat.hyperledger.org/channel/cello?msg=xSSNJ6CEy5uQ7oMFd) @newthinker yes, if they are existed , I suggest you look the logs.
ok, tks @luckydogchina
Clipboard - August 10, 2018 5:53 PM
@luckydogchina i have created Host and given capacity 3 so chains created 3 as a Fabric 1.0 . i would like to create new chain as a Fabric 1.2 .but i could n't able to create bcoz Host name filed is empty. please guide me .
Create Chain - Host is not loading in option field
Hi, I'm using cello deployment using Docker Host. I want to know whether Cello supports K8s. If yes, in what version it is supported.
Has joined the channel.
Has joined the channel.
looking at the code i can say it does support K8s. It creates the K8s cluster for you it seems
Guys, looking at the ansible agent deploy on AWS.. i see that by default it generates all the nodes in the public subnet. Is that configurable in an easy way? i will take a look at the ansible code now.. but i am new to ansible so it might take me some time.. hehe
Error creating host
Has joined the channel.
hii people! I'm testing Cello locally, but my master node can't reach the worker node..... I have followed this guide to configure my worker node using docker
when I execute from my master node `docker -H 192.168.50.22:2375 version`
the result is ```Client:
Version: 18.06.0-ce
API version: 1.38
Go version: go1.10.3
Git commit: 0ffa825
Built: Wed Jul 18 19:11:02 2018
OS/Arch: linux/amd64
Experimental: false
Cannot connect to the Docker daemon at tcp://192.168.50.22:2375. Is the docker daemon running?```
any ideas?? port 2375 is open in worker node
@jk.ananth seems same error, do you solved it? thanks for creating cello!!
I solved the problem modifing the docker systemd service config, this works fine: ```[Service]
DOCKER_OPTS="-H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --api-cors-header='*' --default-ulimit=nofile=8192:16384 --default-ulimit=nproc=8192:16384"
EnvironmentFile=-/etc/default/docker
ExecStart=/usr/bin/dockerd $DOCKER_OPTS```
Maybe the guide from the cello docs 'Setup Docker Host as a Worker Node', should be updated
@Javi thanks, would u like to help fix the doc?
Has joined the channel.
Has joined the channel.
@here Dear all, the Cello project meeting at 12:30 (UTC)/20:30 (UTC+8) this Friday is canceld. FYI. thanks!
when i create chain in user dashboard, the chain shown failed and the socket.io connect failed. Does anyone also has this problem?
Screen Shot 2018-08-16 at 10.25.34 AM.png
Has joined the channel.
I having trouble to get a working bc on k8s. Have build operator dashboard image from master (got tag x86_64-0.9.0-snapshot-77a79c9). The first problem i run into is i have to do this patch:`diff --git a/src/agent/k8s/cluster.py b/src/agent/k8s/cluster.py
index eb14ec1..a093e59 100644
--- a/src/agent/k8s/cluster.py
+++ b/src/agent/k8s/cluster.py
@@ -98,7 +98,7 @@ class ClusterOnKubernetes(ClusterBase):
cluster = ClusterModel.objects.get(id=cid)
cluster_name = cluster.name
- kube_config = KubernetesOperation()._get_from_params(cluster
+ kube_config = KubernetesOperation()._get_config_from_params(cluster
.host
.k8s_param)
`
```
```
the second problem is that fabric-explorer does not work, log is full of `[2018-08-16 08:34:48.581] [ERROR] Query - Error: Failed to deserialize creator identity, err MSP Org2MSP is unknown
at /blockchain-explorer/node_modules/grpc/src/client.js:554:15
[2018-08-16 08:34:48.582] [ERROR] blockscanner - TypeError: Cannot read property 'low' of undefined
at getChainInfo.then.response (/blockchain-explorer/app/query.js:208:32)
error: [client-utils.js]: sendPeersProposal - Promise is rejected: Error: Failed to deserialize creator identity, err MSP Org2MSP is unknown
at /blockchain-explorer/node_modules/grpc/src/client.js:554:15
error: [Channel.js]: Failed Query channel info. Error: Error: Failed to deserialize creator identity, err MSP Org2MSP is unknown
at /blockchain-explorer/node_modules/grpc/src/client.js:554:15
`
[ ](https://chat.hyperledger.org/channel/cello?msg=rxAtf5YygxpanFcYm) @janmpo i am also facingf same issues.did you fix this issues? if yes please let me know how to solve
@Luke_Chen i have created chain and network is a Fabric 1.2/Fabric 1.1 , also started the chain. while checking "docker ps" peer is running temporally after some time automatically stopping peer. checked "docker logs 7ca8df36f7f9 " getting below error.
2018-08-16 07:06:06.630 UTC [main] main -> ERRO 021 Cannot run peer because error when setting up MSP of type bccsp from directory /etc/hyperledger/fabric/msp: could not load a valid signer certificate from directory /etc/hyperledger/fabric/msp/signcerts: stat /etc/hyperledger/fabric/msp/signcerts: no such file or directory .. Could you please help me.
[ ](https://chat.hyperledger.org/channel/cello?msg=Xv2spuaAEMSQkiRPj) @Javi Any idea how to configure this for macOS?
@NicklasSandgren your first trouble could be solved by the patch https://gerrit.hyperledger.org/r/#/c/25391/
@jeevas could you please check worker-setup file, I think you are missing some files in your worker node.
[ ](https://chat.hyperledger.org/channel/cello?msg=kdRbi95MToqHrKbHR) @jk.ananth no idea. I'm testing with ubuntu 16.04
[ ](https://chat.hyperledger.org/channel/cello?msg=Fx5SeOkUiJZN6fa3tS) @baohua yes, how to proceed?
Has joined the channel.
Has left the channel.
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=ytHSivQ9LYz24qggj) @Luke_Chen i have followed below steps in cello for integrate to hyperledger explorer latest version.
Step 1 : cloned cello package
git clone http://gerrit.hyperledger.org/r/cello && cd cello
step 2 : setup the master node with the setup.sh.
cd cello/scripts/master_node/
./setup.sh
step 3 : make setup-master
step 4 : make start
opened another terminal (i have configured setup docker using my machine itself)
step 1 : sudo systemctl stop docker.service
step 2 : sudo dockerd -H tcp://192.168.1.103:2375 -H unix:///var/run/docker.sock --api-cors-header='*' --default-ulimit=nofile=8192:16384 --default-ulimit=nproc=8192:16384 -D &
step 3: make setup-worker
After that i have created host and chain also started the chain too.
while checking "docker ps" in command prompt my peers are running sometimes only automatically it's stopping the chains and peers also not running.
when i check "docker logs CONTAINER_ID" getting below error.
Cannot run peer because error when setting up MSP of type bccsp from directory /etc/hyperledger/fabric/msp: could not load a valid signer certificate from directory /etc/hyperledger/fabric/msp/signcerts: stat /etc/hyperledger/fabric/msp/signcerts: no such file or directory
Please let me know did i miss any steps? Please help me
verify whether you have /opt/cello and that all the configuration files are there. The setup-worker script is supposed to copy those files there. look for any errors in the log
Has joined the channel.
i have some problem when i install cello step by step , look like this problem , anyone have same problem ?
https://stackoverflow.com/questions/49384173/error-during-creation-of-chains-clusters-using-hyperledger-cello-with-docker-a
don't forget to run `make setup-worker` on the server where you are deploying fabric
so , the master node also execute make setup-worker?
i just have two nodes
no. only the nodes that would run fabric containers need it
ok , i'll try it , thanks
excuse me , when i run the `make setup-worker` cmd , it present local artifacts path /opt/cello not existed , create one & no master node addr is provided will ignore nfs setup
is there something preinstall when i execute `make setup-worker`?
and the `make setup-worker` also output setup_worker_node_docker.sh : line 27 : MASTER_NODE: command not found , is there somebody have the same problem?
thanks , i fixed it . brun `make setup-worker` command before should be `edit setup_worker_node_docker.sh , setting master node address . `
thanks , i fixed it . when run `make setup-worker` command before should be `edit setup_worker_node_docker.sh , setting master node address . `
Has joined the channel.
Hi, guys. I'm a new in Cello. I heard that cello is related to kubernetes, so that's why I am in here.
I want to make my hyperledger project more comfortable such as deployment, changing port or install chaincode.
I already made my network, but it is hard to fix these things I mentioned.
Does anyone try to run hyperledger project using cello? Furthermore, How u config your peers and org?
@here It was reported that some TSC election ballots haven't arrived on time.
If your name is in any sheet of https://docs.google.com/spreadsheets/d/1-R43fMOia9P1JsFiSWe3wxkOmkzK_EM7HT2kmYNEn0M/edit#gid=31088272, you should have received the ballot.
Otherwise, please ask Todd (tbenzies@linuxfoundation.org) to retrieve.
Thanks!
@lotty02cho the tutorial should be a good start: https://cello.readthedocs.io/en/latest/tutorial/. Feel free to share your experience and help contribute!
https://gerrit.hyperledger.org/r/25743
Has left the channel.
There seems to be some org1/org2 mixup in https://github.com/hyperledger/cello/blob/master/src/agent/k8s/cluster_resources/resources/explorer-artifacts/config.json . Example snippet ` "org1": {
"admin": {
"cert": "/first-network/crypto-config/peerOrganizations/org2/users/Admin@org2/msp/signcerts",
"key": "/first-network/crypto-config/peerOrganizations/org2/users/Admin@org2/msp/keystore"
},
"ca": "http://ca-org2:7054",`
```
```
Has joined the channel.
Has joined the channel.
@NicklasSandgren Thanks for the report
Has joined the channel.
@NicklasSandgren would you like to create a patchset to help fix? thanks!
Hi - I have a fabric network running in k8s (on AWS), setup using the ansible playbooks. I've got peers, orderers, CA instances etc... But what I do not see is the operator dashboard. I've been looking around the docs for a while, and it seems like you do either ansible/k8s/aws, or run the dashboards and then create a network. Is the Operator Dashboard mutually exclusive with the approach of setting up a network on k8s ? Or is there a way to deploy it onto a k8s cluster?
@baohua Sorry, by company policy I am not allowed to contribute code to open source projects (and the bureaucracy and time overhead to get permission to contribute is just way too big it for such a small patch)
pity! which company are u in?
@baohua Ericsson
hi, i am facing following error while creating chain via operator dashboard
cello-operator-dashboard | [2018-08-22 10:26:36,653] ERROR [resources.cluster_api] [cluster_api.py:329 cluster_list()] - [{'host_id': '4fc6d9ec4a2b44b0ae0d8143b8803af7', 'service_url': {}, 'health': '', 'api_url': '', 'duration': 0.0, 'service_ports': [], 'mapped_ports': {'peer1_org2_event': 7650, 'peer1_org1_grpc': 7350, 'peer0_org1_grpc': 7150, 'ca_org1_ecap': 7850, 'peer1_org1_event': 7250, 'peer0_org2_event': 7450, 'orderer': 8050, 'peer0_org2_grpc': 7550, 'explorer': 8150, 'peer1_org2_grpc': 7750, 'peer0_org1_event': 7050, 'ca_org2_ecap': 7950}, 'host': 'master_worker', 'create_ts': '2018-08-22T10:01:52.631000+00:00', 'id': '2370204f8e664164ac7b8ab568797b54', 'containers': [], 'status': 'creating', 'consensus_plugin': 'solo', 'apply_ts': None, 'release_ts': None, 'user_id': '', 'name': 'master_worker-1', 'network_type': 'fabric-1.2', 'size': 4, 'worker_api': 'tcp://ip_address:2375'}, {'host_id': '4fc6d9ec4a2b44b0ae0d8143b8803af7', 'service_url': {}, 'health': '', 'api_url': '', 'duration': 0.0, 'service_ports': [], 'mapped_ports': {'peer1_org2_event': 7650, 'peer1_org1_grpc': 7350, 'peer0_org1_grpc': 7150, 'ca_org1_ecap': 7850, 'peer1_org1_event': 7250, 'peer0_org2_event': 7450, 'peer0_org1_event': 7050, 'explorer': 8150, 'peer0_org2_grpc': 7550, 'peer1_org2_grpc': 7750, 'orderer': 8050, 'ca_org2_ecap': 7950}, 'host': 'master_worker', 'create_ts': '2018-08-22T10:26:35.430000+00:00', 'id': '95b6c9e88e5a4e15ae263c8471982d01', 'containers': [], 'status': 'creating', 'consensus_plugin': 'solo', 'apply_ts': None, 'release_ts': None, 'user_id': '', 'name': 'master_worker2', 'network_type': 'fabric-1.2', 'size': 4, 'worker_api': 'tcp://ip_address:2375'}]
need help..!!
Has joined the channel.
Hi, I just deployed cello , On the dashboard , I need to change the language to english
Is there an option to change the language ?
[ ](https://chat.hyperledger.org/channel/cello?msg=WPyjNqnk5JyyadJqZ) @chawlanikhil24 Its there ..on right side 'En' ..do that
@knagware9 thanks. One more thing.
There are 2 dashboards I can see here on docker containers
can you please tell the difference b/w cello-operator-dashboard and cello-user-dashboard
One for network operation like adding host and another one for application side purpose
@knagware9 Ok. Now, when I try to login the operator dashboard, I saw, error 404 on container, i.e. Template not found
user id pasword is admin pass
just today i tried ,,its working with this id pass
yes, The same credentials I used
I was able to login correctly but it couldn't find the landing page after login
@knagware9 https://pastebin.com/Q3F7fFyt
[ ](https://chat.hyperledger.org/channel/cello?msg=eSpHRHFRvP2mbrAig) @chawlanikhil24 I think some python script error,,,can you try to update the python,,.just recently used these dashboards
Since they are running on containers, that should be uniform thorough all the platforms
https://gerrit.hyperledger.org/r/#/c/25771/
@baohua @hightall @Luke_Chen please take a look at the patch. appreciate your +2. Thanks. https://gerrit.hyperledger.org/r/#/c/25771/
this patch set is to clean up some of the unused files.
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/25815/ fixed a typo in one line of the commands in the docs.
just one line change :D
@tongli +2 and merged, thanks!
Congrats to @HongweiPeng for his first merged patchset in Cello: https://gerrit.hyperledger.org/r/#/c/25815/! Thanks and look forward to more contributions!
@baohua @HongweiPeng Echo what baohua said, congrats
Thanks
Request Error 400: http://localhost:8081/api/chain
The request was issued with an error. The server did not create or modify data.
Observed this error while creating a new test_chain on cello dashboard
@baohua https://jira.hyperledger.org/secure/RapidBoard.jspa?rapidView=103&projectKey=CE&view=planning&selectedIssue=CE-433
@hightall please help have a look
https://jira.hyperledger.org/browse/CE-433
@baohua @hightall @Luke_Chen fixed a very small for the ansible agent. Please take a look here https://gerrit.hyperledger.org/r/#/c/25822/ thanks.
@baohua @hightall @Luke_Chen fixed a very small issue for the ansible agent. Please take a look here https://gerrit.hyperledger.org/r/#/c/25822/ thanks.
chaincode.png
can someone please suggest what should be function name and arguments, i am using default chaincode_example02 as smart contract
i've used function name as 'Init' and arguments as a,100,b,100 am i doing something wrong
cello-user-dashboard | error: [Peer.js]: sendProposal - timed out after:45000
cello-user-dashboard | error: [client-utils.js]: sendPeersProposal - Promise is rejected: Error: REQUEST_TIMEOUT
cello-user-dashboard | at Timeout._onTimeout (/node_modules/fabric-client/lib/Peer.js:117:19)
cello-user-dashboard | at ontimeout (timers.js:475:11)
cello-user-dashboard | at tryOnTimeout (timers.js:310:5)
cello-user-dashboard | at Timer.listOnTimeout (timers.js:270:5)
cello-user-dashboard | error: [Peer.js]: sendProposal - timed out after:45000
cello-user-dashboard | error: [client-utils.js]: sendPeersProposal - Promise is rejected: Error: REQUEST_TIMEOUT
cello-user-dashboard | at Timeout._onTimeout (/node_modules/fabric-client/lib/Peer.js:117:19)
cello-user-dashboard | at ontimeout (timers.js:475:11)
cello-user-dashboard | at tryOnTimeout (timers.js:310:5)
cello-user-dashboard | at Timer.listOnTimeout (timers.js:270:5)
error.mp4
@here Dear all, we will have project meeting at 8:30PM--9:00PM (UTC+8) this Friday evening , welcome to join, and please help update the agenda at: https://wiki.hyperledger.org/projects/cello/meetings/20180824. Thanks!
https://gerrit.hyperledger.org/r/25853
https://gerrit.hyperledger.org/r/#/c/25854/
a tiny patch to fix some mismatched contents please take a look at your convenience.
CelloLog.txt
Hi All please check the above attached log file. I follwed steps from https://cello.readthedocs.io/en/latest/setup_master/. But hyperledger/cello-user-dashboard is always Exited not able to access user dashboard. please let me know what needs to be done for running cello user and operator dashboards.
i tried with v0.8.0 and v0.8.0-beta branches both are giving same issue
https://gerrit.hyperledger.org/r/#/c/25856/
https://gerrit.hyperledger.org/r/#/c/25854/
@VenkataY please try the latest master branch code, which aligns with the online doc.
Thanks Luke_Chen and baohua its installed now.
Has joined the channel.
@baohua @tongli @hightall please take a look https://gerrit.hyperledger.org/r/#/c/25856/ at your convenience, thanks
Has joined the channel.
+2! @Luke_Chen
Thanks
@baohua Hi baohua, please take a look at your convenience https://wiki.hyperledger.org/groups/tsc/project-updates/cello-2018-aug?current_plans
welcome for comments :slight_smile:
https://gerrit.hyperledger.org/r/#/c/25897/
@Luke_Chen LGTM, thanks!
In cello i see 3 user types admin,operator and user. I see same UI and functionalities for all the 3 types of users in user dashboard. can any one tell me difference between admin,operator and user
Hi I was trying to run cello worker node by following https://github.com/hyperledger/cello/blob/master/docs/setup_worker_ansible_allinone.md on azure virtual machine or virtualBox on my local machine. But I am facing the following error: PLAY [Get start timestamp] **********************************************************************************************************************************************************
TASK [Gathering Facts] **************************************************************************************************************************************************************
ok: [cloud]
TASK [set_fact] *********************************************************************************************************************************************************************
ok: [cloud]
PLAY [fast initialize all the servers] **********************************************************************************************************************************************
TASK [Gathering Facts] **************************************************************************************************************************************************************
fatal: [10.0.0.4]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Permission denied (publickey).\r\n", "unreachable": true}
NO MORE HOSTS LEFT ******************************************************************************************************************************************************************
to retry, use: --limit @/home/anshulmittal/cello/src/agent/ansible/initcluster.retry
PLAY RECAP **************************************************************************************************************************************************************************
10.0.0.4 : ok=0 changed=0 unreachable=1 failed=0
cloud : ok=2 changed=0 unreachable=0 failed=0
Any help would be much appreciated!
@hightall Hi haitao, I can install chaincode to Fabric cluster hosted by Kubernetes agent right now, however this error came when try to instantiate chaincode
```
2018-08-28 02:44:06.944 UTC [chaincode-platform] func1 -> ERRO 58b Failed to generate platform-specific docker build: Error returned from build: 1 "can't load package: package github.com/5b84b696eb41c2004c1f1a2e: cannot find package "github.com/5b84b696eb41c2004c1f1a2e" in any of:
/opt/go/src/github.com/5b84b696eb41c2004c1f1a2e (from $GOROOT)
/chaincode/input/src/github.com/5b84b696eb41c2004c1f1a2e (from $GOPATH)
/opt/gopath/src/github.com/5b84b696eb41c2004c1f1a2e
"
2018-08-28 02:44:06.944 UTC [dockercontroller] deployImage -> ERRO 58c Error building images: Failed to generate platform-specific docker build: Error returned from build: 1 "can't load package: package github.com/5b84b696eb41c2004c1f1a2e: cannot find package "github.com/5b84b696eb41c2004c1f1a2e" in any of:
/opt/go/src/github.com/5b84b696eb41c2004c1f1a2e (from $GOROOT)
/chaincode/input/src/github.com/5b84b696eb41c2004c1f1a2e (from $GOPATH)
/opt/gopath/src/github.com/5b84b696eb41c2004c1f1a2e
"
2018-08-28 02:44:06.944 UTC [dockercontroller] deployImage -> ERRO 58d Image Output:
```
do you have any idea ? by the way, the chaincode I installed is uploaded from the user dashboard.
When I deploy and execute chaincode in CLI pods, the transactions history can be correctly displayed in user-dashboard
What? It seems the the user dashboard is working as intend right now, I deploy the existed chaincode_example02 instead in this time.
Hi ,,for cello setup ,,do we need to setup master node and worker node on the different ubuntu machine ? I tried but its failing with docker deamon issue not connecting
Has joined the channel.
https://gerrit.hyperledger.org/r/25932
https://gerrit.hyperledger.org/r/#/c/25931/
@hightall Hi haitao, please take a look this issue https://jira.hyperledger.org/browse/CE-440?jql=project%20%3D%20CE%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC
Has joined the channel.
@Luke_Chen Hi experts, I am new to cello can you please guide me where to drive in
@Ibraxos ok, you should start from the Cello docs
https://github.com/hyperledger/cello/blob/master/docs/tutorial.md
this tutorial should be a good start
[ ](https://chat.hyperledger.org/channel/cello?msg=6ks9GTEYD7HAZiCnk) @Luke_Chen I also need help..i am able to setup master node but not able to connect worker node to master node.Could you please help to connect both
Die you expose the docker daemon of the worker node correctly?
@knagware9
[ ](https://chat.hyperledger.org/channel/cello?msg=jBEERs8JoqWXn59mt) @Luke_Chen I did ,can we setup master and worked node on one ubuntu machine for testing purpose or we need seperate mahcines?
@knagware9 one ubuntu will be enough for testing
you don't have to separate them
[ ](https://chat.hyperledger.org/channel/cello?msg=SEc7XDH7jMCBTXsJB) @Luke_Chen OK doing same but not able to connect.my operator dashboard gave error
while try to add host
what error?
any log can be tracked?
https://gerrit.hyperledger.org/r/#/c/25940/ support fabric 1.2 in user dashboard
+1!
[ ](https://chat.hyperledger.org/channel/cello?msg=DDot9XdKatq6zd9uD) @Luke_Chen yes..please see error attached
Screenshot from 2018-08-29 14-39-14.png
Screenshot from 2018-08-29 14-42-21.png
Screenshot from 2018-08-29 14-42-40.png
Did you see that? the work is inactive
Did you see that? the worker is inactive
you can simply examine that with "curl localhost:2375"
you can simply examine that with `curl localhost:2375`
Yes.. It's shows ("message",page not found ")
@Luke_Chen uptil now through your guidence i have setup master node Now i want to setup kubernetes on worker node please guide me
@tongli @hightall can you tell me in the cell master branch, the `fabric-initial` target what to do?
@tongli @hightall can you tell me in the cell master branch, the `fabric-initial` target what to do? @Luke_Chen
[ ](https://chat.hyperledger.org/channel/cello?msg=3d1f53eb-0971-48ab-9b51-5e4d9f5d602a) Even now user dashboard is not working due to fabric client 1.0 module not found
@baohua each host share file by the nfs?
Tried Kubernetes as host but no success ,,may be doc are updated properly ..some one please help
Has joined the channel.
Hi, We developed a Helm/Charts for deploying Hyperledger Fabric, please check this link for more details http://www.think-foundry.com/hyperledger-fabric-deployment-using-helm-chart/, beside we also consider to add this tool to Cello, take the fully advantage and flexibility of the Helm
@Ibraxos you can setup a minikube as a worker node
a quite guide can be found here https://kubernetes.io/docs/tasks/tools/install-minikube/
or you can use this script instead https://github.com/hyperledger/cello/blob/master/scripts/worker_node/setup_worker_node_k8s.sh
@asaningmaxchain123 fabfic-initial is just a test image, it's related to design of crypto-config files generated dynamic, it's just generate crypto config files for each fabric network at the initial step.
@hightall but in the master branch it doesn't exist the target
and how the `user-dashboard` konw the chains?
[ ](https://chat.hyperledger.org/channel/cello?msg=7YD8YEx6aBLzf4Zap) @asaningmaxchain123 Correct!
@baohua when i start the dashborad in ide, but i can't visit it, i don't know the python,can you help me?
it must use docker to run it?
[ ](https://chat.hyperledger.org/channel/cello?msg=43xPAqMXbYQsDSbmN) @asaningmaxchain123 user dashboard call operator dashboard api to apply a chain, then will get all node ip & port for a chain.
[ ](https://chat.hyperledger.org/channel/cello?msg=Kesga3K2MYcmxadoJ) @asaningmaxchain123 yes, you need to follow the tutorial: https://cello.readthedocs.io/en/latest/tutorial/
@baohua thx,can you give me a more detail the fields or `Schedulable`, `autofill ` which define in the `Host`
Clipboard - August 30, 2018 12:51 PM
i have a take a look
i have a take a look ,but i can't understand
oh, can't understand which part?
aha, for autofill that means you set a number of chains in a host, then cello will smartly keep that number of chains existed in the host. i.e., if some chain is removed, cello will help create a new one.
what is default username & pass for adding kubernetes worker node
[ ](https://chat.hyperledger.org/channel/cello?msg=aLtpYPNPFWsJSv3L9) @baohua what's about Schedulable
@asaningmaxchain123 can you please guide me what is default username & pass while adding kubernetes as worker node
@Ibraxos i am newer
you can ask @baohua
haha lol
@baohua guide us senpai
Has joined the channel.
hello, I tried the bare bones hyperledger but realised installing the network and instantiating the chaincode is a huge climb. Then chanced upon cello. Could someone guide me to a book/pdf or step by step documention on using cello to bring up a fabric network with chaincode on it?
thanks
@GowriR please refer to https://github.com/hyperledger/cello/blob/master/docs/tutorial.md
thanks @Ibraxos - how is this used? cello for deployment and composer for writing chaincode? and does explorer also come in the pyramid?
@GowriR first of all you have to setup a cello on master node then moving forward you will setup worker node with mutiple options like dockerswarm , kubernetes then your will able to deploy chain code on this network
@GowriR as for as explorer is concerned yes it is possible but complexity depends upon your type of cluster
thanks @Ibraxos.
[ ](https://chat.hyperledger.org/channel/cello?msg=D4YvpeMW2WMWHA9qv) @asaningmaxchain123 Oh, that means you can create new chain on that host. this is for maintenance purpose if you mark some host un-schedulable.
@baohua can you use different host to build a chain?
for example?
@baohua during setup kubernetes as worker what is default username and pass
[ ](https://chat.hyperledger.org/channel/cello?msg=P8xui7EmhwYcXQNT7) @Ibraxos @Luke_Chen may help on this.
Clipboard - August 30, 2018 11:20 AM
Clipboard - August 30, 2018 11:20 AM
https://gerrit.hyperledger.org/r/#/c/25962/
[ ](https://chat.hyperledger.org/channel/cello?msg=uAeTRmY7fToivyhFM) @hightall +2!
Hi experts, I was using fabric in the net mode and dev mode. Then realised that it would be great to have the net mode sort of configuration (2 org, 2 peer, 1 orderer and 2 ca) with ONE chaincode and 2 CLI? I am assuming it is only one chaincode since i have only one chaincode and i would like to use the CLI instead of using a node js app to run my chaincode everytime. How do I modify my docker-compose to have 2-org, 2-peer to have chaincode container and 2 CLI containers? Is it possible? Is there an example that i can refer to? Can be done in cello?
@Ibraxos I prefer to use the cert/key credential, please don't use the username/password unless you know what it is in Kubernetes
@Ibraxos I prefer to use the cert/key credential, please don't use the username/password unless you knew what it is in Kubernetes
@Luke_Chen The mentioned cert/key are $HOME/.minikube/client.crt and $HOME/.minikube/client.crt?
Sorry for bothering you so much, I am new to this
yes
don't worry, you are very welcome to ask question
don't worry
I'm glad to provide any help.
Clipboard - August 30, 2018 12:44 PM
Thank you for helping @Luke_Chen
So when I add the cert and key content and click on submit, it takes me back to the 'Add host' screen.
can you check logs from operator-dashboard ?
Clipboard - August 30, 2018 12:47 PM
Clipboard - August 30, 2018 12:48 PM
You disabled the SSL, is it?
Yes
did you run `make build-admin-js` before?
I think I skipped that somehow, doing that right now.
Just to confirm, I have to run this on master node right?
Has joined the channel.
Has joined the channel.
Hey guys. I am trying to run both a master and a worker node on the same docker host. My master seems to be working fine, and I've created a worker node with no errors. When I try to add a host (127.0.0.1:2375) in the dashboard I get the following error: Request Error 400: http://localhost:8080/api/host The request was issued with an error. The server did not create or modify data.
How do I ensure the worker has started? I assumed it was added to docker compose and would start with "make start"
Seemes to work doing "ifconfig" and using the ip of the docker interface
`root@sc2-rdops-vm09-dhcp-57-57:~# ps -aux | grep 2375
root 459 0.0 0.0 11284 1092 pts/0 S+ 02:56 0:00 grep --color=auto 2375
root 1229 17.2 1.8 3616476 308228 ? Ssl Aug26 934:23 /usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:2375
`
```
root@sc2-rdops-vm09-dhcp-57-57:~# ps -aux | grep 2375
root 459 0.0 0.0 11284 1092 pts/0 S+ 02:56 0:00 grep --color=auto 2375
root 1229 17.2 1.8 3616476 308228 ? Ssl Aug26 934:23 /usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:2375
```
check this in your worker node
I made some progress :)
Hmm. `make start` creates the cello-user-dashboard but doesn't start it
@miceiken did you run `make setup-master`
Yup, both master and worker
@miceiken send me your logs
for operator-dashboard^
I am trying to do a clean install again now, I'll send logs if it doesn't work
Okay let me know if any progress :)
@Ibraxos where is it preferred to paste logs?
@Ibraxos while running `make` -> https://paste2.org/73kezjWe
@miceiken Go into cello directory and run `make log service=operator-dashboard`. Paste the screenshot here afterwards
This is before though - during setup-master, I must have overlooked this error before.
Maybe I shouldn't be doing this over master
Had to manually install "python3-dev" (Ubuntu 18.04 Desktop) and "backports.ssl_match_hostname" (through pip)
I think @tongli , @hightall or @Luke_Chen can guide you better with that
Are you running of the git master branch @Ibraxos ?
Yes. I have 2 VM's, 1 for master and other for worker. On my master node I ran `make setup-master` and then `make start` afterwards. I am following this tutorial: https://cello.readthedocs.io/en/latest/setup/
Okay, I have come a bit further. I am able to access both dashboards, but when I add a new chain it gets stuck on "Waiting" and when I run "docker ps", the new containers keep restarting every few seconds
yeah I am somewhat at your stage as well, waiting for Luke to reply
https://paste2.org/WfjwZsdL
This is the error I get
Well, no error, but this is what appears in `make logs`
Which type of worker are you trying to add?
How do you mean? It is a docker host set up using `make setup-worker`
I added the host in the operators dashboard
So I am guessing these are supposed to be files and not directories https://paste2.org/1yJE4nCt
Has joined the channel.
I'll try the v0.8.0 tag instead
```
ERROR: flake8: could not install deps [flake8, -r/home/miceiken/cello/src/requirements.txt]; v = InvocationError('/home/miceiken/cello/.tox/flake8/bin/pip install flake8 -r/home/miceiken/cello/src/requirements.txt (see /home/miceiken/cello/.tox/flake8/log/flake8-1.log)', 1)
Makefile:178: recipe for target 'check' failed
make: *** [check] Error 1
```
@miceiken I saw the log, it seems you are not able to install flake8 ?
I moved past that by installing the python3-dev package on my linux distro
I've since encountered numerous errors
https://gerrit.hyperledger.org/r/#/c/25984/
hi, I am new to cello, is it possible to setup the network with ansible and then connect the operator dashboard to it?
Has joined the channel.
:wave: Hello all. One of the maintainers of #burrow here. I've been asked to mention that we have helm charts in the centralized helm repo.
We'd like to reach out to tools such as y'all's fine tool to see what we can do on the burrow maintainer end to increase coverage by tooling. If anyone has any ideas for what we can do please let me know. My company is flat out but we do have very well tested charts and have simplified tremendously the chain initiation sequence for burrow chains to the point it can scalably be done in a very simple bash script.
In any event feel free to chat anytime he
In any event feel free to chat anytime. I'll lurk here, also feel free to visit us on burrow rooms.
[ ](https://chat.hyperledger.org/channel/cello?msg=2wNegCS8Aw7yAANjP) @mgalat Welcome, in plan and not ready yet. would you be interested to join the work?
[ ](https://chat.hyperledger.org/channel/cello?msg=HyYvq5Wh5swxwakyQ) @compleatang Welcome!
https://gerrit.hyperledger.org/r/#/c/26013/
@baohua do we have a call today?
Hi Would like to join call pls share the details
Any way to manually delete chains from cello dashboard
Has joined the channel.
Hi Guys I am new to this group
can any one explain how to setup hyperledger fabric on different machines and how they interact with each other?
https://gerrit.hyperledger.org/r/#/c/26034/
@baoha
@baohua @hightall
https://gerrit.hyperledger.org/r/#/c/26002/
[ ](https://chat.hyperledger.org/channel/cello?msg=cr6qFevwNteEiCwcf) @asaningmaxchain123 Thanks, please check and fix the ci error.
Has joined the channel.
@baohua can you please explain how to setting up fabric network on different physical systems
And they are interact with each other
@sureshtedla If you know kubernetes, this article can explain parts of your question.
https://docs.google.com/document/d/1Z52BC95Jfx8i_i9XB18PBVnHt17YU3PeyIq0nWclF8g/edit
@baohua can you tell me the `the jenkins execute target for cello` so i can do it for my local
When is the Cello v0.9.0 release?
https://gerrit.hyperledger.org/r/#/c/26041/
+2!
[ ](https://chat.hyperledger.org/channel/cello?msg=YEGKfwmXEMFCDCtYi) @asaningmaxchain123 basically, you can run `make check`, which is triggered by CI, too
[ ](https://chat.hyperledger.org/channel/cello?msg=t6THiWYEgxpr2gYr3) @VenkataY Plan to be next weekend!
https://gerrit.hyperledger.org/r/#/c/26047/
@baohua @hightall please take a look
https://gerrit.hyperledger.org/r/#/c/26048/
https://gerrit.hyperledger.org/r/#/c/26050/
Has joined the channel.
Has joined the channel.
Hey guys is it possible to deploy bna file created in composer on a network created in cello?
Has joined the channel.
@baohua how can deploy a chain in multiple host?
[ ](https://chat.hyperledger.org/channel/cello?msg=H7qPxczcfR5r58T5G) @asaningmaxchain123 Added comments.
[ ](https://chat.hyperledger.org/channel/cello?msg=7FGyQ5dREF6DtN7Ae) @asaningmaxchain123 Use kubernetes/swarm as the driver agent: http://cello.readthedocs.io/
so in the cello, when i want .to setup cluster it must be use the swarm or k8s
@baohua i will abandon it
@baohua i will abandon it,please ignore it
Clipboard - September 4, 2018 2:01 PM
while creating chain in cello with kubernetes as a worker pods giving me nfs mount error how can I handle this.
[ ](https://chat.hyperledger.org/channel/cello?msg=S2gxYyc3BpDZ2NXNL) @knagware9 I am getting the same were as well. Were u able to fix it?
@baohua are CE-448, CE-449, CE-450, and CE-451 mergable? what actions do I need to take, if any, to get them into master?
I think I am confused. Do you need to install hyperledger favbric then install cello, or does cello create the fabric?
Has joined the channel.
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=z4R9nqnsniKcJndSL) @ameijer CE-449 and CE-450 need one more +2, please ping @hightall @tongli or @Luke_Chen to help review, thanks!
[ ](https://chat.hyperledger.org/channel/cello?msg=pPbxgW4jmqvonGPqT) @StevenP 4 Latter one :)
@baohua currently,cello support k8s
https://gerrit.hyperledger.org/r/26094
@baohua I am new to Hyperledger Cello Can you please share cello related stuff
How to setup Hyperledger Fabric on different machines using the Hyperledger Cello? Can any one help me out
How to setup Hyperledger Fabric on different machines using the Hyperledger Cello? Can any one help me out
How to setup Hyperledger Fabric on different machines using the Hyperledger Cello? Can any one help me out @baohua @rocket.cat @asaningmaxchain123
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=kGrzRRZDWRjAnYoo7) @baohua @baohua , as per your direction I have obtained the requisite reviews on CE-449 and CE-450. Please let me know what else I can do to gelp. Thank you for all your help so far!
[ ](https://chat.hyperledger.org/channel/cello?msg=kGrzRRZDWRjAnYoo7) @baohua @baohua , as per your direction I have obtained the requisite reviews on CE-449 and CE-450. Please let me know what else I can do to help. Thank you for all your help so far!
[ ](https://chat.hyperledger.org/channel/cello?msg=kGrzRRZDWRjAnYoo7) @baohua , as per your direction I have obtained the requisite reviews on CE-449 and CE-450. Please let me know what else I can do to help. Thank you for all your help so far!
Is there a cello roadmap?
Is there a cello roadmap? What is the future of cello?
For going to production, is Docker Swarm or Kubernetes recommended for fabric deployments? with or without cello?
[ ](https://chat.hyperledger.org/channel/cello?msg=WK3D5JgYjRXyEicq5) @theathibm Both swarm and k8s are ready for production. Cello helps make it more automatic and more efficient. But certainly you can do it manually.
@baohua thanks for the response. Do you know the roadmap for cello?
yes, v0.9.0 is coming this week, while v1.0.0 is in the last quarter of the year
https://gerrit.hyperledger.org/r/#/c/26134/
Has joined the channel.
Has joined the channel.
Clipboard - September 7, 2018 1:32 PM
@all https://github.com/HaitaoYue/blogger If you have any experience or idea about cello, you can feel free to commit your PR into this repo, and share with all cello developers, thanks very much. https://blog.cello.ink/
@hightall can you add a README about the https://github.com/HaitaoYue/blogger and explain each folder
@asaningmaxchain123 yeah, ok, I will add a readme
Hi @hightall @baohua!
Can I create my own network using cello?
I mean, can I customize the network configuration in cello?
https://gerrit.hyperledger.org/r/#/c/26164/
[ ](https://chat.hyperledger.org/channel/cello?msg=sp9bJAwkwx3GWqxNr) @akshay.sood Not yet, but i do think it will be a useful feature. Would you like to help?
Hello, contributors are welcomed for HLExplorer , one of the skills required: nodejs, reactjs, docker
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/26186/
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=gJnGdL799d5vp4wwu) @hightall Would it be possible to have it in English?
@tongli would you like to help review https://gerrit.hyperledger.org/r/#/c/26182/?
Has joined the channel.
@here Cello v0.9.0-alpha is released, thanks every contributor's effort to make it out! https://github.com/hyperledger/cello/releases/tag/v0.9.0-alpha
Thanks Baohua
@baohua Since this a alpha release , https://github.com/hyperledger/cello/blob/master/Makefile#L40-L41 should be updated to trigger jenkins-jobs and push the docker-images
@baohua @Luke_Chen @hightall please take a look at this patch which is to generate yaml connection profile.
if you can take a quick look, I will really appreciate it.
https://gerrit.hyperledger.org/r/#/c/26209/
@baohua @tongli Hello, I have set up a fabriic netwrok network using Cello and have configured the block size to be 500txns per block and i made sure that the batch timeout and max bytes are sufficently high for 500txns per block, however when i look at logs, i see that only 85 txns are being written to a block. Any idea about why this is happening?
@baohua @tongli Hello, I have set up a fabriic netwrok network using Cello and have configured the block size on orderer to be 500txns per block and i made sure that the batch timeout and max bytes are sufficently high for 500txns per block, however when i look at logs, i see that only 85 txns are being written to a block. Any idea about why this is happening?
@Gvedam I really have no idea why that is.
I commit a fix in the master branch:
https://jira.hyperledger.org/browse/CE-477
https://gerrit.hyperledger.org/r/#/c/26212/
@luckydogchina good job, will review later
thanks
Clipboard - September 11, 2018 2:28 PM
Has joined the channel.
Has joined the channel.
Can any one provide the document and steps to install cello on windows10
Can any one provide document or steps to install cello on windows10
[ ](https://chat.hyperledger.org/channel/cello?msg=ZErKRgD275bXTq3q5) @BhaskarRaju We do not have support on windows10, and i suppose there will be some technical challenges due to windows itself. Would you like to use a virtualbox to run Linux?
[ ](https://chat.hyperledger.org/channel/cello?msg=376RwZotpiuGMgAbD) @luckydogchina Thanks, and merged now. Welcome for further contributions!
[ ](https://chat.hyperledger.org/channel/cello?msg=BEbsfv3sBod7DGM5E) @Gvedam blocksize only means the maximum number, if you wanna the block include more txs, maybe needs to send the request more quickly.
[ ](https://chat.hyperledger.org/channel/cello?msg=zHrWtkpiXunN95Wk2) @baohua Thank you @baohua , will use linux virtualbox to setup
sure, and i often take ubuntu 18.04 as my dev&test server, which is considering both efficiency and security.
Has joined the channel.
Hi all, I'm trying to setup Cello on my Mac and I successfully went through setting up the master and the worker nodes. I can access both operator and user UIs, and my Docker for Mac is listening correctly on port 2375 through `bobrik/socat`. I know because I can configure a docker host pointing to that and it works just fine
I'm having some hard time with the last step though, setting up my first chain :(
If I try to add a chain (I've tried different versions with no luck) my 4 peers and orderer nodes just keep restarting
The only meaningful error I could see through `docker logs` is this one:
`Cannot run peer because error when setting up MSP [...] /etc/hyperledger/peer/msp/signcerts: no such file or directory`
I've been fighting with this since yesterday but still no luck. Any help would be much appreciated :D
@iMacTia can you check the `/opt/cello` folder in your worker node? please make sure all your certificates have been copied to the right place.
Has joined the channel.
Has joined the channel.
Hi,
I dont see any runhosts file under /run directory after running below command
ansible-playbook -e "mode=apply env=mycloud cloud_type=os" provcluster.yml
I am following this tutorial:
https://cello.readthedocs.io/en/latest/worker_ansible_howto/#cloud-configuration-file-details
Any help would be appreciated :)
After bootstrapping my network with the provided ansible scripts. I can see a docker conatiner :email4tong/registrator running on all three nodes
What is it for?
any idea?
email4tong/registrator:v10 \"/bin/registrator -i…\" 16 minutes ago Up 16 minutes registrator.fabric003"
@Luke_Chen Not sure I get which docker container represents my "worker node". These are the containers I see after I start the master node:
```
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
37331afdcb61 hyperledger/cello-user-dashboard "/bin/sh -c 'ln -sf " 6 minutes ago Up 6 minutes 0.0.0.0:8081->8081/tcp cello-user-das
hboard
dd0381911294 mongo:3.4.10 "docker-entrypoint.s " 7 minutes ago Up 6 minutes 127.0.0.1:27017-27018->27017-27018/tcp cello-mongo
0d65fe56167c rabbitmq "docker-entrypoint.s " 7 minutes ago Up 6 minutes 4369/tcp, 5671-5672/tcp, 25672/tcp cello-dashboar
d_rabbitmq
a358c2b77cdf mongo:3.4.10 "docker-entrypoint.s " 7 minutes ago Up 6 minutes 27017/tcp cello-dashboar
d_mongo
4248f413323d hyperledger/cello-engine "python restserver.py" 7 minutes ago Up 6 minutes 0.0.0.0:80->80/tcp cello-engine
a8070a9efd98 hyperledger/cello-watchdog "python watchdog.py" 7 minutes ago Up 6 minutes cello-watchdog
6eaecffb43cf hyperledger/cello-operator-dashboard "/bin/sh -c '/etc/in " 7 minutes ago Up 6 minutes 0.0.0.0:8080->8080/tcp cello-operator
-dashboard
bc509e65e393 bobrik/socat "socat TCP-LISTEN:23 " 22 hours ago Up 22 hours 0.0.0.0:2375->2375/tcp sharp_bassi
1c832d9cd887 itsthenetwork/nfs-server-alpine:9 "/usr/bin/nfsd.sh" 41 hours ago Up 24 hours 0.0.0.0:2049->2049/tcp cello-nfs
```
And once I ADD a chain to the host, I can see some "peers", an "orderer" and other containers.
but nothing explicitly called "worker node". I guess I could check one of the peers or the orderer, but I can't enter them as they keep restarting :(
@Luke_Chen Actually, my Docker host is my worker node, so on my Mac that would be "Docker for Mac" ? in that case `/opt/cello` would be on my Mac FS?
In case my assumption is correct, here is the result of `ls -l /opt/cello` on my mac:
```
baas
dashboard_mongo
fabric-1.0
fabric-1.2
mongo
```
Not sure if certificates should be here. Also, at which point should the certificate be copied over?
@iMacTia certificates should be placed in the /opt/cello/fabric-1.0/crypto-config folder
walk through the sub directory to see if you missed the *.cert file
Thanks @Luke_Chen, here's the result of `tree /opt/cello/fabric-1.0/crypto-config`
```
/opt/cello/fabric-1.0/crypto-config
├── ordererOrganizations
│ └── example.com
│ └── orderers
│ └── orderer.example.com
│ ├── msp
│ └── tls
└── peerOrganizations
├── org1.example.com
│ ├── ca
│ │ └── ca.org1.example.com-cert.pem
│ └── peers
│ ├── peer0.org1.example.com
│ │ ├── msp
│ │ │ └── keystore
│ │ └── tls
│ └── peer1.org1.example.com
│ ├── msp
│ │ └── keystore
│ └── tls
└── org2.example.com
├── ca
│ └── ca.org2.example.com-cert.pem
└── peers
├── peer0.org2.example.com
│ ├── msp
│ │ └── keystore
│ └── tls
└── peer1.org2.example.com
├── msp
│ └── keystore
└── tls
```
I can only see `.pem` files and no `.cert` ones
So you're probably right, that might be my issue :(
However I still don't know WHEN these files should be copied over and by WHO/WHAT
I followed every step of the guide on the Cello repository and didn't notice any particular error while setting up master and worker nodes
:joy: sorry it should be .pem rather than cert
the files seem right,
peer0.org1.example.com
|----msp
what did you find under the msp?
I saw you only list one sub directory, which is `keystore`
that's exactly what I found :(
and the folder looks empty
That's where there should be a folder called `signcerts`, right? Hence the error I get
@Luke_Chen Something has probably gone wrong at some point but I couldn't see any error. Do you know which script is responsible for generating these certificates? If I knew that, I could investigate why that didn't happen
Is there any way we can use composer to deploy chaincodes for a chain on cello? @Luke_Chen @baohua @hightall
+1, that's the next thing I'd like to do :D Once I can get Cello up and running, that's it
@Luke_Chen Thanks for pointing me in the correct direction, after I manually copied the certificates from the repository into my `/opt/cello` folder, the peers are up an running now!
FYI - `cp -R src/agent/docker/_compose_files/fabric-1.2/crypto-config /opt/cello/fabric-1.2/crypto-config`
orderer is still not happy, but I guess it's something else and investigate it later on
thanks again for the help
[ ](https://chat.hyperledger.org/channel/cello?msg=k7pJ3sw4oJEt7E2c7) @Ibraxos Definitely we like the feature and welcome for any contribution to that direction!
Has joined the channel.
@iMacTia Did you have any log from the orderer ?
@Luke_Chen Similar issue, this time the `orderer.genesis.block` was missing
or actually, it was created as a folder for some reason
I deleted the folder and manually copied it over, and then the orderer was up and running as well :)!
There's definitely something odd when running the scripts on macOS, but thanks to your help I was able to get them working :D
For future reference:
`cp src/agent/docker/_compose_files/fabric-1.2/solo/channel-artifacts/orderer.genesis.block /opt/cello/fabric-1.2/solo/channel-artifacts/`
Has joined the channel.
https://wiki.hyperledger.org/projects/cello/meetings/20180914
Has joined the channel.
Hi, I've try to install the lastest version and the release v0.8.0, but I've the problem: "cello-mongo | chown: invalid user: ‘mongodb’". Please, someone can help me? Thanks
[ ](https://chat.hyperledger.org/channel/cello?msg=KBiFoNEfsAoNSs9n9) @bperetti would you like to follow our tutorial: https://cello.readthedocs.io/en/latest/tutorial/?
and please try the latest master branch first, which fixed some bugs recently.
Hi, I've installed cello today and try to create chains, however I don't get any chain to start up properly
the health will always turn to FAIL and the chain gets deleted after a while
the logs from the docker container will say 2018-09-14 13:10:39.658 UTC [orderer/main] initializeSecureServerConfig -> INFO 002 Starting orderer with TLS enabled
2018-09-14 13:10:39.658 UTC [orderer/main] initializeSecureServerConfig -> CRIT 003 Failed to load ServerCertificate file '/var/hyperledger/orderer/tls/server.crt' (open /var/hyperledger/orderer/tls/server.crt: no
[ ](https://chat.hyperledger.org/channel/cello?msg=GwnoiiXCm9WkJev4W) @JohnR can you help check if you have`/opt/cello` path with data on each node? if you follow the tutorial at https://cello.readthedocs.io/en/latest/tutorial/, the NFS will automatically do that sync.
there is a /opt/cello but it is a local filesystem
and it's exactly the tutorial I followed
Has joined the channel.
But you got me onto something
[ ](https://chat.hyperledger.org/channel/cello?msg=PNRmo2JNY3AKdPWGh) @baohua Hi, I've try this with the latest sometimes hapens, but in the release at all the time hapens. What is right way to install a specific release? thanks
I never installed the NFS client it seems
[ ](https://chat.hyperledger.org/channel/cello?msg=pqdsRYEKjLR933QJ4) @bperetti Then you can checkout the v0.9.0-alpha release code and try, that's latest release.
Has joined the channel.
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/26366/
+2!
Has joined the channel.
composer network ping -c admin@med-net
Error: Error trying to ping. SyntaxError: Unexpected end of JSON input
Command failed
Has joined the channel.
Has joined the channel.
Has joined the channel.
Hello!
Is it possible to use cello's ansible agent to bring up a fabric network on a VM in cloud from digitalocean?
How should the config file for the cloud look like?
@baohua what's the `balance` which define in the `profile` model
I've managed to install cello and deploy some chains, but there's a few strange things I run into
1) I can't seem to create chains with a different size then 4 (can't change the options)
2) I can't create other chains but fabric 1.0 with the solo consensus
I've followed the tutorial on https://cello.readthedocs.io/en/latest/tutorial/ but it seems I'm stuck at this point
@baohua in the next release,the mongo will be remove?
[ ](https://chat.hyperledger.org/channel/cello?msg=CTw9v5xhGqBNTut2q) @asaningmaxchain123 Why?
https://github.com/hyperledger/cello/blob/205700f607cf1b724f5e8fd2dc5844e751066b68/docker-compose-files/docker-compose.yml#L131
oh, we will use mongo for a while, currently no find better alternative
Has joined the channel.
Hi all, I am trying to set up a new host using cello/ec2. I have one ec2 machine as the master, and went to the :8080 url, but when I try to add host of another ec2 machine I get this error: "Request Error 400: http://ec2-
Has joined the channel.
Has left the channel.
also can someone explain to me the concepts of 'chains' in Cello, and how it relates to channels/ledgers/peers in Fabric?
Also in the cello admin dashboard you can create users with a 'Balance' field, but what if you have no concept of a balance in your network?
[ ](https://chat.hyperledger.org/channel/cello?msg=4T7KcXvAeFiEpvttY) @ColeBoudreau a chain is actually a blockchain network, e.g., a fabric network.
Clipboard - September 23, 2018 8:12 PM
Has joined the channel.
ı want to add host but get error called "Request Error 400: http://localhost:8080/api/host
The request was issued with an error. The server did not create or modify data." How can I fix it. Thx
Has joined the channel.
Hi I'm new to cello. Trying to setup a cello master node as per the: https://github.com/hyperledger/cello/blob/master/docs/setup_master.md
Hi I'm new to cello. Trying to setup a cello master node as per the: https://github.com/hyperledger/cello/blob/master/docs/setup_master.md getting below error when try to " DEV=true make start"
Hi I'm new to cello. Trying to setup a cello master node as per the: https://github.com/hyperledger/cello/blob/master/docs/setup_master.md getting below error when try to " DEV=true make start" how to fix this? the docker engine version i have is 18.06. "version"in docker-compose-nfs.yml is set to 3.2
Clipboard - September 24, 2018 6:31 AM
upgrade your docker-composer version and then it will works well.
[ ](https://chat.hyperledger.org/channel/cello?msg=3Ym9XeKXx2AigDMku) @Randyshu2018 I'm facing the same issue
I manged to setup cello master node and two worker nodes. Both the workers are visible trough admin (operator) dashboard. But chain creation gives some trouble.
I manged to setup cello master node and two worker nodes. Both the workers are visible trough admin (operator) dashboard. But chain creation gives some trouble.
1) selected Consensus - KAFKA : chain is not created "Request Error 400: http://178.128.80.210:8080/api/cluster/d86f2958fb4c47a487015d2445bdd1fe
The request was issued with an error. The server did not create or modify data."
2) Selected Consensus "Solo": Chain created but health is indicated as fail.
I manged to setup cello master node and two worker nodes. Both the workers are visible trough admin (operator) dashboard. But chain creation gives some trouble.
1) selected Consensus - KAFKA : chain is not created and gets the error pop on the dashboard saying "Request Error 400: http://178.128.80.210:8080/api/cluster/d86f2958fb4c47a487015d2445bdd1fe
The request was issued with an error. The server did not create or modify data."
2) Selected Consensus "Solo": Chain created but health is indicated as fail.
Has left the channel.
Clipboard - September 24, 2018 9:01 PM
@baohua Thanks! Is it possible to deploy networks in Cello using .bna files created in composer?
also there seem to be two versions of the docs:
1) https://hyperledger-cello.readthedocs.io/en/latest/ and
2) https://cello.readthedocs.io/en/latest/
both of which seem to differ slightly in some aspects. Is one more updated/supported than the other?
2) seems latest
https://gerrit.hyperledger.org/r/#/c/26542/
Can multiple worker nodes can be assigned to a chain in cello? If so how to do this?
I have few questions
1)Can multiple worker nodes can be assigned to a chain in cello? If so how to do this?
I have few questions
1)Can multiple worker nodes can be assigned to a chain in cello? If so how to do this?
2) Also can KAFKA be used as the consensus for a given chain
hi. I cant add any chain or host on dashboard. Is there any configuration settings what I need to do. Thx
[ ](https://chat.hyperledger.org/channel/cello?msg=yWwBfC4yYRa4Rqgio) @hightall +2!
[ ](https://chat.hyperledger.org/channel/cello?msg=NrdQQQkMjqgd4LpwB) @geethanisp 1) Using swarm or k8s type of host. 2) yes
@baohua
thanks for the response. I have setup a cello 1 master node and 1 worker node. Then I tried to create chain with KAFKA as the consensus protocol then got the error "Request Error 400: http://
@baohua
thanks for the response. I have setup a cello 1 master node and 1 worker node. Then I tried to create chain with KAFKA as the consensus protocol then got the error "Request Error 400: http://
do you have kafka cluster in operational dashboard already?
otherwise, need to create one from it.
@baohua
Thanks. Where do you suggest to have kafka cluster. 1) Master node 2) Worker Node. On the owrker node i used Fabric and expectation was Fabric comes with Kafka as the consensus protocol by default. Also what's the recommended OS for cello Master and Worker Nodes. I have used Ubuntu 18.04
@baohua
Thanks. Where do you suggest to have kafka cluster. 1) Master node 2) Worker Node. On the worker node i used Fabric and expectation was Fabric comes with Kafka as the consensus protocol by default. Also what's the recommended OS for cello Master and Worker Nodes. I have used Ubuntu 18.04
Hello, we are doing a webinar with the LI on deploying with Helm Charts:
https://medium.com/aid-tech/webinar-a-hitchhikers-guide-to-deploying-hyperledger-fabric-on-kubernetes-1bcecc2a4ade
In case anyone from the Cello team/userbase is interested in attending
I used cello and got my operator dashboard working at port 8080, but when I try 8081 for the user dashboard I get a refused to connect, ERR_CONNECTION_REFUSED. Is there something simple causing this? Note this is also hosted on AWS, so it could be my security group permissions or similar, but I can't seem to figure it out
Okay I figured the above out, it just wasn't configured properly on the master node. Although now a follow up: Under create new chain in the operator the chain size option defaults to '4' and can't be set of anything else. Why is this?
Is there any documentation on the format of the .zip file of the 'new smart contract' under the Create New Smart Contract section?
Hello all, is there any way to change the blocksize on orderer once the network is setup, without having to recreate the network again?
[ ](https://chat.hyperledger.org/channel/cello?msg=6cQ9auzBfus4ZriHW) @geethanisp Could you help follow https://cello.readthedocs.io/en/latest/tutorial/? i guess this can answer most of your questions.
[ ](https://chat.hyperledger.org/channel/cello?msg=7bnKWBemkAtd82BW2) @Gvedam which blocksize?
[ ](https://chat.hyperledger.org/channel/cello?msg=RT6tMe5i2dGShEgWY) @baohua The block size on the orderer. Can i change it to maybe 100 txns per block without bringing down the network?
[ ](https://chat.hyperledger.org/channel/cello?msg=T7MH5qaTP8QitjWHm) Currently, i have set the maximum number of txns on a block to 50 and the network has been up and i can see that each block has 50 txns. Can i change it without having to run Setupfabric again? Im using Cello with ansible.
[ ](https://chat.hyperledger.org/channel/cello?msg=h3qMcw4jaWZJ5cRrc) @Gvedam This depends on the transaction throughput and your orderering configuration. please have a look at https://hlf.readthedocs.io/en/latest/kafka.html
[ ](https://chat.hyperledger.org/channel/cello?msg=EKEn8JTqEL5iDhzfi) @baohua while setting up a fabric network in cello using ansible, and in configtx.yml we can configure the block size on orderer. can i do that without restarting the networrk
hi again
I m trying to setup master. Running "make setup-master" code OK. After run make start. But an error looks "Encountered errors while bringing up the project.
Makefile:208: recipe for target 'start' failed"
How can I fix it, whats the problem
Can some one help me with, Programming Languages used in Hyperledger Cello Development
Hi, I am using ansible to launch a network on aws. When I run "sudo ansible-playbook -e "mode=apply env=casewarecloud cloud_type=aws" provcluster.yml -vvv" I get:
TASK [Gathering Facts] ******************************************************************************************************
task path: /home/ubuntu/cello/src/agent/ansible/roles/cloud_aws/plays.yml:30
fatal: [18.222.68.190]: UNREACHABLE! => {
"changed": false,
"msg": "Failed to connect to the host via ssh: Warning: Permanently added '18.222.68.190' (ECDSA) to the list of known hosts.\r\nubuntu@18.222.68.190: Permission denied (publickey).\r\n",
"unreachable": true
}
<18.218.132.97> (255, '', "Warning: Permanently added '18.218.132.97' (ECDSA) to the list of known hosts.\r\nubuntu@18.218.132.97: Permission denied (publickey).\r\n")
fatal: [18.218.132.97]: UNREACHABLE! => {
"changed": false,
"msg": "Failed to connect to the host via ssh: Warning: Permanently added '18.218.132.97' (ECDSA) to the list of known hosts.\r\nubuntu@18.218.132.97: Permission denied (publickey).\r\n",
"unreachable": true
}
<18.224.19.15> (255, '', "Warning: Permanently added '18.224.19.15' (ECDSA) to the list of known hosts.\r\nubuntu@18.224.19.15: Permission denied (publickey).\r\n")
fatal: [18.224.19.15]: UNREACHABLE! => {
"changed": false,
"msg": "Failed to connect to the host via ssh: Warning: Permanently added '18.224.19.15' (ECDSA) to the list of known hosts.\r\nubuntu@18.224.19.15: Permission denied (publickey).\r\n",
"unreachable": true
}
Hi, I am using ansible to launch a network on aws. When I run "sudo ansible-playbook -e "mode=apply env=mycloud cloud_type=aws" provcluster.yml -vvv" I get:
TASK [Gathering Facts] ******************************************************************************************************
task path: /home/ubuntu/cello/src/agent/ansible/roles/cloud_aws/plays.yml:30
fatal: [XX.XX.XX.XX]: UNREACHABLE! => {
"changed": false,
"msg": "Failed to connect to the host via ssh: Warning: Permanently added 'XX.XX.XX.XX' (ECDSA) to the list of known hosts.\r\nubuntu@XX.XX.XX.XX: Permission denied (publickey).\r\n",
"unreachable": true
}
Any help?
[ ](https://chat.hyperledger.org/channel/cello?msg=wsHdnY5YAWCcW9c3m) @Gvedam i would suggest you follow the fabric guideline, it's not straightforward currently.
@here Dear all, we canceled the meetings in following two weeks, enjoy holidays! https://wiki.hyperledger.org/projects/cello/meetings.
Has left the channel.
@ColeBoudreau you tried directly to ssh your AWS instance ?
@pankajcheema yes, it works fine
@ColeBoudreau you ssh using passwd or key ?
using key, both master and host nodes are ec2 instances
@ColeBoudreau try with passwd
selected pem in ansible script also ? @ColeBoudreau
Has joined the channel.
Can I deploy customized network using Cello ?
Can I deploy customized network using Cello ? means use of couchDB / level DB for Peer, use of kafka based /solo ordering service & number of ORG participating in network ?
[ ](https://chat.hyperledger.org/channel/cello?msg=sN5AnMQ4cupHbhgdY) @ColeBoudreau Check Key permissions and Ensure Keys are generated and stored in Authrozied_keys files
Has joined the channel.
Hello. I've used the Ansible client to setup a cluster on AWS, using Kubernetes. However, someone has managed to slip a Monero mining script into Kubernetes DNS containers, which I noticed when the CPU usage went over 100%. Any thoughts?
Apparently, it's something called a Kubelet exploit: https://github.com/kayrus/kubelet-exploit
Also explained here: https://medium.com/handy-tech/analysis-of-a-kubernetes-hack-backdooring-through-kubelet-823be5c3d67c
Has joined the channel.
Has joined the channel.
Has joined the channel.
Has joined the channel.
Kby7 1:26 PM
hey, i installed Cello (the master node and some worker nodes) this weekend (on ubuntu via virtual box) If i browse to the dashboard for operator I miss some functions (like acitve clusters, inused clusters) , also when I follow https://cello.readthedocs.io/en/latest/tutorial/ I have the impression that this not represent the most current version. I have the impression I missed a part on the installation steps or I use the wrong ports (:8080 and :8081) to connect to the dashboards. Can some one help me out a bit to get this running well.
Has joined the channel.
Has joined the channel.
Has joined the channel.
Has joined the channel.
Has joined the channel.
Has joined the channel.
Has joined the channel.
Has joined the channel.
Can some one provide me help with the Installation of master and worker node of hyperledger cello v0.9.0-alpha
urgent please
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=SeGmApknLFvxHSFxv) @nishanthkp This might be helpful: https://cello.readthedocs.io/en/latest/tutorial/
Hi, I tried setting up cello and was able to successfully but then i modified my iptables and when now i am trying to setup again cello
its failing
Starting cello-mongo ... error
-A DOCKER -p tcp -d 0/0 --dport 80 -j DNAT --to-destination 172.29.0.4:80 ! -i br-2fbba59b9e35: iptables: No chain/target/match by that name.
(exit status 1))
ERROR: for cello-mongo Cannot start service mongo: driver failed programming external connectivity on endpoint cello-mongo (7630ba27b03083dfc2dd0193e5dbab878affdb502f38ee8ae649fb8f1a1744a5): (iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 127.0.0.1 --dport 27018 -j DNAT --to-destination 172.29.0.6:27018 ! -i br-2fbba59b9e35: iptables: No chain/target/match by that name.
Starting cello-operator-dashboard ... error
Starting cello-mongo ... error
-A DOCKER -p tcp -d 0/0 --dport 80 -j DNAT --to-destination 172.29.0.4:80 ! -i br-2fbba59b9e35: iptables: No chain/target/match by that name.
(exit status 1))
ERROR: for cello-mongo Cannot start service mongo: driver failed programming external connectivity on endpoint cello-mongo (7630ba27b03083dfc2dd0193e5dbab878affdb502f38ee8ae649fb8f1a1744a5): (iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 127.0.0.1 --dport 27018 -j DNAT --to-destination 172.29.0.6:27018 ! -i br-2fbba59b9e35: iptables: No chain/target/match by that name.
Starting cello-operator-dashboard ... error
could any one please provide me some leads
this is solved by re-installing docker
One query do we need cello project in all VMs which we require as to be slave nodes ?
Has joined the channel.
@hightall i can't start the user-dashborad in the folder `user-dashborad` by using npm run start?
Has joined the channel.
Hi, I started exploring cello. My primary goal is to be able to setup multi-node Fabric cluster and manage the cluster add node/ delete node/upgrade... . I cam across cello and Ansible agent in cello. I read the documentation and also previous chat, i still have few pending questions. Hope someone can help me out here.
1) Though Cello and Ansible agent are in the same repo they seem to completely different projects. Is my understanding true. When would you recommend using Cello vs Ansible agent to set up the network?
2) If i use cello i couldn't find a way to design my network ( Select number of orgs, number of peers .... and so on). Is that part of some configuration file ??.
3) Once i setup the network is there a way i could add new org to the network through cello ??.
Has left the channel.
@baohua can you please take a look at the question above.
Has joined the channel.
Has joined the channel.
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=yeycWydvHrrFcsi9Q) @venkat1 2 currently, 2) and 3) are not supported in cello, the Ansible is just to deploy fabric . The fabirc network is deployed by fixable template in cello.
Has joined the channel.
Has joined the channel.
I use the lastest images and meet a problem that hyperledger/cello-operator-dashboard can't access by localhost:8080.
And the cello-user-dashboard can't sign in by admin:pass because can't access localhost:8081/login("message":"Internal Server Error")
But,i can use that last friday.
How can i solve it.@baohua
@baohua
[ ](https://chat.hyperledger.org/channel/cello?msg=yeycWydvHrrFcsi9Q) @venkat1 2 you can use sdk like nodejs-sdk to custom your network.
[ ](https://chat.hyperledger.org/channel/cello?msg=pncCibZeTFEZQCZeM) @fnst_zhangxin the default user account should never change. what's the log when you login?
[ ](https://chat.hyperledger.org/channel/cello?msg=3BAEZ2vim2eB3Dr3h) @baohua I solve the problem when i use the docker17.06.0ce. Before i change the docker version, i used the docker17.03.0ce. That may be the crux of the problem.
Dear all,
Please help review https://docs.google.com/document/d/1Dw6cEKaul6FenORNkDcxvPDDKwpl0A5EmcJBlqAWJoU/ before the meeting.
We will discuss the new Blockchain Consortium Governing Model in the meeting.
Thanks a lot!
@here
[ ](https://chat.hyperledger.org/channel/cello?msg=TdWBFtiBnscKtqHxb) @fnst_zhangxin Interesting, do not hear that before, would you like to help add in our FAQ? https://wiki.hyperledger.org/projects/cello/faq
Has joined the channel.
Has joined the channel.
Is the meeting dropped ?
@baohua
fabric中PBFT的作用是什么?有大神可以帮忙简单的解释一下吗?
What is the role of PBFT in fabrics?
[ ](https://chat.hyperledger.org/channel/cello?msg=kAsCdMd7Y8wS4Rfcj) @shuailaotou PBFT is one of the fabric consensus protocols.
Hyperledger Cello – Need Help
Hi,
I am looking to deploy the Hyperledger Fabric using the ansible Cello Agent.
Referring to the below link
https://cello.readthedocs.io/en/latest/worker_ansible_howto/
I am executing the below command
ansible-playbook -e "mode=apply env=mycloud cloud_type=aws" provcluster.yml
The above command is provisioning the Servers in AWS, however once the servers are provisioned, I am getting the below Error. I do understand that it is due to certificate configuration and I understand the I need to provided private key ( in my case “fd” is private key file name ) to the ansible playbook. Please do help in modifying the Ansible playbook.
Error is below
TASK [Wait until servers are up and runnning] **************************************************************************************
ok: [cloud] => (item=35.175.216.229)
ok: [cloud] => (item=34.206.72.196)
ok: [cloud] => (item=34.205.81.81)
PLAY [change /etc/hosts files so that each node knows other nodes] *****************************************************************
TASK [Gathering Facts] *************************************************************************************************************
fatal: [35.175.216.229]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Warning: Permanently added '35.175.216.229' (ECDSA) to the list of known hosts.\r\nno such identity: /home/ansible/.ssh/fd: No such file or directory\r\nPermission denied (publickey).\r\n", "unreachable": true}
fatal: [34.205.81.81]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Warning: Permanently added '34.205.81.81' (ECDSA) to the list of known hosts.\r\nno such identity: /home/ansible/.ssh/fd: No such file or directory\r\nPermission denied (publickey).\r\n", "unreachable": true}
fatal: [34.206.72.196]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Warning: Permanently added '34.206.72.196' (ECDSA) to the list of known hosts.\r\nno such identity: /home/ansible/.ssh/fd: No such file or directory\r\nPermission denied (publickey).\r\n", "unreachable": true}
Ansible code responsible for the above Error is present in the plays.yml file
File Pathi - /home/ubuntu/cello/src/agent/ansible/roles/cloud_aws/plays.yml
**CODE**
- name: change /etc/hosts files so that each node knows other nodes
hosts: allnodes
strategy: free
user: "{{ cluster.ssh_user }}"
become: true
become_user: ubuntu
vars_files:
- "{{ playbook_dir }}/../../vars/{{ env }}.yml"
tasks:
- include_tasks: "../cloud_os/namenode/tasks/apply.yml"
tags: "namenode"
with_items: "{{ groups['allnodes'] }}"
the above code is calling the below playbook which is present in Cloud_os directory.
FILE Pathi - /home/ubuntu/cello/src/agent/ansible/roles/cloud_os/namenode/tasks/apply.yml
**CODE**
---
- name: Setup /etc/hosts for the node
lineinfile:
dest: "/etc/hosts"
insertafter: EOF
state: present
line: "{{ item }}"
with_lines: cat "{{ playbook_dir }}/../../vars/fabrichosts"
Now I would like to use the private key certificate to login to the Hosts or need to do necessary modification for the code to work.
Regards,
Nishanth Kumar
Can anchor nodes communicate between different channel?
[ ](https://chat.hyperledger.org/channel/cello?msg=8rg6s68WaZrp8Yk8M) @shuailaotou NO. As far as I know
@GowriR Thx
Does anywhere here deploy fabric using only swarm network manually ?
Has joined the channel.
Has joined the channel.
Hi all, im trying to deploy a smart contract in the user-dashboard of hyperledger cello but i have the following error in the logs:
cello-user-dashboard | error: [client-utils.js]: sendPeersProposal - Promise is rejected: Error: 2 UNKNOWN: Error starting container: Failed to generate platform-specific docker build: Error returned from build: 1 "can't load package: package github.com/5bcf32f00c1b010046f9d090: cannot find package "github.com/5bcf32f00c1b010046f9d090" in any of:
cello-user-dashboard | /opt/go/src/github.com/5bcf32f00c1b010046f9d090 (from $GOROOT)
cello-user-dashboard | /chaincode/input/src/github.com/5bcf32f00c1b010046f9d090 (from $GOPATH)
cello-user-dashboard | /opt/gopath/src/github.com/5bcf32f00c1b010046f9d090
cello-user-dashboard | "
cello-user-dashboard | at new createStatusError (/packages/fabric-1.0/node_modules/grpc/src/client.js:64:15)
cello-user-dashboard | at /packages/fabric-1.0/node_modules/grpc/src/client.js:583:15
Anyone can help me?
Has joined the channel.
Greetings everyone. I am setting up multi-node Hyperledger Fabrics with Hyperledger. I've already set up Hyperledger Cello and added to the network 2 hosts but I can't connect one host to another so that they will be able to synchronize they Chaincodes (same on 2 hosts). How can I do this?
Has joined the channel.
Hello, I'm trying to setup fabric network with the ansible agent and it fails with the following error```
```
Hello, I'm trying to setup fabric network with the ansible agent and it fails with the following error```
``` fatal: [10.74.22.146]: FAILED! => {"msg": "The conditional check 'fabric.composer == true or fabric.composer is undefined' failed. The error was: error while evaluating conditional (fabric.composer == true or fabric.composer is undefined): 'dict object' has no attribute 'composer'\n\nThe error appears to have been in '/opt/cello/cello/src/agent/ansible/roles/deploy_compose/plays.yml': line 56, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n tasks:\n - include_tasks: \"composersetup/tasks/{{ mode }}.yml\"\n ^ here\nWe could be wrong, but this one looks like it might be an issue with\nmissing quotes. Always quote template expression brackets when they\nstart a value. For instance:\n\n with_items:\n - {{ foo }}\n\nShould be written as:\n\n with_items:\n - \"{{ foo }}\"\n"}
Hello, I'm trying to setup fabric network with the ansible agent and it fails with the following error```
``` fatal: [10.74.22.146]: FAILED! => {"msg": "The conditional check 'fabric.composer == true or fabric.composer is undefined' failed. The error was: error while evaluating conditional (fabric.composer == true or fabric.composer is undefined): 'dict object' has no attribute 'composer'\n\nThe error appears to have been in '/opt/cello/cello/src/agent/ansible/roles/deploy_compose/plays.yml': line 56, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n tasks:\n - include_tasks: \"composersetup/tasks/{{ mode }}.yml\"\n ^ here\nWe could be wrong, but this one looks like it might be an issue with\nmissing quotes. Always quote template expression brackets when they\nstart a value. For instance:\n\n with_items:\n - {{ foo }}\n\nShould be written as:\n\n with_items:\n - \"{{ foo }}\"\n"}
Does anyone had the same problem? I'm using the example from the tutorial https://cello.readthedocs.io/en/latest/worker_ansible_docker/
Hello everyone. Can I add organization peers on remote host with only Cello? Or I need swarm and/or k8s?
@dlarukov yes you need that, Cello could deploy Fabric cluster to docker or k8s, since Cello have different agents to handle that.
Has joined the channel.
Has joined the channel.
Has joined the channel.
@baohua I
```
11:55:57
11:55:57 -- Docs: https://docs.pytest.org/en/latest/warnings.html
11:55:57 ===================== 3 passed, 1 warnings in 0.30 seconds =====================
11:55:58 flake8 create: /w/workspace/cello-verify-x86_64/.tox/flake8
11:56:01 flake8 installdeps: flake8, -r/w/workspace/cello-verify-x86_64/src/requirements.txt
11:56:16 flake8 installed: amqp==2.3.2,aniso8601==4.0.1,asn1crypto==0.24.0,bcrypt==3.1.0,billiard==3.5.0.4,cached-property==1.5.1,cachetools==2.1.0,celery==4.2.1,certifi==2018.10.15,cffi==1.11.5,Click==7.0,cryptography==2.3.1,docker==2.7.0,docker-compose==1.18.0,docker-pycreds==0.3.0,dockerpty==0.4.1,docopt==0.6.2,enum34==1.1.6,eventlet==0.22.1,Faker==0.7.18,flake8==3.6.0,Flask==1.0.2,Flask-Login==0.4.0,Flask-RESTful==0.3.6,Flask-SocketIO==2.9.6,Flask-Testing==0.6.2,google-auth==1.5.1,greenlet==0.4.12,gunicorn==19.6.0,holdup==1.6.0,idna==2.7,ipaddress==1.0.22,ItsDangerous==1.0.0,Jinja2==2.10,jsonschema==2.6.0,kombu==4.2.1,kubernetes==5.0.0,lxml==4.2.5,MarkupSafe==1.0,marshmallow==2.14.0,mccabe==0.6.1,mongoengine==0.13.0,oauthlib==2.1.0,pkg-resources==0.0.0,pyasn1==0.4.4,pyasn1-modules==0.2.2,pycodestyle==2.4.0,pycparser==2.19,pyflakes==2.0.0,pymongo==3.4.0,pyOpenSSL==18.0.0,python-dateutil==2.7.4,python-engineio==2.3.2,python-socketio==2.0.0,pytz==2018.6,pyvmomi==6.5,PyYAML==3.13,redis==2.10.6,requests==2.13.0,requests-oauthlib==1.0.0,rsa==4.0,six==1.11.0,texttable==0.9.1,urllib3==1.24,vine==1.1.4,websocket-client==0.53.0,Werkzeug==0.14.1
11:56:16 flake8 runtests: PYTHONHASHSEED='1291940183'
11:56:16 flake8 runtests: commands[0] | flake8 --ignore=E402,F401 /w/workspace/cello-verify-x86_64/src
11:56:18 /w/workspace/cello-verify-x86_64/src/modules/host.py:123:12: W504 line break after binary operator
11:56:18 /w/workspace/cello-verify-x86_64/src/modules/host.py:272:15: W504 line break after binary operator
11:56:18 /w/workspace/cello-verify-x86_64/src/common/utils.py:72:22: W504 line break after binary operator
11:56:18 /w/workspace/cello-verify-x86_64/src/common/stringvalidator.py:11:21: W605 invalid escape sequence '\/'
11:56:18 /w/workspace/cello-verify-x86_64/src/common/stringvalidator.py:12:31: W605 invalid escape sequence '\.'
11:56:18 /w/workspace/cello-verify-x86_64/src/common/stringvalidator.py:15:11: W605 invalid escape sequence '\-'
11:56:18 /w/workspace/cello-verify-x86_64/src/common/stringvalidator.py:15:18: W605 invalid escape sequence '\.'
11:56:18 /w/workspace/cello-verify-x86_64/src/common/stringvalidator.py:15:27: W605 invalid escape sequence '\-'
11:56:18 /w/workspace/cello-verify-x86_64/src/common/stringvalidator.py:15:35: W605 invalid escape sequence '\.'
11:56:18 /w/workspace/cello-verify-x86_64/src/common/stringvalidator.py:17:13: W605 invalid escape sequence '\_'
11:56:18 /w/workspace/cello-verify-x86_64/src/common/stringvalidator.py:44:13: F841 local variable 'e' is assigned to but never used
11:56:18 /w/workspace/cello-verify-x86_64/src/common/stringvalidator.py:58:9: F841 local variable 'e' is assigned to but never used
11:56:18 /w/workspace/cello-verify-x86_64/src/common/stringvalidator.py:74:9: F841 local variable 'e' is assigned to but never used
11:56:18 /w/workspace/cello-verify-x86_64/src/common/stringvalidator.py:81:9: F841 local variable 'e' is assigned to but never used
11:56:18 /w/workspace/cello-verify-x86_64/src/common/fabric_network.py:144:17: W504 line break after binary operator
11:56:18 /w/workspace/cello-verify-x86_64/src/agent/vsphere/host_operations.py:321:21: W504 line break after binary operator
11:56:18 /w/workspace/cello-verify-x86_64/src/agent/vsphere/host_operations.py:440:13: F841 local variable 'e' is assigned to but never used
11:56:18 /w/workspace/cello-verify-x86_64/src/agent/vsphere/host.py:47:9: F841 local variable 'e' is assigned to but never used
11:56:18 /w/workspace/cello-verify-x86_64/src/agent/k8s/cluster_operations.py:91:12: W504 line break after binary operator
11:56:18 /w/workspace/cello-verify-x86_64/src/agent/k8s/cluster_operations.py:131:16: W504 line break after binary operator
11:56:18 /w/workspace/cello-verify-x86_64/src/agent/k8s/cluster_operations.py:434:20: W504 line break after binary operator
11:56:18 /w/workspace/cello-verify-x86_64/src/agent/docker/docker_swarm.py:37:31: W504 line break after binary operator
11:56:18 /w/workspace/cello-verify-x86_64/src/agent/docker/docker_swarm.py:56:19: W504 line break after binary operator
11:56:18 /w/workspace/cello-verify-x86_64/src/agent/docker/docker_swarm.py:524:27: W504 line break after binary operator
11:56:18 ERROR: InvocationError: '/w/workspace/cello-verify-x86_64/.tox/flake8/bin/flake8 --ignore=E402,F401 /w/workspace/cello-verify-x86_64/src'
11:56:18 ___________________________________ summary ____________________________________
11:56:18 py35: commands succeeded
11:56:18 ERROR: flake8: commands failed
11:56:18 Makefile:180: recipe for target 'check' failed
11:56:18 make: *** [check] Error 1
```
@baohua my new patch didn't pass the jobbuilder's verification, I saw some format check was failed in jenkins log, however, I didn't touch any of these files in my patch
```
11:55:57
11:55:57 -- Docs: https://docs.pytest.org/en/latest/warnings.html
11:55:57 ===================== 3 passed, 1 warnings in 0.30 seconds =====================
11:55:58 flake8 create: /w/workspace/cello-verify-x86_64/.tox/flake8
11:56:01 flake8 installdeps: flake8, -r/w/workspace/cello-verify-x86_64/src/requirements.txt
11:56:16 flake8 installed: amqp==2.3.2,aniso8601==4.0.1,asn1crypto==0.24.0,bcrypt==3.1.0,billiard==3.5.0.4,cached-property==1.5.1,cachetools==2.1.0,celery==4.2.1,certifi==2018.10.15,cffi==1.11.5,Click==7.0,cryptography==2.3.1,docker==2.7.0,docker-compose==1.18.0,docker-pycreds==0.3.0,dockerpty==0.4.1,docopt==0.6.2,enum34==1.1.6,eventlet==0.22.1,Faker==0.7.18,flake8==3.6.0,Flask==1.0.2,Flask-Login==0.4.0,Flask-RESTful==0.3.6,Flask-SocketIO==2.9.6,Flask-Testing==0.6.2,google-auth==1.5.1,greenlet==0.4.12,gunicorn==19.6.0,holdup==1.6.0,idna==2.7,ipaddress==1.0.22,ItsDangerous==1.0.0,Jinja2==2.10,jsonschema==2.6.0,kombu==4.2.1,kubernetes==5.0.0,lxml==4.2.5,MarkupSafe==1.0,marshmallow==2.14.0,mccabe==0.6.1,mongoengine==0.13.0,oauthlib==2.1.0,pkg-resources==0.0.0,pyasn1==0.4.4,pyasn1-modules==0.2.2,pycodestyle==2.4.0,pycparser==2.19,pyflakes==2.0.0,pymongo==3.4.0,pyOpenSSL==18.0.0,python-dateutil==2.7.4,python-engineio==2.3.2,python-socketio==2.0.0,pytz==2018.6,pyvmomi==6.5,PyYAML==3.13,redis==2.10.6,requests==2.13.0,requests-oauthlib==1.0.0,rsa==4.0,six==1.11.0,texttable==0.9.1,urllib3==1.24,vine==1.1.4,websocket-client==0.53.0,Werkzeug==0.14.1
11:56:16 flake8 runtests: PYTHONHASHSEED='1291940183'
11:56:16 flake8 runtests: commands[0] | flake8 --ignore=E402,F401 /w/workspace/cello-verify-x86_64/src
11:56:18 /w/workspace/cello-verify-x86_64/src/modules/host.py:123:12: W504 line break after binary operator
11:56:18 /w/workspace/cello-verify-x86_64/src/modules/host.py:272:15: W504 line break after binary operator
11:56:18 /w/workspace/cello-verify-x86_64/src/common/utils.py:72:22: W504 line break after binary operator
11:56:18 /w/workspace/cello-verify-x86_64/src/common/stringvalidator.py:11:21: W605 invalid escape sequence '\/'
11:56:18 /w/workspace/cello-verify-x86_64/src/common/stringvalidator.py:12:31: W605 invalid escape sequence '\.'
11:56:18 /w/workspace/cello-verify-x86_64/src/common/stringvalidator.py:15:11: W605 invalid escape sequence '\-'
11:56:18 /w/workspace/cello-verify-x86_64/src/common/stringvalidator.py:15:18: W605 invalid escape sequence '\.'
11:56:18 /w/workspace/cello-verify-x86_64/src/common/stringvalidator.py:15:27: W605 invalid escape sequence '\-'
11:56:18 /w/workspace/cello-verify-x86_64/src/common/stringvalidator.py:15:35: W605 invalid escape sequence '\.'
11:56:18 /w/workspace/cello-verify-x86_64/src/common/stringvalidator.py:17:13: W605 invalid escape sequence '\_'
11:56:18 /w/workspace/cello-verify-x86_64/src/common/stringvalidator.py:44:13: F841 local variable 'e' is assigned to but never used
11:56:18 /w/workspace/cello-verify-x86_64/src/common/stringvalidator.py:58:9: F841 local variable 'e' is assigned to but never used
11:56:18 /w/workspace/cello-verify-x86_64/src/common/stringvalidator.py:74:9: F841 local variable 'e' is assigned to but never used
11:56:18 /w/workspace/cello-verify-x86_64/src/common/stringvalidator.py:81:9: F841 local variable 'e' is assigned to but never used
11:56:18 /w/workspace/cello-verify-x86_64/src/common/fabric_network.py:144:17: W504 line break after binary operator
11:56:18 /w/workspace/cello-verify-x86_64/src/agent/vsphere/host_operations.py:321:21: W504 line break after binary operator
11:56:18 /w/workspace/cello-verify-x86_64/src/agent/vsphere/host_operations.py:440:13: F841 local variable 'e' is assigned to but never used
11:56:18 /w/workspace/cello-verify-x86_64/src/agent/vsphere/host.py:47:9: F841 local variable 'e' is assigned to but never used
11:56:18 /w/workspace/cello-verify-x86_64/src/agent/k8s/cluster_operations.py:91:12: W504 line break after binary operator
11:56:18 /w/workspace/cello-verify-x86_64/src/agent/k8s/cluster_operations.py:131:16: W504 line break after binary operator
11:56:18 /w/workspace/cello-verify-x86_64/src/agent/k8s/cluster_operations.py:434:20: W504 line break after binary operator
11:56:18 /w/workspace/cello-verify-x86_64/src/agent/docker/docker_swarm.py:37:31: W504 line break after binary operator
11:56:18 /w/workspace/cello-verify-x86_64/src/agent/docker/docker_swarm.py:56:19: W504 line break after binary operator
11:56:18 /w/workspace/cello-verify-x86_64/src/agent/docker/docker_swarm.py:524:27: W504 line break after binary operator
11:56:18 ERROR: InvocationError: '/w/workspace/cello-verify-x86_64/.tox/flake8/bin/flake8 --ignore=E402,F401 /w/workspace/cello-verify-x86_64/src'
11:56:18 ___________________________________ summary ____________________________________
11:56:18 py35: commands succeeded
11:56:18 ERROR: flake8: commands failed
11:56:18 Makefile:180: recipe for target 'check' failed
11:56:18 make: *** [check] Error 1
```
https://gerrit.hyperledger.org/r/#/c/27146/
@Luke_Chen seems flake8 errors
can you pass the checking locally? otherwise, needs to check the format problems.
I can pass locally
I didn't touch these files :joy:
hmm, maybe the ci server changes its flake version. can you check if the version matched?
no worry, it's code format issues, let's fix the format quickly
ok, let's figure it out
@baohua yes, the CI server has a higher version of flask8 which is 3.6.0
@baohua yes, the CI server has a higher version of flake8 which is 3.6.0
but we used the 3.5.0 before
https://gerrit.hyperledger.org/r/#/c/27221/
@baohua @tongli @hightall would you please help to review this patch? since it's blocking our further work
Thanks
+2!
let's switch to flake 3.6.0, too
https://gerrit.hyperledger.org/r/#/c/27204/
+2!
https://gerrit.hyperledger.org/r/#/c/27241/
https://gerrit.hyperledger.org/r/#/c/27241/ @Luke_Chen @baohua
https://gerrit.hyperledger.org/r/#/c/27241/ @Luke_Chen @baohua
please review, thanks.
https://gerrit.hyperledger.org/r/#/c/27247/ @Luke_Chen @baohua
please review, thanks.
[ ](https://chat.hyperledger.org/channel/cello?msg=sxM7kWMeqmds6fGvP) @luckydogchina CI reports error, please help fix first :)
[ ](https://chat.hyperledger.org/channel/cello?msg=fX6diQv2J2B9SMr6v) @baohua OK
Has left the channel.
https://gerrit.hyperledger.org/r/#/c/27245/ @Luke_Chen @baohua
please review.
@baohua this patch https://gerrit.hyperledger.org/r/#/c/27247/1/ has multiple commits, will these commits be squashed automatically when merging
@baohua this patch https://gerrit.hyperledger.org/r/#/c/27247/1/ has multiple commits, will these commits are squashed automatically when merging
Has joined the channel.
Hello does cello supports only Fabric networks ?? is there any plan to support sawtooth networks?
Ibm will buy redhat for $34b. Big news
No automatic squash @Luke_Chen [ ](https://chat.hyperledger.org/channel/cello?msg=hqCjWmYX3nwoEmqKQ)
@luckydogchina https://gerrit.hyperledger.org/r/#/c/27247/ added some comments
please take a look at your convenience.
[ ](https://chat.hyperledger.org/channel/cello?msg=GEXkpGjuzeiNozRdB) @Luke_Chen ok
[ ](https://chat.hyperledger.org/channel/cello?msg=F5TwGJJPLaR3qMdEt) @Luke_Chen ok
Hi All, Is cello supports fabric 1.3 release for aws??
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=m9TS6yGbNFOQQXDdUG) @tongli big news, and definitely exciting for the open-source community!
https://gerrit.hyperledger.org/r/#/c/27247/
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=qxbyojWA5MNaadR4J) @Luke_Chen Merged, thanks!
https://gerrit.hyperledger.org/r/#/c/27322/
https://gerrit.hyperledger.org/r/#/c/27331/
[ ](https://chat.hyperledger.org/channel/cello?msg=bpnHs5K4QfS3sRapW) @Luke_Chen +2!
https://gerrit.hyperledger.org/r/27342
https://gerrit.hyperledger.org/r/#/c/27353/
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=FR5LTfQo4yW3RWFwX) @luckydogchina Merged!
[ ](https://chat.hyperledger.org/channel/cello?msg=e9mCyC5XWXAHtoACP) @baohua thanks.
Has joined the channel.
@baohua: can I install cello with linux subsystem ubuntu on windows 10?
I alread tried to install and successful
but I can not add Host
this is error: [2018-11-06 06:54:51,114] ERROR [agent.docker.docker_swarm] [docker_swarm.py:173 check_daemon()] - Exception in check_daemon Error while fetching server API version: HTTPConnectionPool(host='10.61.189.133', port=2375): Max retries exceeded with url: /version (Caused by NewConnectionError('
@tuanpvi i suppose there will be challenges on windows. however, cello's running is based on docker now.
besides, linux and macos are evaluated with cello.
@baohua: thanks!
I'm trying to find some thing from the docs
some configuration i can not find in ubuntu 16.04 as the docs suggested, for example i can not find the directory usr/lib/systemd/system/docker.service
Has joined the channel.
@baohua @hightall @Luke_Chen Please review this patch to fix a rather urgent issue. https://gerrit.hyperledger.org/r/#/c/27403/
Thanks for taking time to see it.
@baohua @Luke_Chen @hightall Please review the patch. It actually fixed a rather bad problem introduced by an earlier patch by the other developer.
https://gerrit.hyperledger.org/r/#/c/27403/
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=yRaNetEh5gBsP5MJs) @tuanpvi path /lib/systemd/system/docker.service is right.
Has joined the channel.
hello, every one. now the cello is support for fabric 1.3?
Has left the channel.
@baohua @hightall @Luke_Chen Please review this patch for allowing multiple cloud to participating in a fabric network. https://gerrit.hyperledger.org/r/27470
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=GXyyYLwkviNANWmF9) @tongli +2!
Has joined the channel.
Hi, everyone. Is there a workable release I can use for test ? I try to build in my local machine or my cloud servers using master and 0.9-alpha, I failed all of these works. Maybe wrong docker version? I try docker 1.13.0, but it didn't work. Thanks
[ ](https://chat.hyperledger.org/channel/cello?msg=QwsTbRapq3roL8RhT) @meilier docker1.13.0 may be stale,1.17+ may work.
I tried. The peer container in worker(I use docker as worker) always restarting. Cannot run peer because error when setting up MSP from directory /etc/hyperledger/fabric/msp: err Could not load a valid signer certificate from directory /etc/hyperledger/fabric/msp/signcerts, err stat /etc/hyperledger/fabric/msp/signcerts: no such file or directory
I tried. The peer container in worker(I use docker as worker) always restarting. ```
``` Cannot run peer because error when setting up MSP from directory /etc/hyperledger/fabric/msp: err Could not load a valid signer certificate from directory /etc/hyperledger/fabric/msp/signcerts, err stat /etc/hyperledger/fabric/msp/signcerts: no such file or directory
I tried. The peer container in worker(I use docker as worker) always restarting.
The docker logs is below, how can I fix this ? Any wrong operation cause this problem? Thanks for replay:grinning:
Cannot run peer because error when setting up MSP from directory /etc/hyperledger/fabric/msp: err Could not load a valid signer certificate from directory /etc/hyperledger/fabric/msp/signcerts, err stat /etc/hyperledger/fabric/msp/signcerts: no such file or directory
@baohua, thx
Works Now:smile:
@baohua @hightall @Luke_Chen found a bug in the ansible agent when creating json connection profile. Fixed in this patch, please review https://gerrit.hyperledger.org/r/27489. Thanks very much!
Hello Team
Need help for the below error
ubuntu@ip-172-31-30-36:~/test1$ grep -rn "Restart the docker services" .
./vars/ansible.log:622:2018-11-08 12:58:43,201 p=17470 u=ubuntu | TASK [Restart the docker services] ********************************** *******************************************************************
./vars/ansible.log:1044:2018-11-08 13:09:29,116 p=23942 u=ubuntu | TASK [Restart the docker services] ********************************* ********************************************************************
./roles/env_flanneld/dnsdocker/tasks/apply.yml:31:- name: Restart the docker services
TASK [Restart the docker services] *****************************************************************************************************
failed: [34.232.95.135] (item=docker.socket) => {"changed": false, "item": "docker.socket", "msg": "Unable to enable service docker.sock et: Failed to execute operation: Unit file is masked\n"}
failed: [35.175.131.81] (item=docker.socket) => {"changed": false, "item": "docker.socket", "msg": "Unable to enable service docker.sock et: Failed to execute operation: Unit file is masked\n"}
failed: [100.24.97.159] (item=docker.socket) => {"changed": false, "item": "docker.socket", "msg": "Unable to enable service docker.sock et: Failed to execute operation: Unit file is masked\n"}
failed: [34.232.95.135] (item=docker) => {"changed": false, "item": "docker", "msg": "Unable to restart service docker: Failed to restar t docker.service: Unit docker.socket is masked.\n"}
failed: [35.175.131.81] (item=docker) => {"changed": false, "item": "docker", "msg": "Unable to restart service docker: Failed to restar t docker.service: Unit docker.socket is masked.\n"}
failed: [100.24.97.159] (item=docker) => {"changed": false, "item": "docker", "msg": "Unable to restart service docker: Failed to restar t docker.service: Unit docker.socket is masked.\n"}
to retry, use: --limit @/home/ubuntu/test1/initcluster.retry
Has left the channel.
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=LywEmCGkgT8rsQ8Zw) @tongli @hightall any some one provide solution for this issue
Has joined the channel.
Hi All , can anyone help me y health failed in hyperledger cello
@manoj485 can you provide any log?
Screenshot from 2018-11-15 11-48-07.png
@Luke_Chen
@manoj485 how about your peer and orderer containers?
Did they look normal?
yes
demo_90
Host: demo Network Type: fabric-1.0
Consensus Plugin: solo
Owner
Empty
Create Time
2018-11-15 11:38:32
Status
Deleting
Health
FAIL
demo_90
Host: demo Network Type: fabric-1.0
Consensus Plugin: solo
Owner
Empty
Create Time
2018-11-15 11:38:32
Status
Deleting
Health
FAIL
now is deleting
before status in running state but now it is deleting
it means the Fabric containers didn't set up correctly.
How did you setup your Cello worker node?
sudo systemctl stop docker.service
sudo dockerd -H fd:// -H unix:///var/run/docker.sock -H tcp://0.0.0.0:2375 --default-ulimit=nofile=8192:16384 --default-ulimit=nproc=8192:16384 -D &
i used these commands after that make setup-worker
health check will only ping these containers to see whether the service is alive, check your firewall provision and make sure all ports of your worker node can be accessed.
after that firewall setup
should firewall is active in both systems master and worker
you should disable it first to avoid influence from the network policy.
demo_91
Host: demo Network Type: fabric-1.0
Consensus Plugin: solo
Owner
Empty
Create Time
2018-11-15 12:40:22
Status
Running
Health
FAIL
@Luke_Chen
docker is not working continues in worker node @Luke_Chen
Has joined the channel.
owner id empty in hyperledgercello , can any one kown why this issue has been raised.
I was tring to install hyperledger cello with ansible on aws and failed with below error:
ailed: [13.57.57.100] (item=docker.socket) => {"changed": false, "item": "docker.socket", "msg": "Unable to enable service docker.socket: Failed to execute operation: Unit file is masked\n"} failed: [54.183.152.177] (item=docker.socket) => {"changed": false, "item": "docker.socket", "msg": "Unable to enable service docker.socket: Failed to execute operation: Unit file is masked\n"} failed: [13.57.57.100] (item=docker) => {"changed": false, "item": "docker", "msg": "Unable to restart service docker: Failed to restart docker.service: Unit docker.socket is masked.\n"} failed: [54.183.152.177] (item=docker) => {"changed": false, "item": "docker", "msg": "Unable to restart service docker: Failed to restart docker.service: Unit docker.socket is masked.\n"}
The same scripts ran fine earlier.. now it has failed.
Found that the problem is with docker version. Below are the differences:
## Working deployment: ii docker-ce 18.06.1~ce~3-0~ubuntu amd64 Docker: the open-source application container engine rc docker.io 17.03.2-0ubuntu2~16.04.1 amd64 Linux container runtime ## Non working deplyment: ii docker-ce 5:18.09.0~3-0~ubuntu-xenial amd64 Docker: the open-source application container engine ii docker-ce-cli 5:18.09.0~3-0~ubuntu-xenial amd64 Docker CLI: the open-source application container engine rc docker.io 17.03.2-0ubuntu2~16.04.1 amd64 Linux container runtime
## Working deployment: ii docker-ce 18.06.1~ce~3-0~ubuntu amd64 Docker: the open-source application container engine rc docker.io 17.03.2-0ubuntu2~16.04.1 amd64 Linux container runtime ## Non working deplyment: ii docker-ce 5:18.09.0~3-0~ubuntu-xenial amd64 Docker: the open-source application container engine ii docker-ce-cli 5:18.09.0~3-0~ubuntu-xenial amd64 Docker CLI: the open-source application container engine rc docker.io 17.03.2-0ubuntu2~16.04.1 amd64 Linux container runtime
## Working deployment:
ii docker-ce 18.06.1~ce~3-0~ubuntu amd64 Docker: the open-source application container engine rc docker.io 17.03.2-0ubuntu2~16.04.1 amd64 Linux container runtime
## Non working deplyment:
ii docker-ce 5:18.09.0~3-0~ubuntu-xenial amd64 Docker: the open-source application container engine ii docker-ce-cli 5:18.09.0~3-0~ubuntu-xenial amd64 Docker CLI: the open-source application container engine rc docker.io 17.03.2-0ubuntu2~16.04.1 amd64 Linux container runtime
## Working deployment:
ii docker-ce 18.06.1~ce~3-0~ubuntu amd64 Docker: the open-source application container engine
rc docker.io 17.03.2-0ubuntu2~16.04.1 amd64 Linux container runtime
## Non working deplyment:
ii docker-ce 5:18.09.0~3-0~ubuntu-xenial amd64 Docker: the open-source application container engine
ii docker-ce-cli 5:18.09.0~3-0~ubuntu-xenial amd64 Docker CLI: the open-source application container engine
rc docker.io 17.03.2-0ubuntu2~16.04.1 amd64 Linux container runtime
Need help to resovle this issue.
@manoj485 You need to make sure all ports needed in worker machine is open, better to restart your service and machine to make it work. I get the same error 'failed' for worker node. You can type 'docker log + your_container_peer_id', see if you get error like this '/etc/hyperledger/fabric/msp/signcerts: no such file or directory'
@Gaurang I also found that Docker Daemon Setup won't work in new version of docker 18.09, so I back to 18.06.
https://github.com/hyperledger/cello/blob/master/docs/setup_worker_docker.md
@meilier Thanks for your update.
actually I am using ansible script to setup the fabric and I am running ansible scripts from my PC. I could see 2-3 places where we are setting docker.j2 templete for docker.service.
Could you please let me know where should I update the above mentioned setting to make sure that we use docker version 18.06 ?
I'm not familiar with ansible script. But you can install specific version of docker like this `apt-get install docker-ce=
I'm not familiar with ansible script. But you can install specific version of docker like this `apt-get install docker-ce=
I'm not familiar with ansible script. But you can install specific version of docker like this `apt-get install docker-ce=
@meilier Actually ansible script them self create ec2 instance and then install docker in it as a node initial setup. Thats the reason i can not go and install the docker version as you mentioned. I need to find the hook where exactly the step at which it is downloading the docker and installing it. As per my finding I have found below two hook but not sure whether it will resolve the issue or not.
https://github.com/hyperledger/cello/blob/master/src/agent/ansible/roles/cloud_aws/provision/templates/ubuntu.j2
@meilier Actually ansible script them self create ec2 instance and then install docker in it as a node initial setup. Thats the reason i can not go and install the docker version as you mentioned. I need to find the hook where exactly the step at which it is downloading the docker and installing it. As per my finding I have found below two hook but not sure whether it will resolve the issue or not.
https://github.com/hyperledger/cello/blob/master/src/agent/ansible/roles/cloud_aws/provision/templates/ubuntu.j2
https://github.com/hyperledger/cello/blob/master/src/agent/ansible/roles/env_flanneld/fastinitnode/tasks/apply.yml
I think @tongli can help us finding that hook. I could see his commit on these files.
@all https://jira.hyperledger.org/browse/CE-508 jira link for Redesign data model, anyone can feel free to add your comments
Has left the channel.
Has joined the channel.
I followed the guide to run the latest version of Cello, everything looks good, but I just can't access localhost:8080, can anyone help ?
I can access https://localhost:8084, which looks good, although I haven no idea what it is
@baohua
@Luke_Chen
[ ](https://chat.hyperledger.org/channel/cello?msg=cwxuPZ9x7yJNZPbbZ) @Jim1 1 can you help post the container log information? i.e., make logs
@all Dear all, this week's Cello meeting is cancelled due to thanksgiving days. Thanks!
Has left the channel.
Has joined the channel.
Has joined the channel.
need some help with the getting started instructions for Cello provided here: https://hyperledger-cello.readthedocs.io/en/latest/ansible-agent-all-in-one/
need some help with the getting started instructions for Cello provided here: https://hyperledger-cello.readthedocs.io/en/latest/ansible-agent-all-in-one/
the command ansible-playbook -i run/runhosts -e "mode=apply env=vb" initcluster.yml --skip-tags="resetconn" gives an error message under "Restart the docker services", cannot restart docker.socket is masked
the command ansible-playbook -i run/runhosts -e "mode=apply env=vb" initcluster.yml --skip-tags="resetconn" gives an error message under "Restart the docker services", cannot restart docker.socket is masked
TASK [Reload daemon service] ******************************************************************************************************
changed: [10.0.2.15]
TASK [Restart the docker services] ************************************************************************************************
failed: [10.0.2.15] (item=docker.socket) => {"changed": false, "item": "docker.socket", "msg": "Unable to enable service docker.socket: Failed to execute operation: Unit file is masked\n"}
failed: [10.0.2.15] (item=docker) => {"changed": false, "item": "docker", "msg": "Unable to restart service docker: Failed to restart docker.service: Unit docker.socket is masked.\n"}
to retry, use: --limit @/home/vagrant/cello/src/agent/ansible/initcluster.retry
PLAY RECAP ************************************************************************************************************************
10.0.2.15 : ok=45 changed=24 unreachable=0 failed=1
cloud : ok=2 changed=0 unreachable=0 failed=0
TASK [Reload daemon service] ******************************************************************************************************
changed: [10.0.2.15]
TASK [Restart the docker services] ************************************************************************************************
failed: [10.0.2.15] (item=docker.socket) => {"changed": false, "item": "docker.socket", "msg": "Unable to enable service docker.socket: Failed to execute operation: Unit file is masked\n"}
failed: [10.0.2.15] (item=docker) => {"changed": false, "item": "docker", "msg": "Unable to restart service docker: Failed to restart docker.service: Unit docker.socket is masked.\n"}
to retry, use: --limit @/home/vagrant/cello/src/agent/ansible/initcluster.retry
PLAY RECAP ************************************************************************************************************************
10.0.2.15 : ok=45 changed=24 unreachable=0 failed=1
cloud : ok=2 changed=0 unreachable=0 failed=0
TASK [Reload daemon service] ******************************************************************************************************
changed: [10.0.2.15]
TASK [Restart the docker services] ************************************************************************************************
failed: [10.0.2.15] (item=docker.socket) => {"changed": false, "item": "docker.socket", "msg": "Unable to enable service docker.socket: Failed to execute operation: Unit file is masked\n"}
failed: [10.0.2.15] (item=docker) => {"changed": false, "item": "docker", "msg": "Unable to restart service docker: Failed to restart docker.service: Unit docker.socket is masked.\n"}
to retry, use: --limit @/home/vagrant/cello/src/agent/ansible/initcluster.retry
PLAY RECAP ************************************************************************************************************************
10.0.2.15 : ok=45 changed=24 unreachable=0 failed=1
cloud : ok=2 changed=0 unreachable=0 failed=0
Has left the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=dsNuHte8pWnACW8Cw) @Gaurang @Gaurang was able to get past this error by changing 'docker-ce' to ''docker-ce=18.06.0~ce~3-0~ubuntu' in file apply.yml in folder ~/cello/src/agent/ansible/roles/env_flanneld/fastinitnode/tasks
Working through the getting started example here: https://hyperledger-cello.readthedocs.io/en/latest/ansible-agent-all-in-one/
Running 'ansible-playbook -i run/runhosts -e "mode=apply env=bc2nd" setupfabric.yml' throws the error shown below
TASK [Create peers rootca list] ********************************************************************************************************
skipping: [10.0.2.15] => (item={'org': u'sin', 'role': u'anchor', 'name': u'peer1st-sin'})
skipping: [10.0.2.15] => (item={'org': u'sin', 'role': u'worker', 'name': u'peer2nd-sin'})
TASK [Create orderers rootca list] *****************************************************************************************************
TASK [Joining the orderer rootca list and peerorg rootca list] *************************************************************************
skipping: [10.0.2.15]
TASK [Setup and initialize variables] **************************************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: IndexError: list index out of range
fatal: [10.0.2.15]: FAILED! => {"msg": "Unexpected failure during module execution.", "stdout": ""}
to retry, use: --limit @/home/ubuntu/cello/src/agent/ansible/setupfabric.retry
PLAY RECAP *****************************************************************************************************************************
10.0.2.15 : ok=43 changed=7 unreachable=0 failed=1
cloud : ok=40 changed=18 unreachable=0 failed=0
cello-keycloak-server | 03:22:23,769 ERROR [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (ServerService Thread Pool -- 54) Duplicate entry 'master-keycloak-admin' for key 'UK_RU8TT6T700S9V50BU18WS5HA6'
cello-keycloak-server | 03:22:23,771 INFO [org.hibernate.engine.jdbc.batch.internal.AbstractBatchImpl] (ServerService Thread Pool -- 54) HHH000010: On release of batch it still contained JDBC statements
cello-keycloak-server | 03:22:23,794 ERROR [org.keycloak.services] (ServerService Thread Pool -- 54) KC-SERVICES0010: Failed to add user 'keycloak-admin' to realm 'master': user with username exists
cello-keycloak-server | 03:22:23,875 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 54) RESTEASY002225: Deploying javax.ws.rs.core.Application: class org.keycloak.services.resources.KeycloakApplication
cello-keycloak-server | 03:22:23,878 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 54) RESTEASY002205: Adding provider class org.keycloak.services.error.KeycloakErrorHandler from Application class org.keycloak.services.resources.KeycloakApplication
cello-keycloak-server | 03:22:23,881 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 54) RESTEASY002200: Adding class resource org.keycloak.services.resources.ThemeResource from Application class org.keycloak.services.resources.KeycloakApplication
cello-keycloak-server | 03:22:23,882 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 54) RESTEASY002200: Adding class resource org.keycloak.services.resources.JsResource from Application class org.keycloak.services.resources.KeycloakApplication
cello-keycloak-server | 03:22:23,882 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 54) RESTEASY002205: Adding provider class org.keycloak.services.filters.KeycloakTransactionCommitter from Application class org.keycloak.services.resources.KeycloakApplication
cello-keycloak-server | 03:22:23,883 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 54) RESTEASY002220: Adding singleton resource org.keycloak.services.resources.admin.AdminRoot from Application class org.keycloak.services.resources.KeycloakApplication
cello-keycloak-server | 03:22:23,883 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 54) RESTEASY002220: Adding singleton resource org.keycloak.services.resources.RobotsResource from Application class org.keycloak.services.resources.KeycloakApplication
cello-keycloak-server | 03:22:23,883 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 54) RESTEASY002220: Adding singleton resource org.keycloak.services.resources.WelcomeResource from Application class org.keycloak.services.resources.KeycloakApplication
cello-keycloak-server | 03:22:23,883 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 54) RESTEASY002210: Adding provider singleton org.keycloak.services.util.ObjectMapperResolver from Application class org.keycloak.services.resources.KeycloakApplication
cello-keycloak-server | 03:22:23,884 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 54) RESTEASY002220: Adding singleton resource org.keycloak.services.resources.RealmsResource from Application class org.keycloak.services.resources.KeycloakApplication
cello-keycloak-server | 03:22:24,020 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 54) WFLYUT0021: Registered web context: '/auth' for server 'default-server'
cello-keycloak-server | 03:22:24,054 INFO [org.jboss.as.server] (ServerService Thread Pool -- 40) WFLYSRV0010: Deployed "keycloak-server.war" (runtime-name : "keycloak-server.war")
cello-keycloak-server | 03:22:24,179 INFO [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0212: Resuming server
cello-keycloak-server | 03:22:24,189 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://127.0.0.1:9990/management
cello-keycloak-server | 03:22:24,189 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://127.0.0.1:9990
cello-keycloak-server | 03:22:24,191 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: Keycloak 4.5.0.Final (WildFly Core 5.0.0.Final) started in 22124ms - Started 626 of 904 services (623 services are lazy, passive or on-demand)
cello-watchdog | [2018-11-22 03:22:25,571] INFO [__main__] [watchdog.py:126 watch_run()] - Watchdog run checks with period = 15 s
@baohua this is the only RED error I can see when I ran make logs
$curl http://10.101.110.94:8080
Redirecting...
the above message was what I got when I tried to access http://
but failed
@hightall could you help have a look?
it looks like the new added keycloak causes my problem, is there a way I can disable it ?
[ ](https://chat.hyperledger.org/channel/cello?msg=7kaPCDxmMHkJAAnLN) @mehulshahnj @mehulshahnj Thanks for the help!!
ignore my preivous questions. I fixed all of them but i got a new problem
I can start cello operator dashboard successfully now, but when I tried to create a chain, its status is always "Deleting" and "Health" value is always "Fail"
there is no errors in the log after I ran make logs
anybody can help ?
@Gaurang
@Jim1 1 I am using cello ansible to deploy the fabric. I have not used dashboard so do not have much idea about it. Sorry.
@Gaurang it's OK, thank you
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=4k66uRaQS9je2AZdY) @Jim1 1 @Jim1 1 Please tell me how you solved this problem?
I ran "make all" before "make start"
@SadOraz
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=goEahjbHjqw4BSqzh) @Jim1 1 I tried, but did not help
Has left the channel.
what is your problem?
@SadOraz
[ ](https://chat.hyperledger.org/channel/cello?msg=Dtg3oKHi3uz86fieC) @Jim1 1
I'm trying to install the cello master node in a virtual box on ubuntu 16.04 according to the instructions. After make start, all the containers are started, but I cannot enter the dashboard operator (8080). At the same time, the keycloak panel works.
open .evn file under ./cello path, make sure SERVER_PUBLIC_IP has an valid ip address
@SadOraz
sorry, .env file
Has joined the channel.
Has joined the channel.
@Jim1 1 01:53:08,580 ERROR [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (ServerService Thread Pool -- 60) Duplicate entry 'master-keycloak-admin' for key 'UK_RU8TT6T700S9V50BU18WS5HA6'. How to fix this problem
:boy:
Has joined the channel.
Hello!
I would like to try out Cello! I've cloned the repo and run make setup-master then make start.
I am not sure what's happening....most of the time I got the following error: env variable SERVER_PUBLIC_IP not set. If somehow the service starts (after a couple of restart and clean). The dashboards are not reachable. (the user-dashboard exits immediately). What am I missing?
openssl
@Levilk "SERVER_PUBLIC_IP=0.0.0.0 make start" worked for me. At your Dashboard problem: Is everything up and running on docker ps? And what does you log (make logs) say?
[ ](https://chat.hyperledger.org/channel/cello?msg=LgPDh8ZQgSSLPGadP) @PaddyBN https://pastebin.com/S2aZZXHg and https://pastebin.com/zP2X6xmm are the logs of the dashboards. None of them reachable.
Running docker containers
Has joined the channel.
Has joined the channel.
Has left the channel.
https://jira.hyperledger.org/projects/CE/issues/CE-510
Does Cello support adding org dynamically or will it support this and when? @baohua
is there a production plan that what Cello is going to do in the future ?
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=YWmiwMTkPa7WpmvX6) @Jim1 1 I also have the same question..I am new to this group..I have some requirement to deploy blockchain network in multi node multi peer multi org and multi machine way..I tried with cello however I failed..is there any better documentation?
[ ](https://chat.hyperledger.org/channel/cello?msg=PMYun6icm2e9D2eJE) @Jim1 1 Yes, it's in current plan.
Updated the example workflow section, welcome to comment: https://docs.google.com/document/d/1Dw6cEKaul6FenORNkDcxvPDDKwpl0A5EmcJBlqAWJoU/edit#heading=h.rlnp2zcmpdu5 @here
[ ](https://chat.hyperledger.org/channel/cello?msg=ugzHeWX4YjKw7iSvK) @baohua @baohua - can we also configure off chain database in cello? Please let me know. I am new to this chat group, however I will keep learning to ask some sensible questions within some days..
Has joined the channel.
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=jcgg7abdTTMoEtXhY) @BhaskarMondal hmm, it's an interesting topic, but i'm not aware anyone wanna it in real production. so not in recent plan actually.
Has joined the channel.
Can cello mange partner onboarding? Suppose I have a running Fabric network and I have to add a new Organization to the existing network. How does cello takes care of this?
Can cello manage partner onboarding? Suppose I have a running Fabric network and I have to add a new Organization to the existing network. How does cello takes care of this?
@baohua solo is ok, but when I tried to create a Kafka one, the oder container always failed to start. I am sure it is not my problem, could you pls help?
Has joined the channel.
can I set up a Hyperledger Composer Application (BNA) using Cello?
[ ](https://chat.hyperledger.org/channel/cello?msg=S6aEPsQry9SwqG3q8) @Jim1 1 please read https://wiki.hyperledger.org/projects/cello/faq, if still not resolved, feel free to open a jira issue.
[ ](https://chat.hyperledger.org/channel/cello?msg=bYbaDS2jXzfwbbHMQ) @AlphaAsteroid hmm, you may use composer to connect the blockchains started by cello. From cello side, there's no blocking or special supporting with composer.
so Cello is used to setup the Fabric network right?
then I can use it as if it were deployed manually, right?
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=wJiZqKnu9byngsqye) @AlphaAsteroid The target is to help people manage blockchain network without spending unnecessary efforts like setup, combine as network, operate transactions and monitor status.
i cannot start the cello dashboard on my browser
i looked at the docker logs of the dashboard and found that:
AssertionError [ERR_ASSERTION]: [egg-passport-oauth2] config.passportOauth2.key required
at module.exports.app (/node_modules/egg-passport-oauth2/app.js:10:3)
at Hook.configDidLoad (/node_modules/egg-core/lib/lifecycle.js:93:9)
at Lifecycle.triggerConfigDidLoad (/node_modules/egg-core/lib/lifecycle.js:144:14)
at Lifecycle.triggerConfigWillLoad (/node_modules/egg-core/lib/lifecycle.js:138:10)
at AppWorkerLoader.loadCustomApp (/node_modules/egg-core/lib/loader/mixin/custom.js:37:20)
at AppWorkerLoader.load (/node_modules/egg/lib/loader/app_worker_loader.js:33:10)
at new Application (/node_modules/egg/lib/application.js:66:19)
at Object.
and more errors given by npm
i have tried changing the port 8080 to a different one as suggested in a stackoverflow forum
does anyone know the reason?
checked the docker, docker-compose version and running on ubuntu 16.04
so that shouldn't be the problem D:
I managed to make the v0.7 to work
Im stuck in adding Hosts
Has joined the channel.
When does cello will add support for sawtooth? Is there any development in this going on?
Has joined the channel.
@baohua @hightall @Luke_Chen Please review this patch which allows more configurations to be done for orderering system and kafka setup
https://gerrit.hyperledger.org/r/#/c/27921/
+2!
Has joined the channel.
Thanks Baohua.
https://gerrit.hyperledger.org/r/#/c/27962/
https://gerrit.hyperledger.org/r/#/c/28003/
@baohua how each org share crypto content
I am getting error while starting the dashboard on my browser after installation
AssertionError [ERR_ASSERTION]: [egg-passport-oauth2] config.passportOauth2.key required
at module.exports.app (/node_modules/egg-passport-oauth2/app.js:10:3)
at Hook.configDidLoad (/node_modules/egg-core/lib/lifecycle.js:93:9)
at Lifecycle.triggerConfigDidLoad (/node_modules/egg-core/lib/lifecycle.js:144:14)
at Lifecycle.triggerConfigWillLoad (/node_modules/egg-core/lib/lifecycle.js:138:10)
at AppWorkerLoader.loadCustomApp (/node_modules/egg-core/lib/loader/mixin/custom.js:37:20)
at AppWorkerLoader.load (/node_modules/egg/lib/loader/app_worker_loader.js:33:10)
at new Application (/node_modules/egg/lib/application.js:66:19)
at Object.
Has joined the channel.
Has joined the channel.
https://stackoverflow.com/questions/53492680/unable-to-login-into-hyperledger-cello-operator-dashboard
Installed cell after giving make start getting error in keycloak-server.
cello-initial-keycloak-server | 18:23:45,846 INFO [org.keycloak.connections.jpa.updater.liquibase.LiquibaseJpaUpdaterProvider] (ServerService Thread Pool -- 56) Initializing database schema. Using changelog META-INF/jpa-changelog-master.xml
cello-initial-keycloak-server | 18:28:37,796 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0348: Timeout after [300] seconds waiting for service container stability. Operation will roll back. Step that first updated the service container was 'add' at address '[
cello-initial-keycloak-server | ("core-service" => "management"),
cello-initial-keycloak-server | ("management-interface" => "http-interface")
cello-initial-keycloak-server | ]'
cello-initial-keycloak-server | 18:28:42,801 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0190: Step handler org.jboss.as.server.DeployerChainAddHandler$FinalRuntimeStepHandler@54b2cf9e for operation add-deployer-chains at address [] failed handling operation rollback -- java.util.concurrent.TimeoutException: java.util.concurrent.TimeoutException
cello-initial-keycloak-server | at org.jboss.as.controller.OperationContextImpl.waitForRemovals(OperationContextImpl.java:520)
cello-initial-keycloak-server | at org.jboss.as.controller.AbstractOperationContext$Step.handleResult(AbstractOperationContext.java:1518)
cello-initial-keycloak-server | at org.jboss.as.controller.AbstractOperationContext$Step.finalizeInternal(AbstractOperationContext.java:1472)
cello-initial-keycloak-server | at org.jboss.as.controller.AbstractOperationContext$Step.finali
Any suggestion on above issue would be great help .. i see recently few are facing same issue while starting cello.
Has joined the channel.
Has joined the channel.
Hi , I have some questions want to get some help. Can I discuss it?
I get some error info : “unable to read root page from file:sizeStorer.wt: WT_ERROR: non-specific WiredTiger error”
Clipboard - December 11, 2018 4:34 PM
Not certain that whether Master node and Worker node can be deployed in one machine? Please help, Thanks
[ ](https://chat.hyperledger.org/channel/cello?msg=xhtK2HiNPFzuhkymL) @yinzhiwu They could, but we highly recommend not doing so because of the resilience considerations.
@baohua how share crypto-config material among organization in the cello?
[ ](https://chat.hyperledger.org/channel/cello?msg=5GCnMZAAdTFztbfwM) @asaningmaxchain123 the old way is via NFS and now we are changing to new object storage to improve the performance.
in my mind, each organization should have an cell instance ,so if the organization one want to make channel with other organization, how should the organization one do
`new object storage ` ,can you provide more detail?
`new object storage ` ,can you provide more detail? @baohua
[ ](https://chat.hyperledger.org/channel/cello?msg=FMGvECA98o8NKnwrx) @asaningmaxchain123 considering this: https://www.minio.io/
CELLO: V0.9.0
After the command running of "SERVER_PUBLIC_IP=0.0.0.0 make start",
the Container “cello-user-dashboard” cannot start, please see the errors below. How to fix it. Thanks a lot.
------------------------------------------------------------------------------------------------------------
2018-12-12 10:30:10,767 ERROR 178 [master] app_worker#1:194 start fail, exiting with code:1
2018-12-12 10:30:10,767 ERROR 178 [master] exit with code:1
2018-12-12 10:30:10,770 ERROR 184 [agent_worker] receive disconnect event on child_process fork mode, exiting with code:110
2018-12-12 10:30:10,774 INFO 184 [agent_worker] beforeExit success
2018-12-12 10:30:10,774 ERROR 184 [agent_worker] exit with code:110
⚠️ Error: /node_modules/egg-bin/lib/start-cluster {"stricky":true,"port":8081,"baseDir":"/var/www","workers":1,"framework":"/var/www/node_modules/egg"} exit with code 1
⚠️ Command Error, enable `DEBUG=common-bin` for detail
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! user-dashboard@1.0.0 start:dev: `egg-bin dev --stricky --port 8081`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the user-dashboard@1.0.0 start:dev script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-12-12T10_30_10_786Z-debug.log
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! user-dashboard@1.0.0 start: `if-env NODE_ENV=production && npm run start:prod || npm run start:dev`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the user-dashboard@1.0.0 start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-12-12T10_30_10_798Z-debug.log
basically, we need some shared storage
SERVER_PUBLIC_IP=0.0.0.0
[ ](https://chat.hyperledger.org/channel/cello?msg=jJ6pouFodgiNdcPym) @baohua Thanks. I See
this should point to the master service
not 0.0.0.0
please follow https://cello.readthedocs.io/en/latest/tutorial/
[ ](https://chat.hyperledger.org/channel/cello?msg=hR8hjWRWWKyhDaR5s) @baohua
Clipboard - December 12, 2018 6:48 PM
Could some person show your cello related image version snapshot? Thanks
Followed following below showns steps still keycloak-server docker container getting failed and cello dashboard shows relam error.
git clone http://gerrit.hyperledger.org/r/cello && cd cello
make setup-master
SERVER_PUBLIC_IP=xx.xx.xx.xx make start
docker logs cello-keycloak-server
18:36:25,946 INFO [org.keycloak.connections.jpa.updater.liquibase.LiquibaseJpaUpdaterProvider] (ServerService Thread Pool -- 62) Updating database. Using changelog META-INF/jpa-changelog-master.xml
18:36:25,994 ERROR [org.keycloak.connections.jpa.updater.liquibase.conn.DefaultLiquibaseConnectionProvider] (ServerService Thread Pool -- 62) Change Set META-INF/jpa-changelog-2.1.0.xml::2.1.0-KEYCLOAK-5461::bburke@redhat.com failed. Error: Table 'BROKER_LINK' already exists [Failed SQL: CREATE TABLE keycloak.BROKER_LINK (IDENTITY_PROVIDER VARCHAR(255) NOT NULL, STORAGE_PROVIDER_ID VARCHAR(255) NULL, REALM_ID VARCHAR(36) NOT NULL, BROKER_USER_ID VARCHAR(255) NULL, BROKER_USERNAME VARCHAR(255) NULL, TOKEN TEXT NULL, USER_ID VARCHAR(255) NOT NULL)]
18:36:26,005 INFO [org.jboss.as.server] (Thread-2) WFLYSRV0220: Server shutdown has been requested via an OS signal
18:36:26,015 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 62) MSC000001: Failed to start service jboss.undertow.deployment.default-server.default-host./auth: org.jboss.msc.service.StartException in service jboss.undertow.deployment.default-server.default-host./auth: java.lang.RuntimeException: RESTEASY003325: Failed to construct public org.keycloak.services.resources.KeycloakApplication(javax.servlet.ServletContext,org.jboss.resteasy.core.Dispatcher)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:81)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
Caused by: liquibase.exception.MigrationFailedException: Migration failed for change set META-INF/jpa-changelog-2.1.0.xml::2.1.0-KEYCLOAK-5461::bburke@redhat.com:
Reason: liquibase.exception.DatabaseException: Table 'BROKER_LINK' already exists [Failed SQL: CREATE TABLE keycloak.BROKER_LINK (IDENTITY_PROVIDER VARCHAR(255) NOT NULL, STORAGE_PROVIDER_ID VARCHAR(255) NULL, REALM_ID VARCHAR(36) NOT NULL, BROKER_USER_ID VARCHAR(255) NULL, BROKER_USERNAME VARCHAR(255) NULL, TOKEN TEXT NULL, USER_ID VARCHAR(255) NOT NULL)]
at liquibase.changelog.ChangeSet.execute(ChangeSet.java:619)
at liquibase.changelog.visitor.UpdateVisitor.visit(UpdateVisitor.java:51)
at liquibase.changelog.ChangeLogIterator.run(ChangeLogIterator.java:79)
at liquibase.Liquibase.update(Liquibase.java:214)
at liquibase.Liquibase.update(Liquibase.java:192)
at liquibase.Liquibase.update(Liquibase.java:188)
at org.keycloak.connections.jpa.updater.liquibase.LiquibaseJpaUpdaterProvider.updateChangeSet(LiquibaseJpaUpdaterProvider.java:182)
at org.keycloak.connections.jpa.updater.liquibase.LiquibaseJpaUpdaterProvider.update(LiquibaseJpaUpdaterProvider.java:102)
... 57 more
Caused by: liquibase.exception.DatabaseException: Table 'BROKER_LINK' already exists [Failed SQL: CREATE TABLE keycloak.BROKER_LINK (IDENTITY_PROVIDER VARCHAR(255) NOT NULL, STORAGE_PROVIDER_ID VARCHAR(255) NULL, REALM_ID VARCHAR(36) NOT NULL, BROKER_USER_ID VARCHAR(255) NULL, BROKER_USERNAME VARCHAR(255) NULL, TOKEN TEXT NULL, USER_ID VARCHAR(255) NOT NULL)]
Followed following below shown steps still keycloak-server container getting failed and cello dashboard shows relam error.
git clone http://gerrit.hyperledger.org/r/cello && cd cello
make setup-master
SERVER_PUBLIC_IP=xx.xx.xx.xx make start
docker logs cello-keycloak-server
18:36:25,946 INFO [org.keycloak.connections.jpa.updater.liquibase.LiquibaseJpaUpdaterProvider] (ServerService Thread Pool -- 62) Updating database. Using changelog META-INF/jpa-changelog-master.xml
18:36:25,994 ERROR [org.keycloak.connections.jpa.updater.liquibase.conn.DefaultLiquibaseConnectionProvider] (ServerService Thread Pool -- 62) Change Set META-INF/jpa-changelog-2.1.0.xml::2.1.0-KEYCLOAK-5461::bburke@redhat.com failed. Error: Table 'BROKER_LINK' already exists [Failed SQL: CREATE TABLE keycloak.BROKER_LINK (IDENTITY_PROVIDER VARCHAR(255) NOT NULL, STORAGE_PROVIDER_ID VARCHAR(255) NULL, REALM_ID VARCHAR(36) NOT NULL, BROKER_USER_ID VARCHAR(255) NULL, BROKER_USERNAME VARCHAR(255) NULL, TOKEN TEXT NULL, USER_ID VARCHAR(255) NOT NULL)]
18:36:26,005 INFO [org.jboss.as.server] (Thread-2) WFLYSRV0220: Server shutdown has been requested via an OS signal
18:36:26,015 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 62) MSC000001: Failed to start service jboss.undertow.deployment.default-server.default-host./auth: org.jboss.msc.service.StartException in service jboss.undertow.deployment.default-server.default-host./auth: java.lang.RuntimeException: RESTEASY003325: Failed to construct public org.keycloak.services.resources.KeycloakApplication(javax.servlet.ServletContext,org.jboss.resteasy.core.Dispatcher)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:81)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
Caused by: liquibase.exception.MigrationFailedException: Migration failed for change set META-INF/jpa-changelog-2.1.0.xml::2.1.0-KEYCLOAK-5461::bburke@redhat.com:
Reason: liquibase.exception.DatabaseException: Table 'BROKER_LINK' already exists [Failed SQL: CREATE TABLE keycloak.BROKER_LINK (IDENTITY_PROVIDER VARCHAR(255) NOT NULL, STORAGE_PROVIDER_ID VARCHAR(255) NULL, REALM_ID VARCHAR(36) NOT NULL, BROKER_USER_ID VARCHAR(255) NULL, BROKER_USERNAME VARCHAR(255) NULL, TOKEN TEXT NULL, USER_ID VARCHAR(255) NOT NULL)]
at liquibase.changelog.ChangeSet.execute(ChangeSet.java:619)
at liquibase.changelog.visitor.UpdateVisitor.visit(UpdateVisitor.java:51)
at liquibase.changelog.ChangeLogIterator.run(ChangeLogIterator.java:79)
at liquibase.Liquibase.update(Liquibase.java:214)
at liquibase.Liquibase.update(Liquibase.java:192)
at liquibase.Liquibase.update(Liquibase.java:188)
at org.keycloak.connections.jpa.updater.liquibase.LiquibaseJpaUpdaterProvider.updateChangeSet(LiquibaseJpaUpdaterProvider.java:182)
at org.keycloak.connections.jpa.updater.liquibase.LiquibaseJpaUpdaterProvider.update(LiquibaseJpaUpdaterProvider.java:102)
... 57 more
Caused by: liquibase.exception.DatabaseException: Table 'BROKER_LINK' already exists [Failed SQL: CREATE TABLE keycloak.BROKER_LINK (IDENTITY_PROVIDER VARCHAR(255) NOT NULL, STORAGE_PROVIDER_ID VARCHAR(255) NULL, REALM_ID VARCHAR(36) NOT NULL, BROKER_USER_ID VARCHAR(255) NULL, BROKER_USERNAME VARCHAR(255) NULL, TOKEN TEXT NULL, USER_ID VARCHAR(255) NOT NULL)]
@baohua @Luke_Chen @hightall can you please look at this patch to fix an issue? https://gerrit.hyperledger.org/r/#/c/28024/ we need to include all the peers in the connection profile for gosdk to work appropriately.
[ ](https://chat.hyperledger.org/channel/cello?msg=bYbaDS2jXzfwbbHMQ) @AlphaAsteroid BNA is deployed by composer
Has left the channel.
Has joined the channel.
HI All,
I have deployed the Hyperledger fabric on AWS using cello ansible with dockers. Everything works fine. I was able to do a transaction using composer playground. In a process for vertical scale-up the system, I have created a bigger AWS instance using old AMI. I have correct all the dns, docker and flannel settings. Updated /etc/hosts files on both the VMs. Brought all the docker container up and running. Now when I try to do a transaction, I am getting below error by composer-playground:
Error: Error trying invoke business network. Error: Failed to send peer responses for transaction '02be502e532dfe5c153fa2fc5ecbb599a387834e32f4eb5b1806949335cfcd26' to orderer. Response status 'SERVICE_UNAVAILABLE
I have even tried crearting participant user composer CLI but still getting same error.
I have checked all the docker logs(i.e. orderer, peer, kafka, zookeeper) could not able to find exact error.
Can anyone please tell me what I am missing here?
Thanks.
Also after a day, orderer dockers shuts down with below error:
2018-12-12 00:33:33.799 UTC [orderer/consensus/kafka] startThread -> CRIT 1a3 [channel: orderersystemchannel] Cannot set up channel consumer = kafka server: The requested offset is outside the range of offsets maintained by the server for the given topic/partition.
panic: [channel: orderersystemchannel] Cannot set up channel consumer = kafka server: The requested offset is outside the range of offsets maintained by the server for the given topic/partition.
goroutine 30 [running]:
github.com/hyperledger/fabric/vendor/github.com/op/go-logging.(*Logger).Panicf(0xc42024e0f0, 0xd1d562, 0x31, 0xc420114540, 0x2, 0x2)
/opt/gopath/src/github.com/hyperledger/fabric/vendor/github.com/op/go-logging/logger.go:194 +0x134
github.com/hyperledger/fabric/orderer/consensus/kafka.startThread(0xc42015f8c0)
/opt/gopath/src/github.com/hyperledger/fabric/orderer/consensus/kafka/chain.go:261 +0xb33
created by github.com/hyperledger/fabric/orderer/consensus/kafka.(*chainImpl).Start
Has joined the channel.
Has joined the channel.
Has joined the channel.
```Hello, I have set up a cello master node on a GCE instance using the docs at https://cello.readthedocs.io/en/latest/setup_master/#run-setup
I start the instance with "SERVER_PUBLIC_IP=X.X.X.XCE THEME=vue make start" . When I visit X.X.X.X:8080 I'm redirected to X.X.X.X:8083 (keycloak) and I'm greeted with the login screen. Using admin:pass to log in, the server does not give any response and the request times out in the browser. Any ideas?
```
Has joined the channel.
For temporary purpose you can run follow below steps to fix this issue ..
This issue is happening because of SSL which is enabled by default and we are hitting HTTP URL which redirects and throws error message.
Resolution After installation of Cello we need to disable SSL to Make it work on HTTP URL:
Login to cello-keycloak-mysql container :
docker exec -it cello-keycloak-mysql bash
use keycloak;
update REALM set ssl_required = 'NONE' where id = 'master';
[ ](https://chat.hyperledger.org/channel/cello?msg=JSxtER2jrxYSk46W2) @sivak2018 Thanks sivak2018! @hightall is this correct?
Has anyone tried to setup a network with Cello on Azure? I am using principal method for authentication but it throws an Error "No valid tennant ID". I am able to access my azure account with the specified settings using powershell. Not sure why it's not working with cello.
Just found out that the default configuration just works with username and password. The Cello scripts for Azure tries to read and set username and password. It never consider any other authorization method. /cello/src/operator-dashboard/agent/ansible/roles/cloud_azure/prepare/tasks/apply.yml `- name: Setup env specific variables
set_fact:
azure_ad_user: "{{ auth.username }}"
azure_ad_pwd: "{{ auth.password }}"
azure_sub_id: "{{ auth.subscription_id }}"
region: "{{ cluster.region_name | default('eastus') }}"
res_grp: "{{ cluster.resource_group }}"`
Just found out that the default configuration just works with username and password. The Cello scripts for Azure tries to read and set username and password. It never considers any other authorization method. /cello/src/operator-dashboard/agent/ansible/roles/cloud_azure/prepare/tasks/apply.yml `- name: Setup env specific variables
set_fact:
azure_ad_user: "{{ auth.username }}"
azure_ad_pwd: "{{ auth.password }}"
azure_sub_id: "{{ auth.subscription_id }}"
region: "{{ cluster.region_name | default('eastus') }}"
res_grp: "{{ cluster.resource_group }}"`
Has joined the channel.
Has joined the channel.
Hey all. I am trying to setup hyperledger cello operator dashboard on a AWS EC2 instance. After both ` make setup-master ` and ` SERVER_PUBLIC_IP=
Screenshot 2018-12-17 at 3.51.36 PM.png
Using git tag ` v0.9.0 `
Has joined the channel.
```
@baohua After setting up and using SERVER_PUBLIC_IP=X.X.X.X THEME=react make start. i attempt to connect to X.X.X.X which redirects me to X.X.X.X:8083/auth which then gives me an https required error
```
@baohua After setting up and using SERVER_PUBLIC_IP=X.X.X.X THEME=react make start. i attempt to connect to X.X.X.X which redirects me to X.X.X.X:8083/auth which then gives me an https required error
```
@baohua After setting up and using SERVER_PUBLIC_IP=X.X.X.X THEME=react make start. i attempt to connect to X.X.X.X which redirects me to X.X.X.X:8083/auth which then gives me an https required error
Has left the channel.
Has joined the channel.
@mkhraisha do you fix it?
Has joined the channel.
@hightall can you take a look?
Clipboard - December 20, 2018 3:02 PM
i notice the `redirect_uri=` is localhost:8080
not the ip:8080
and i go into the `cello-operator-dashboard` i find the client_secrets.json has been replaced with real env var
Clipboard - December 20, 2018 3:04 PM
however
however when i set the` redirect_uri=ip:8080`
however when i set the `redirect_uri=ip:8080` it doesn't work
@hightall
Clipboard - December 20, 2018 3:26 PM
hi all,who can help me?
Clipboard - December 20, 2018 6:35 PM
Cannot extend existing Fabric deployed using Cello Ansible. Cryptogen is failing because of Cello's use of 'mv' command to move signature (_sk) files to a new name. I think these files should be copied instead of moved to the new name. This is leading to loss of the info that is contained in the original filename. Is there a workaround? @tongli @liuruifei @asaningmaxchain123
[ ](https://chat.hyperledger.org/channel/cello?msg=GbHttXdPYSwWGgB87) @Gaurang in the prod mode,the crypto-config should be dynamic generate by the fabric-ca service
@baohua today the meeting will open at 8:30 pm?
yes, as regular!
@asaningmaxchain123
@baohua ok,i will join
@asaningmaxchain123 Thanks for the help. Is there any documentation where I can read more about this. Hyperledger fabric documents are very scattered. Could not able to find a resource where we could find steps to deploy a production system.
https://github.com/hyperledger/fabric-samples/tree/release-1.3/fabric-ca
@Gaurang take a look,
@baohua do you start?
@baohua the meeting is going?
@asaningmaxchain123 thanks for the link :)
Hi All,
When cello ansible will support fabric 1.3 deployement ?
[ ](https://chat.hyperledger.org/channel/cello?msg=QvGg2DDCNBpMbKt3R) We are waiting for this functionality as well
Cello ansible agent support all the fabric version starting 1.0, you just need to change the configuration to specify your desired version.
Due to the holidays, the cello meeting will cancel this week. Happy holidays to every developers! @all
Has joined the channel.
Hi I am following this tutorial https://github.com/hyperledger/cello/blob/master/docs/setup_master.md . After I run the command:
SERVER_PUBLIC_IP=192.168.100.193 make start
i went to the operator dashboard at localhost:8080. However I get the error:
Invalid parameter: redirect_uri
Can anybody help me with it?
Screenshot from 2018-12-25 10-43-11.png
[ ](https://chat.hyperledger.org/channel/cello?msg=EU4wh09qs9sU94S9BM) @tongli thanks @tongli . I will try deploying fabric 1.3.
[ ](https://chat.hyperledger.org/channel/cello?msg=EU4wh09qs9sU94S9BM) @tongli thanks . I will try deploying fabric 1.3.
Has joined the channel.
I am running hyperledger cello trying to fabric 1.2...getting this error from fabric-ca container
Error: Failed to find private key for certificate in '/etc/hyperledger/fabric-ca-server-config/ca.org2.example.com-cert.pem': Could not find matching private key for SKI: Failed getting key for SKI [[245 124 59 164 162 172 35 149 31 177 235 12 13 85 185 180 83 185 164 107 186 232 124 10 7 213 91 76 41 146 233 92]]: Key with SKI f57c3ba4a2ac23951fb1eb0c0d55b9b453b9a46bbae87c0a07d55b4c2992e95c not found in /etc/hyperledger/fabric-ca-server/msp/keystore
I am running hyperledger cello trying to run fabric 1.2...getting this error from fabric-ca container
Error: Failed to find private key for certificate in '/etc/hyperledger/fabric-ca-server-config/ca.org2.example.com-cert.pem': Could not find matching private key for SKI: Failed getting key for SKI [[245 124 59 164 162 172 35 149 31 177 235 12 13 85 185 180 83 185 164 107 186 232 124 10 7 213 91 76 41 146 233 92]]: Key with SKI f57c3ba4a2ac23951fb1eb0c0d55b9b453b9a46bbae87c0a07d55b4c2992e95c not found in /etc/hyperledger/fabric-ca-server/msp/keystore
Has joined the channel.
hey guys i have single blockchain whihc has 2 networks running and in one network there are 3 peers and in second there are 4 peers and the networks have 1 anchor peer and overall there in only one channel. Can this setup work in cello? if yes how? and if not how?
which*
[ ](https://chat.hyperledger.org/channel/cello?msg=HEEKnG4AwzCDD9NEa) @sam3313 maybe our tutorial can give you some first experience with cello :) https://cello.readthedocs.io/en/latest/tutorial/
Has joined the channel.
Hello All,
Hello All,
Am trying to set cello for deploying Hyperledger fabric over Docker Swarm
Am running "make setup-master" and when I try to hit the URL X.X.X.X:8080 it gets redirected to
http://X.X.X.X:8083/auth/realms/cello-realm/protocol/openid-connect/auth?client_id=cello-operator-dashboard&redirect_uri
and this site gets unreachable
Has joined the channel.
found below Jira entry for this but is there any solution to fix it
https://jira.hyperledger.org/browse/CE-523?jql=project%20%3D%20CE%20AND%20component%20%3D%20Cello-Dashboard
[ ](https://chat.hyperledger.org/channel/cello?msg=JSxtER2jrxYSk46W2) @sivak2018 where do we set the ssl_required variable after logging to the container
hi @tongli , Just trying to install fabric 1.3 with cello ansible. I am able to bring up the fabric but not able to start the BNA with the help of composer cli. It seems composer cli commads have been changed in the newer version. I even tried starting BNA with composer cli 20.5 but getting below error:
Error trying to start business network. Error: Failed to connect to any peer event hubs. It is required that at least 1 event hub has been connected to receive the commit event
Command failed
Has left the channel.
Has joined the channel.
Hello! Just found out about this chat. I am currently trying to deploy Fabric with a custom chaincode onto AWS using the Ansible Agent in the Cello project. While I can get the network up and install the default chaincode onto the peers, I cannot switch the default with our custom chaincode that uses the Go extensions in the vendor folder (I think it uses fabric v1.3). Does anyone have any sources for deploying Fabric 1.3 onto AWS with a custom chaincode using Go extensions?
Hello! Just found out about this chat. I am currently trying to deploy Fabric with a custom chaincode onto AWS using the Ansible Agent in Cello. While I can get the network up and install the default chaincode onto the peers, and run test queries successfully, I cannot switch the default chaincode with our custom chaincode that uses the Go extensions in the vendor folder (I think it uses fabric v1.3). Does anyone have any sources for deploying Fabric 1.3 onto AWS with a custom chaincode using Go extensions?
Hello! Just found out about this chat. I am currently trying to deploy Fabric with a custom chaincode onto AWS using the Ansible Agent in Cello. While I can get the network up and install the default chaincode onto the peers, and run test queries successfully, I cannot switch the default chaincode with our custom chaincode that uses the Go extensions in the vendor folder (I think it uses fabric v1.3). Does anyone have any sources for deploying Fabric 1.3 onto AWS with a custom chaincode that uses Go extensions?
Has joined the channel.
when I login the user dashboard, it return 500 code. error message :`cello-keycloak-server | 09:05:37,573 WARN [org.keycloak.events] (default task-6) type=CODE_TO_TOKEN_ERROR, realmId=4567ef6d-bbaa-41da-8801-d22166990765, clientId=cello-user-dashboard, userId=null, ipAddress=39.96.78.20, error=invalid_code, grant_type=authorization_code, code_id=44f0130c-285a-41d0-8e9b-1e790fbcd500, client_auth_method=client-secret`
Has joined the channel.
hi....Does cello supports fabric deployment on GKE ?? and Do we have to setup manually network configuration files of fabric to be deployed on GKE ??
Has joined the channel.
Has joined the channel.
I would appreciate your assistance in getting these changes merged. Changes to repos without CI setup will need V+1 manually set. Thank you.
https://gerrit.hyperledger.org/r/#/c/28403/
https://gerrit.hyperledger.org/r/#/c/25641/
https://gerrit.hyperledger.org/r/#/c/28408/
https://gerrit.hyperledger.org/r/#/c/28407/
Has joined the channel.
@baohua @hightall @Luke_Chen Please take a look at this patch set which is to fix a space issue in the connection profile generated by ansible agent. https://gerrit.hyperledger.org/r/#/c/28452/
Thanks very much!
+2!
Thanks @baohua
More patches to improve the agent.
https://gerrit.hyperledger.org/r/28459 [CE-536]ansible agent should use persistent storage
Please take a look at this one. I am also working on a patch to use fabric 1.4 operator features when sets up fabric.
@baohua @Luke_Chen @hightall please review this patch. https://gerrit.hyperledger.org/r/28459 [CE-536]ansible agent should use persistent storage
Has joined the channel.
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=3fs8fEMEEy4TWD52p) @tongli +2!
Has joined the channel.
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/28476/
@yinzhiwu , did you fix the problem which the Container “cello-user-dashboard” cannot start
@tongli would you like to update this or we can abandon? https://gerrit.hyperledger.org/r/#/c/21615/
https://gerrit.hyperledger.org/r/#/c/24373/
Has joined the channel.
@baohua , I have meet some problems when start cello, the user-dashboard container exited automatically ```
``````
These logs as following, look forward your reply, thanks!
```
-----------------------------------------------------------------
``` ~/cello$ make logs
docker-compose -f bootup/docker-compose-files/docker-compose.yml logs -f --tail=200
WARNING: Some services (mongo) use the 'deploy' key, which will be ignored. Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm.
Attaching to cello-user-dashboard, cello-operator-dashboard, cello-watchdog, cello-keycloak-server, cello-parse-server, cello-keycloak-mysql, cello-mongo, cello-dashboard_rabbitmq, cello-engine
cello-user-dashboard | at AppWorkerLoader.loadCustomApp (/node_modules/egg-core/lib/loader/mixin/custom.js:37:20)
cello-user-dashboard | at AppWorkerLoader.load (/node_modules/egg/lib/loader/app_worker_loader.js:33:10)
cello-user-dashboard | at new Application (/node_modules/egg/lib/application.js:66:19)
cello-user-dashboard | at Object.
Has left the channel.
Has joined the channel.
hi there. are hyperledger cello and hyperledger composer interoperable? does it make sense to use cello to deploy a fabric network and use hyperledger composer to model the business logic/define assets etc.?
[ ](https://chat.hyperledger.org/channel/cello?msg=EWWtqAtrGzojAgn8D) @gnostikos Not yet :)
Has joined the channel.
hi. I'm trying to run cello now. Should cello-realm be made automatically when I run "make start"? In my keycloak-myql, it seems that there is only master realm, so when I request user-dashboard or admin-dashboard, "404 not found" error occurs. What should I check additionally?
[ ](https://chat.hyperledger.org/channel/cello?msg=8YtFMo66DqX2sabJd) I solved my problem as executing "KEYCLOAK_REAM=cello-realm make initial-keycloak". Thanks.
Has joined the channel.
@tongli could I get this merged? thanks. https://gerrit.hyperledger.org/r/#/c/28408/
@rjones it is now.
thx
Thank you!
Has joined the channel.
Has joined the channel.
The min requirements listed for a Master and Worker is given as
Master : 8cores, 16G RAM, 100G disk
Worker : 4 cores , 8G RAM, 100G disk
The min requirements listed for a Cello Master and Worker is given as
Master : 8cores, 16G RAM, 100G disk
Worker : 4 cores , 8G RAM, 100G disk
@baohua The min requirements listed for a Cello Master and Worker is given as
Master : 8cores, 16G RAM, 100G disk
Worker : 4 cores , 8G RAM, 100G disk
Would a dev setup with 4 cores, 16 GRAM, 100G disk (for master ) and 2 cores 8g RAM, 100G disk (for worker) work ?
or will the resource requirements have to be tweaked in the docker compose files for it to work with the lesser hardware
another question: does Cello have any notion of Channels (i.e channels in hyperledger fabric) does it have the capability to create and manage channels ?
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=posJrMdGpMrRnQkao) @clydedacruz Lower resource may introduce risks of boot-up, but maybe work, too
[ ](https://chat.hyperledger.org/channel/cello?msg=4DvP7FrnAHuE7FaxZ) @clydedacruz Yes, using the user-dashboard.
@baohua After I "make setup-master" and "make start" , I try to hit the URL X.X.X.X:8080 and it gets redirected to http://X.X.X.X:8083/auth/realms/cello-realm/protocol/openid-connect/auth?client_id=cello-operator-dashboard&redirect_uri and this site gets unreachable.I have saw many problems similar to me before, but I could not find an effective solution.I'd appreciate if you could give me some advice.
I tried 2 things
@baohua @tongli @hightall I tried the setup using both the master branch and the v0.9.0 .In both cases the user-dashboard does not work.
1. using the "master" branch of the project, completed the master node installation using `make setup-master` followed by `make start`
Installation completes successfully. No indication of error.
This brings up all the containers. I'm able to access the operator dashboard. But when I go to the user dashboard, it redirects to the the authentication page, and after i enter the creds, it goes back to
@baohua @tongli @hightall I tried the setup using both the master branch and the v0.9.0 .In both cases the user-dashboard does not work.
1. using the "master" branch of the project, completed the master node installation using `make setup-master` followed by `make start`
Installation completes successfully. No indication of error.
This brings up all the containers. I'm able to access the operator dashboard. But when I go to the user dashboard, it redirects to the the authentication page, and after i enter the creds, it goes back to
@baohua @tongli @hightall I tried the setup using both the master branch and the v0.9.0 .In both cases the user-dashboard does not work.
1. using the "master" branch of the project, completed the master node installation using `make setup-master` followed by `make start`
Installation completes successfully. No indication of error.
This brings up all the containers. I'm able to access the operator dashboard. But when I go to the user dashboard, it redirects to the the authentication page, and after i enter the creds, it goes back to
@baohua @tongli @hightall I tried the setup using both the master branch and the v0.9.0 .In both cases the user-dashboard does not work.
1. using the "master" branch of the project, completed the master node installation using `make setup-master` followed by `make start`
Installation completes successfully. No indication of error.
This brings up all the containers. I'm able to access the operator dashboard. But when I go to the user dashboard, it redirects to the the authentication page, and after i enter the creds, it goes back to
@baohua @tongli @hightall I tried the setup using both the master branch and the v0.9.0 .In both cases the user-dashboard does not work.
1. using the "master" branch of the project, completed the master node installation using `make setup-master` followed by `make start`
Installation completes successfully. No indication of error.
This brings up all the containers. I'm able to access the operator dashboard. But when I go to the user dashboard, it redirects to the the authentication page, and after i enter the creds, it goes back to
@baohua @tongli @hightall I tried the setup using both the master branch and the v0.9.0 .In both cases the user-dashboard does not work.
1. using the "master" branch of the project, completed the master node installation using `make setup-master` followed by `make start`
Installation completes successfully. No indication of error.
This brings up all the containers. I'm able to access the operator dashboard. But when I go to the user dashboard, it redirects to the the authentication page, and after i enter the creds, it goes back to
I has the same problem . I find their docker-hub is still being updated with tags v0.9.0. I found a historical mirror ,fixed it.
user-dashboard-logs-v0.9.0.txt
@baohua Which branch of the cello repository is the stable branch at this moment ?
v0.9.0 is the stable branch. the master branch is the latest code targeting v1.0.0
what docker images works with the 0.9.0 code ? x86_64-v0.9.0 ?
@ baohua what docker images works with the 0.9.0 code ? I can find a v0.9.0 tag for docker images
@ baohua what docker images work with the 0.9.0 code ? I can find a v0.9.0 tag for docker images. Any comments about the error log I posted above ( I was seeing this when using the 0.9.0 branch)
[ ](https://chat.hyperledger.org/channel/cello?msg=pyMY5CuaNcuW2GYoS) @clydedacruz hmm, you can build the image locally, see the dockerhub folder
Hello everyone. For docker host, I setup docker daemon and I made fabric cluster successfully by fabric cello. For docker swarm host, what should I do? In your tutorial, you just link to document for setup docker swarm. And, cello/scripts/worker_node/setup_worker_node_swarm.sh seems not to do something special for making daemon and etc. I wonder what should I do additionally and what is proper value to SWARM type Daemon Url in Create New Host menu.
[ ](https://chat.hyperledger.org/channel/cello?msg=d2tPfsaGS9KzWHfiF) @baohua Thanks for the reply. I did a `make docker` which built all the images and tagged them as `latest` but when I do `make setup-master` it tries to look for images that are x86_64-latest
[ ](https://chat.hyperledger.org/channel/cello?msg=d2tPfsaGS9KzWHfiF) @baohua Thanks for the reply. I did a `make docker` which built all the images and tagged them as `latest` but when I do `make setup-master` it tries to look for images that are x86_64-latest hence it downloads images from docker hub again
[ ](https://chat.hyperledger.org/channel/cello?msg=6WLDtw6XYJWuqbb8v) @clydedacruz hmm, you can tag the images with latest tag as x86_64-latest.
Hello, everyone. I’m a newby in hyperledger so I have a question. Which is the proper method for a question about hyperledger cello, this chat or jira? In other oss solution, I used github Issues board usually.
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=LgfKnB5Kdco5pwHiC) @baohua The user dashboard comes up with the built 0.9.0 images by the operator dashboard is now broken :)
[ ](https://chat.hyperledger.org/channel/cello?msg=LgfKnB5Kdco5pwHiC) @baohua The user dashboard comes up with the built 0.9.0 images but the operator dashboard is now broken :)
[ ](https://chat.hyperledger.org/channel/cello?msg=LgfKnB5Kdco5pwHiC) @baohua The user dashboard comes up with the built 0.9.0 images but the operator dashboard is now broken :confused:
cello_master_V0.9.0_logs.txt
Has joined the channel.
@Luke_Chen @baohua @hightall please take a look at this patch that suryan has submitted, it is to enable the peer and orderer metrics if 1.4 fabric is used. https://gerrit.hyperledger.org/r/#/c/28840/
very useful feature.
please take a look. thanks.
+2!
Has joined the channel.
https://gerrit.hyperledger.org/r/#/c/28840/
@baohua @hightall @Luke_Chen Needed to make a small change. it is good now. Please review it. thanks.
+2!
Thanks @baohua
There is one other patch which allows people more control how peers and orderers can be setup using various parameters when setup network. Please take a look. https://gerrit.hyperledger.org/r/#/c/28897/
@baohua @hightall @Luke_Chen That patch is a different one by Surya
he will provide one more to convert fabric component from pod to steatefulset.
which is a good improvement.
Thanks for the nice improvement, approved!
Clipboard - January 23, 2019 2:34 PM
@baohua when I try to create a chain on the user-dashboard, I get a 400(bad request). I can see teh following logs in the containers (using `make logs`)
[ ](https://chat.hyperledger.org/channel/cello?msg=rkyfFcLxfnYkmQFPn) @baohua @hightall any ideas about the above problem ? I'm using the 0.9.0 tag of cello
@baohua @hightall @Luke_Chen please review this one, we need to make a change to make sure that the cello produced prometheus configuration file work as is without further processing. here is the patch. https://gerrit.hyperledger.org/r/#/c/28932/ thanks.
Has left the channel.
Has joined the channel.
Is it possible to build the Cello docker images from local code? At present, running 'make start' downloads the code from Github and then builds it. @baohua @hightall @tongli
Is it possible to build the Cello docker images from local code? At present, running 'make docker' downloads the code from Github and then builds it. @baohua @hightall @tongli
Has left the channel.
Has left the channel.
Has joined the channel.
@eric_mesquita yes, you can.
Hyperledger wiki (wiki.hyperledger.org) has done the migration to confluence (much faster), old content can be found at https://wiki-archive.hyperledger.org.
[ ](https://chat.hyperledger.org/channel/cello?msg=ChM4EKLWrSouCM9sD) @tongli Is there a make command to do this?
Has joined the channel.
When starting Hyperledger cello as master node, I have to set the SERVER_PUBLIC_IP to docker0 ip address e.g. 172.17.0.1 so that I can open operator dashboard and the user dashboard. How can I set the SERVER_PUBLIC_IP to eth0 IP address instead of using docker0 IP address.
@tongli When starting Hyperledger cello as master node, I have to set the SERVER_PUBLIC_IP to docker0 ip address e.g. 172.17.0.1 so that I can open operator dashboard and the user dashboard. How can I set the SERVER_PUBLIC_IP to eth0 IP address instead of using docker0 IP address.
@baohua When starting Hyperledger cello as master node, I have to set the SERVER_PUBLIC_IP to docker0 ip address e.g. 172.17.0.1 so that I can open operator dashboard and the user dashboard. How can I set the SERVER_PUBLIC_IP to eth0 IP address instead of using docker0 IP address.
Has joined the channel.
hey guys, were can I see the roadmap for cello? thanks :)
Has left the channel.
@netanyahuyasser I am not familiar with that part, please ask @baohua
From the makefile, it shows to use `make dockerhub`.
Has joined the channel.
Has joined the channel.
hello, i am trying to run cello from master and getting the following error
`ERROR: for cello-keycloak-server Cannot start service keycloak: oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:364: container init caused \"rootfs_linux.go:54: mounting \\\"/thirdparty/keycloak/resources/img/keycloak-logo-text.png\\\" to rootfs \\\"/var/lib/docker/overlay2/817a7917a2535e3d652a9bbae0b2b02e3bcbe0b019ba7c5e47d39d60decca185/merged\\\" at \\\"/var/lib/docker/overlay2`
i see a reference on message board to same error in early jan but it seems not fixed and I don't seem to be able to fix it. I see the files available in the path specified
@Bentipe I am trying to find the same. Cello is a great initiative from the community. It would be great if someone can share a road map for it. I am sure we all are waiting for it's release 1.0 (which can be used for production).
@baohua @Luke_Chen @hightall Can you take a quick look at this https://gerrit.hyperledger.org/r/#/c/29191/?
The build issue finally gone, really need this to be merged.
Thanks.
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=hnk5NaSAwnJjDfj5e) Hi...I am getting the same error when I execute this command: `sudo MODE=dev SERVER_PUBLIC_IP=0.0.0.0 make start` @tongli @baohua Could you please take a look at this and suggest a fix.
Hi...I am getting the same error when I execute this command: `sudo MODE=dev SERVER_PUBLIC_IP=0.0.0.0 make start` @tongli @baohua Could you please take a look at this and suggest a fix. https://chat.hyperledger.org/channel/cello?msg=hnk5NaSAwnJjDfj5e
Hi...I am getting the same error when I execute this command: `sudo MODE=dev SERVER_PUBLIC_IP=0.0.0.0 make start` @tongli @baohua Could you please take a look at this and suggest a fix. https://chat.hyperledger.org/channel/cello?msg=hnk5NaSAwnJjDfj5e
[ ](https://chat.hyperledger.org/channel/cello?msg=v9amureWQMqWSLG5b) @bcook 1 Hi...I am getting the same error when I execute this command: `sudo MODE=dev SERVER_PUBLIC_IP=0.0.0.0 make start` @tongli @baohua Could you please take a look at this and suggest a fix.
[ ](https://chat.hyperledger.org/channel/cello?msg=XpbQb9nfn2hYgrC5J) @jarvis26 seems docker mount problem. what version are you using?
@hightall I remember it happened before with some specific version of Docker?
Has joined the channel.
Has joined the channel.
Anyone know how to make the readthedocs index panel to the left side, instead of the top? https://cello.readthedocs.io/en/latest/
Only the latest branch has that problem
https://gerrit.hyperledger.org/r/#/c/29298/
I can login to operater dashboard but can not login to userdashboard on 8081, logs: cello-keycloak-server | 12:06:48,851 WARN [org.keycloak.services] (default task-2) KC-SERVICES0091: Request is missing scope 'openid' so it's not treated as OIDC, but just pure OAuth2 request.
chrome shows : Internal Server Error, real status: 500
@baohua
I can login to operater dashboard but can not login to userdashboard on 8081, logs: cello-keycloak-server | 12:06:48,851 WARN [org.keycloak.services] (default task-2) KC-SERVICES0091: Request is missing scope 'openid' so it's not treated as OIDC, but just pure OAuth2 request.
chrome shows : Internal Server Error, real status: 500
anyone can help me to resolve this issue?
06:06:05,937 WARN [org.keycloak.services] (default task-1) KC-SERVICES0091: Request is missing scope 'openid' so it's not treated as OIDC, but just pure OAuth2 request.
06:06:39,913 WARN [org.keycloak.services.managers.CodeGenerateUtil] (default task-1) Code '337eb6f3-1140-4969-a993-0951eb8ad07c' already used for userSession 'dbe06bee-38b6-4edf-8ae5-9f4a99fbc378' and client 'bec7098d-c8de-47e6-ae15-007dddf894d9'.
06:06:39,914 WARN [org.keycloak.events] (default task-1) type=CODE_TO_TOKEN_ERROR, realmId=277c1cbd-f841-4882-ba03-213b8b27a85a, clientId=cello-user-dashboard, userId=null, ipAddress=172.18.0.1, error=invalid_code, grant_type=authorization_code, code_id=dbe06bee-38b6-4edf-8ae5-9f4a99fbc378, client_auth_method=client-secret
hi all, @baohua @jarvis26 @tongli @nitishbhardwaj19 @bcook 1 my cello Installation completes successfully. No indication of error.
and I'm able to access the operator dashboard. But when I go to the user dashboard, it redirects to the the authentication page, and after i enter the creds, it goes back to
only the cello-keycloak-server show the logs: 06:06:05,937 WARN [org.keycloak.services] (default task-1) KC-SERVICES0091: Request is missing scope 'openid' so it's not treated as OIDC, but just pure OAuth2 request.
can you help me with this problem, and is there anyone get the same error?
[ ](https://chat.hyperledger.org/channel/cello?msg=FyZ5a8hSHxMD9Ktpd) @baohua Hi @baohua . Thanks for the reply. I got the following information with `docker version`:
```Client:
Version: 18.09.1
API version: 1.39
Go version: go1.10.6
Git commit: 4c52b90
Built: Wed Jan 9 19:35:23 2019
OS/Arch: linux/amd64
Experimental: false
```
And `docker-compose version`:
```docker-compose version 1.14.0, build c7bdf9e
docker-py version: 2.3.0
CPython version: 2.7.13
OpenSSL version: OpenSSL 1.0.1t 3 May 2016
```
[ ](https://chat.hyperledger.org/channel/cello?msg=WEY9Jmwkmsr48zscB) @chengloy Could you please share the docker and docker-compose version that you are using.
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=WEY9Jmwkmsr48zscB) @chengloy @chengloy I have faced the same error and it's gone after I run this command: "make build-admin-js"
@jarvis26 the version: Docker version 18.09.2, build 6247962 and docker-compose version 1.23.2, build 1110ad0
Client:
Version: 18.09.2
API version: 1.39
Go version: go1.10.6
Git commit: 6247962
Built: Sun Feb 10 04:13:27 2019
OS/Arch: linux/amd64
Experimental: false
docker-compose version 1.23.2, build 1110ad0
docker-py version: 3.7.0
CPython version: 2.7.5
OpenSSL version: OpenSSL 1.0.2k-fips 26 Jan 2017
Clipboard - February 15, 2019 4:26 PM
cello-keycloak-server | 08:26:58,772 WARN [org.keycloak.services] (default task-1) KC-SERVICES0091: Request is missing scope 'openid' so it's not treated as OIDC, but just pure OAuth2 request.
cello-keycloak-server | 08:27:08,035 WARN [org.keycloak.services.managers.CodeGenerateUtil] (default task-1) Code '27cd22cd-5444-450c-a26f-55558be9b3d5' already used for userSession '3a378053-8228-45ed-b51e-826b390ac2c7' and client '79257b65-324b-495c-ba9f-8feb518b71b9'.
cello-keycloak-server | 08:27:08,036 WARN [org.keycloak.events] (default task-1) type=CODE_TO_TOKEN_ERROR, realmId=db5f362f-32c5-425e-8705-c171302fc816, clientId=cello-user-dashboard, userId=null, ipAddress=172.18.0.1, error=invalid_code, grant_type=authorization_code, code_id=3a378053-8228-45ed-b51e-826b390ac2c7, client_auth_method=client-secret
Clipboard - February 15, 2019 4:27 PM
please help me
https://gerrit.hyperledger.org/r/#/c/29338/
https://gerrit.hyperledger.org/r/#/c/29339/
https://gerrit.hyperledger.org/r/#/c/29341/
https://gerrit.hyperledger.org/r/#/c/29342/
hi @baohua @lamnh2410 @jarvis26 @tongli @nitishbhardwaj19 I wanna ask for some help, my browser web socket communication in user dashboard is fail, when i access to user-dashboard and get the chain.
the log is
POST http://172.20.102.110:8081/socket.io/?EIO=3&transport=polling&t=MZ_IF5z&sid=Kzja3ziWuybHSyzlAAAe 400 (Bad Request)
Could you give me some suggestions?
it returned {"code":1,"message":"Session ID unknown"}
Has joined the channel.
"Create chain" operation fails on user dashboard
The request payload that ges to the API is
{"name":"newchain1","type":"fabric-1.0","size":4}
This results in a 400 (bad request)
response body : {"success":false}
is there anyone can support this problem?
[ ](https://chat.hyperledger.org/channel/cello?msg=hxN67AFrrZtwmsKGL) @chengloy Hi...I upgraded the docker and docker-compose version. Now upon execution of `sudo MODE=dev SERVER_PUBLIC_IP=0.0.0.0 make start`, I still get the same error: ```ERROR: for keycloak Cannot start service keycloak: OCI runtime create failed: container_linux.go:344: starting container process caused "process_linux.go:424: container init caused \"rootfs_linux.go:58: mounting \\\"/thirdparty/keycloak/resources/img/keycloak-logo-text.png\\\" to rootfs \\\"/var/lib/docker/overlay2/88a65e699aaaa6de125ccaf2519c9925c236a37fd0ed85bf80103c3691b3645b/merged\\\" at \\\"/var/lib/docker/overlay2/88a65e699aaaa6de125ccaf2519c9925c236a37fd0ed85bf80103c3691b3645b/merged/opt/jboss/keycloak/themes/keycloak/login/resources/img/keycloak-logo-text.png\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
ERROR: Encountered errors while bringing up the project.
Makefile:224: recipe for target 'start' failed
make: *** [start] Error 1``` I checked in the `/var/lib/docker/overlay2/88a65e699aaaa6de125ccaf2519c9925c236a37fd0ed85bf80103c3691b3645b` folder, it doesn't have the `merged` directory. @baohua @tongli Could you also take a look. Docker version used: 18.09.2, docker-compose version: 1.23.2, build 1110ad01.
@jarvis26 thank you, Oh as if you just upgrade the docker and docker-compose to the latest version, so do I
[ ](https://chat.hyperledger.org/channel/cello?msg=QJA6CoooWGvKArYsy) @chengloy any other reason which might be causing this?
[ ](https://chat.hyperledger.org/channel/cello?msg=jPmyQnXyzcvvtgcW8) @jarvis26 No clue, but the `SERVER_PUBLIC_IP` should be set to a real IP in your server. Besides, are u trying to start from empty env?
@jarvis26 no any reason, I just think the latest version might be suitable, and I also found that the cello docs update the system reqiurment for docker version.
[ ](https://chat.hyperledger.org/channel/cello?msg=ezkxeCNc99R5TuSii) @chengloy Yes, for valid docker versions range, welcome to help provide feedbacks.
@baohua, I just found that the docker hub build for ansible agent container has been failing ever since we changed the structure of cello.
I mean moving stuff from under src to operator-dashboard.
[ ](https://chat.hyperledger.org/channel/cello?msg=tBQmx9mkXMcQeCy4g) @tongli Which one? currently it should keep the old files.
@baohua I only found it is broken for ansible agent. the last succesful one was from 2 month ago.
I dont know if other images are also broken.
oh, not notice that happened.
let's try to see how to fix.
yeah, I was planning to use the anisble agent container to show stuff in Hong Kong.
BTW, I just booked my flight and hotel.
it is funny that the conference place is the same place when I worked there for two years.
ibm has an office there.
@baohua @hightall @Luke_Chen please look at this patch to fix the docker hub build issue. https://gerrit.hyperledger.org/r/29415
cool, see u in HK!
Has joined the channel.
@here dear all, we will cancel today's meeting due to 3 maintainers are attending hl events, thanks!
Has joined the channel.
Has joined the channel.
master node 16gb
Has joined the channel.
Has joined the channel.
@baohua @tongli @jarvis26 @lamnh2410 can you help to explain the api-engine module and what's the role of this api-engine, is it only used to make documents for api? I found everthing is good, when I docker stop cello-engine container.
sorry, it's my foolish, cello-engine used for operator-dashboard restfule server port:80 and operator-dashboard used for socketio server port: 8080, is it right?
[ ](https://chat.hyperledger.org/channel/cello?msg=f53iLDHQG6SPKgsWN) @chengloy basically correct and for the latest master branch, the dashboard port should be 8080 (operator) and 8081 (user)
Clipboard - February 28, 2019 3:05 PM
hi ,I set up cello service with command `make start`. Everything seems fine.When I try to access cello dashboard with url `SERVICE_IP:8080`, the link jumps to another url `127.0.0.1:8083`. Seems this is for user authentication? I want to know how to configure these .. thanks!
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=WdZRRHg6by62g6w5f) @bjwswang same issue here, I follow the start instructions and cello is not working
The service_IP should be specified as the public IP of the server, not 127.0.0.1.
Hi...is kubernetes support not provided yet for cluster in cello. I see the scripts env_k8s/apply.yml but they all call the flannelId script for docker swarm.
Hi...is kubernetes support not provided yet for cluster in cello? I see the scripts env_k8s/apply.yml but they all call the flannelId script for docker swarm.
```tasks:
- include_tasks: "../env_flanneld/fastinitnode/tasks/{{ mode }}.yml"
```
```
```
Also while running the initcluster step with env as k8s, I am getting the following error:
Also while running the initcluster step with env as k8s, I am getting the following error: @tongli, @baohua could you please take a look.
Also while running the initcluster step with env as k8s, I am getting the following error: @tongli, @baohua could you please take a look. Also when I logged in into the master node to check if the services were up and running - I executed the `kubectl get nodes` command which returned ```The connection to the server localhost:8080 was refused - did you specify the right host or port?```. On further investigation, I found that there was no kubeadm in /etc/kubernetes folder. I found some blogs which said executing `kubeadm init` created the admin.conf file which has to be referenced in `KUBECONFIG` environment variable for this the kubectl command to work. How do I fix this.
```FAILED! => {"changed": true, "cmd": ["/opt/fabric/bin/kubectl", "--server=
@jarvis26 r u using ansible agent?
Has left the channel.
Besides, i will be in HK bootcamp these 2 days, feel free to find me if you're here 🙂
Has joined the channel.
Has joined the channel.
cello-bootcamp.jpeg
cello.jpg
cello-bootcamp.jpg
Has joined the channel.
Has joined the channel.
@baohua After I "make setup-master" and "make start" , I try to hit the URL X.X.X.X:8080 and it gets redirected to http://X.X.X.X:8083/auth/realms/cello-realm/protocol/openid-connect/auth?client_id=cello-operator-dashboard&redirect_uri and this site gets unreachable.I have saw many problems similar to me before, but I could not find an effective solution.I'd appreciate if you could give me some advice.
which version are you using?
@hightall i think we fix this issue recently?
[ ](https://chat.hyperledger.org/channel/cello?msg=JZNrcThVxeXbomYQvh) @tongli yes
[ ](https://chat.hyperledger.org/channel/cello?msg=JZNrcThVxeXbomYQvh) @tongli Yes
[ ](https://chat.hyperledger.org/channel/cello?msg=RtwT5NKGpfRjvxH6n) @baohua where I can find the video??
[ ](https://chat.hyperledger.org/channel/cello?msg=cGiNYn7CPXDEvH5uA) @Javi Pity we do not have the video, and my slides is available at https://wiki.hyperledger.org/display/cello. the 6th slides.
https://gerrit.hyperledger.org/r/#/c/29951/
ok, thanks @baohua !!
Hello folks , can we have a link to video where cello deployment to demo hands on sample tutorial is available .
Has joined the channel.
Hello @baohua , I'm facing an issue with cello operator dashboard where I'm not able to open the dashboard by it's url, all the time I'm getting the uri_redirect error. Can you please help me on the same to fix this keycloak error. Since you're one of the cello contributor, please help me to resolve the same.
Thanks in advance
This may be due to some inconsistency in local file. If you are using Linux OS, then you may run `make reset`, and then follow the tutorial's steps to reconfig and start.
@tongli FYI: https://jira.hyperledger.org/browse/CE-577
@baohua do you think this is the issue with local files? Since I have downloaded the same several times from github, do you have any troubleshooting steps with you? Can you please share the same with me?
Has left the channel.
Hello folks !! can i have a video hands on workshop kind of stuff as i m naive to cello ..
@pikvik As of now, you will get the video link for cello in Youtube
any specific which is super easy to dive into and detailed on conceptual aspect . Please suggest @ArindamBhattacharjee
Since it's in Incubation phase, that's why there are lack of documentation
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=3ozrbKkH6qtX2xjWg) @ArindamBhattacharjee Have you followed this? https://cello.readthedocs.io/en/latest/tutorial/
I successfully start the service following the tutorial.
If there're still issues, would you like to help create a jira issue with more details? Thanks!
Has joined the channel.
Cannot find setup_k8s_worker.sh in master stated here - https://cello.readthedocs.io/en/latest/setup/setup_worker_kubernetes/
@baohua any hands on workshop from your end would be highly helpful
@baohua any hands on workshop from your end would be highly helpful , I mean link to any walkthrough kind
[ ](https://chat.hyperledger.org/channel/cello?msg=zdskHPcBqDndtJtBb) @rranjan3 it's in `scripts/worker_node`, would you like to help fix that doc issue?
[ ](https://chat.hyperledger.org/channel/cello?msg=NS5mAP9nqpAzbuWwq) @pikvik Thanks, we will consider to get more video tutorials after the release. Currently all busy with the development. Thanks!
[ ](https://chat.hyperledger.org/channel/cello?msg=khLcwx7Cj9K4ke3kJ) @baohua Thanks @baohua. Sure, I will raise a PR shortly.
[ ](https://chat.hyperledger.org/channel/cello?msg=khLcwx7Cj9K4ke3kJ) @baohua Thanks. Sure, I will raise a PR shortly.
Has joined the channel.
Not able to clone the repo using - git clone ssh://rranjan3@gerrit.hyperledger.org:29418/cello && scp -p -P 29418 rranjan3@gerrit.hyperledger.org:hooks/commit-msg cello/.git/hooks/. Do I need to get permissions somewhere? Able to clone the project otherwise though.
I want to ask which cello version used most, master branch or tag v 0.9.0 ? @baohua @hightall
@baohua I'm getting the below error attached in screenshot.
Not able to raise Jira request, can you please let me know how to raise the access for Jira board?
Has joined the channel.
@baohua I have raised the Jira request on the issue.
cello-keycloak-server | 13:15:25,802 WARN [org.keycloak.events] (default task-1) type=LOGIN_ERROR, realmId=9f993934-fa89-431b-b7fb-1dd1d4d8ff74, clientId=cello-operator-dashboard, userId=null, ipAddress=10.207.151.41, error=invalid_redirect_uri
Hi, I am trying to install cello dashboard by following "https://hyperledger-cello.readthedocs.io/en/latest/installation/ " totorial . Everything come up without an issue. I mean all the docker container but when I try to browse through localhost:8080 or nodeip:8080 I am getting redirected to link.
http://127.0.0.1:8083/auth/realms/cello-realm/protocol/openid-connect/auth?client_id=cello-operator-dashboard&redirect_uri=http%3A%2F%2Flocalhost%3A8080%2Foidc_callback&scope=openid+email&access_type=offline&response_type=code&state=eyJjc3JmX3Rva2VuIjogIjlPSmVtclFqYWVuaEV3enhYWTJiVmZ5QkFxTDdRb3JsIiwgImRlc3RpbmF0aW9uIjogImV5SmhiR2NpT2lKSVV6STFOaUo5LkltaDBkSEE2THk5c2IyTmhiR2h2YzNRNk9EQTRNQzhpLmFTdFNUMXU1cVJZY0J1M09LLWF6eXdTalhqM0duN0Viam52NFVfSVlUbWMifQ%3D%3D&openid.realm=http%3A%2F%2F127.0.0.1%3A8080%2Foidc_callback
not able to resolve this problem. Can anyone have any clue about this issue ?
thanks for help!!!
Hi, I am trying to install cello dashboard by following "https://hyperledger-cello.readthedocs.io/en/latest/installation/ " tutorial . Everything comes up without an issue. I mean all the docker container but when I try to browse through localhost:8080 or nodeip:8080 I am getting redirected to link.
http://127.0.0.1:8083/auth/realms/cello-realm/protocol/openid-connect/auth?client_id=cello-operator-dashboard&redirect_uri=http%3A%2F%2Flocalhost%3A8080%2Foidc_callback&scope=openid+email&access_type=offline&response_type=code&state=eyJjc3JmX3Rva2VuIjogIjlPSmVtclFqYWVuaEV3enhYWTJiVmZ5QkFxTDdRb3JsIiwgImRlc3RpbmF0aW9uIjogImV5SmhiR2NpT2lKSVV6STFOaUo5LkltaDBkSEE2THk5c2IyTmhiR2h2YzNRNk9EQTRNQzhpLmFTdFNUMXU1cVJZY0J1M09LLWF6eXdTalhqM0duN0Viam52NFVfSVlUbWMifQ%3D%3D&openid.realm=http%3A%2F%2F127.0.0.1%3A8080%2Foidc_callback
not able to resolve this problem. Can anyone have any clue about this issue?
thanks for the help!!!
Has joined the channel.
Hello!
May I know how do I use Hyperledger Cello to build a web & mobile app? It's for my school project.
[ ](https://chat.hyperledger.org/channel/cello?msg=kznmGi9LkavEvChKt) @ArindamBhattacharjee Ask for more details in that issue, thanks
[ ](https://chat.hyperledger.org/channel/cello?msg=9gNw6NDeMx97YTauT) @Gaurang guess you set local ip to the service ip?
@baohua @hightall Please review this patch. https://gerrit.hyperledger.org/r/#/c/30104/
this patch is to set gossip port correctly for multiple clouds env.
+2, thanks!
[ ](https://chat.hyperledger.org/channel/cello?msg=z46bzXB6WN975AteF) @baohua I have not changed anything in the cello scripts. Just following the tutorial as is.
Has left the channel.
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=sajPyK2AJSetgv7zc) @MohammadObaid yess
Screen Shot 2019-03-15 at 4.19.29 PM.png
Is it because of docker connection ??
https://gerrit.hyperledger.org/r/#/c/30132/
I am not getting dashboard at the host address .. Please guide me where m doing wrong . i followed official installation guide
[ ](https://chat.hyperledger.org/channel/cello?msg=jQqosyPGtX549vPfR) @Gaurang Facing the same issue ..
@Gaurang Have u got the issue fixed .. please let me know where i m going wrong ...
@baohua Please help with the fix to the issue ... where am i doing wrong ..
[ ](https://chat.hyperledger.org/channel/cello?msg=q6C5T3YcPnmBMhLjT) @pikvik no.. I have not found the fix yet.
@Gaurang How many ports have you defined ?? I m deploying on vm and have only enabled ports 8080 and 8081 is it because of that i m getting this error ... Am i supposed to define 8083,84,and 85 too ??? @baohua Please suggest
Got it ... thanks folks . In my vm i needed to map the ports ..
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=AxMnuB4AxY27stbvL) @pikvik I was simply trying to bring up the master node on my ubuntu machine.
[ ](https://chat.hyperledger.org/channel/cello?msg=FWscZzqkC3jJf946t) @pikvik Are you trying on cloud or local VM's ??
[ ](https://chat.hyperledger.org/channel/cello?msg=NQNt8oFgpJhT5KbfP) @Gaurang yess
[ ](https://chat.hyperledger.org/channel/cello?msg=36N24nhtDacfnrbwi) @pikvik this yess is for local VMs?
[ ](https://chat.hyperledger.org/channel/cello?msg=537YLdHXmtfeFTLmf) @Gaurang its cloud vm
[ ](https://chat.hyperledger.org/channel/cello?msg=Ad5wJx89pEePzouy9) @pikvik thanks. What was your docker engine version?
[ ](https://chat.hyperledger.org/channel/cello?msg=jbG2mrd8dncAPNs5g) @Gaurang 18.09.3
Screen Shot 2019-03-22 at 12.05.23 PM.png
[ ](https://chat.hyperledger.org/channel/cello?msg=JarSCvzSeWmk8Regw) @pikvik thanks. I will try with the same on my aws account.
Has joined the channel.
@Gaurang but i m facing invalid parameter error on user dashboard end
Clipboard - March 22, 2019 4:38 PM
Clipboard - March 22, 2019 4:44 PM
Clipboard - March 22, 2019 4:49 PM
[ ](https://chat.hyperledger.org/channel/cello?msg=7NxntqA4SwcvevCz3) @pikvik I am still facing the same issue even after opening 8081,8080,8083,85,84,2049,22 ports.
@pikvik Surely some other steps are involved bringing up cello master mode. I am following "https://hyperledger-cello.readthedocs.io/en/latest/installation_master/" this tutorial.
@pikvik I am also getting similar invalid parameter issue . Also I have figure out that you have to start cello server with this command: SERVER_PUBLIC_IP=x.x.x.x make start where as in documents it is mentioned make start only.
@pikvik Use this URL: http://
[ ](https://chat.hyperledger.org/channel/cello?msg=PaFivE9y7tsPXpBu5) @Gaurang @Gaurang Thanks . but for me still the same
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=sRkxf8tQXpJd5YhTn) @pikvik try fresh installation and make sure all required ports open before you start the services.
Clipboard - March 24, 2019 11:54 PM
Is there any official docs to get started with Cello?
[ ](https://chat.hyperledger.org/channel/cello?msg=QpAFz9WMLJhMWEEkx) @Vanitha https://hyperledger-cello.readthedocs.io/en/latest/tutorial/ would be a nice starter I guess
Thank you @JohnR
Screenshot from 2019-03-26 16-25-51.png
Could someone help us to resolve this error
When checked for containers my USER dashboard is exciting
and it's not showing in containers list, though I have my operator dashboard running I was unable to login
Did you check the container logs?
yes
and what did the logs from the exited container say?
cello-user-dashboard error.png
This are the logs of my user dashboard
@Vanitha For me this error was solved when i defined and enabled the ports and mapped them
@pikvik where should we define these ports and map them?
I did it on my cloud since the deployment was on cloud , u can do that on ur local just check if the ports are enabled
I am doing in my local machine and ports were enabled
this is because of keycloak authentication redirecting .. check with keycloak config file
I presume
This env.tmpl file looks good, is there any need to modify this file?
Has joined the channel.
Hello Everyone, I'm facing problem while running make start cmd for first time error looks like:
*Recreating cello-engine ... error
ERROR: for cello-engine Cannot start service engine: driver failed programming external connectivity on endpoint cello-engine (c27380e5ec2af1*
*
ERROR: for engine Cannot start service engine: driver failed programming external connectivity on endpoint cello-engine (c27380e5ec2af198249a2b1156cf23bc173a155c4d18621dd56cb45ceec5a350): Error starting userland proxy: listen tcp 0.0.0.0:80: bind: address already in use
ERROR: Encountered errors while bringing up the project.
Makefile:237: recipe for target 'start-old' failed
make[1]: *** [start-old] Error 1
make[1]: Leaving directory '/home/amrit/cello'
Makefile:253: recipe for target 'start' failed
make: *** [start] Error 2*
can anybody help me out please?
@ShrmaAmrit your port 80 is already bind to cello-engine..first you sttop the processes on that port and try again
I have follwed the cello tutorial for master node from https://hyperledger-cello.readthedocs.io/en/latest/installation_master/
I am getting warning as
"WARNING: Found orphan containers (cello-user-dashboard, cello-watchdog, cello-mongo, cello-operator-dashboard, cello-engine, cello-dashboard_mongo, cello-dashboard_rabbitmq) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
" and then its getting like "Now you can visit operator-dashboard at localhost:8080, or user-dashboard at localhost:8081" while going through the port its saying refused to connect...
what is the issue...can anyone help it out?
Clipboard - March 28, 2019 3:37 PM
Clipboard - March 29, 2019 11:52 AM
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=rRGiNm7SsMoJKH7tr) @KalyaniPullela you need to kill all the docker container before starting it again. "make stop" can help you here.
Screenshot from 2019-03-29 14-31-38.png
I am unable to access the github link given in official docs *http://gerrit.hyperledger.org/r/cello*
Has left the channel.
Has joined the channel.
Has joined the channel.
Does it look like your problem is solved?
Clipboard - April 2, 2019 10:24 AM
Hi...does cello still run on fabric 1.0.5..? Or does it support later releases?
[ ](https://chat.hyperledger.org/channel/cello?msg=uDyqGAsXi2afyMAXu) @jarvis26 Yes, already supports newer version.
Welcome to review: https://gerrit.hyperledger.org/r/#/c/30721/
@baohua Can you help me out?
[ ](https://chat.hyperledger.org/channel/cello?msg=pgy4cwBcj3QRe7oie) @Xiaozhang Please open a jira issue including the steps and details to repeat the error msg, which will help to dig out the root cause. Thanks!
Clipboard - April 10, 2019 9:15 AM
[ ](https://chat.hyperledger.org/channel/cello?msg=zmiEWkasSMSXGpkNx) @baohua
Is there a downloadable document? print
Clipboard - April 11, 2019 1:24 AM
Has joined the channel.
Has joined the channel.
Morning all, apologies for the basic question but I want to use a blockchain solution but not host anode. Does Cello offer a complete BAAS solution so I can develop a dapp but not have to deal with node deployment and hosting?
[ ](https://chat.hyperledger.org/channel/cello?msg=mssfcsWpW2YqLE3q3) @baohua Hi.. @baohua I tried seeting up fabric network with this. I am getting the following error. Could you please help me with this. Thanks. ```TASK [Create the certificate files] *********************************************************************************************************************************************************
fatal: [cloud]: FAILED! => {"changed": false, "cmd": "/home/ansible-dev/cello/src/operator-dashboard/agent/ansible/roles/deploy_k8s/../../vars/bc1st/bin/cryptogen generate --config crypto-config.yml --output keyfiles", "msg": "[Errno 8] Exec format error", "rc": 8}
to retry, use: --limit @/home/ansible-dev/cello/src/operator-dashboard/agent/ansible/setupfabric.retry```
[ ](https://chat.hyperledger.org/channel/cello?msg=mssfcsWpW2YqLE3q3) @baohua Hi.. I tried seeting up fabric network with this. I am getting the following error. Could you please help me with this. Thanks. ```TASK [Create the certificate files] *********************************************************************************************************************************************************
fatal: [cloud]: FAILED! => {"changed": false, "cmd": "/home/ansible-dev/cello/src/operator-dashboard/agent/ansible/roles/deploy_k8s/../../vars/bc1st/bin/cryptogen generate --config crypto-config.yml --output keyfiles", "msg": "[Errno 8] Exec format error", "rc": 8}
to retry, use: --limit @/home/ansible-dev/cello/src/operator-dashboard/agent/ansible/setupfabric.retry```
@baohua I applied this patch(https://gerrit.hyperledger.org/r/#/c/30721/) on cello's current code to setup a network on 1.4.0. I am got the same error
@baohua I applied this patch(https://gerrit.hyperledger.org/r/#/c/30721/) on cello's current code to setup a network on 1.4.0. I am got the same error `TASK [Create the certificate files] *********************************************************************************************************************************************************
fatal: [cloud]: FAILED! => {"changed": false, "cmd": "/home/ansible-dev/cello/src/operator-dashboard/agent/ansible/roles/deploy_k8s/../../vars/bc1st/bin/cryptogen generate --config crypto-config.yml --output keyfiles", "msg": "[Errno 8] Exec format error", "rc": 8}
to retry, use: --limit @/home/ansible-dev/cello/src/operator-dashboard/agent/ansible/setupfabric.retry`
@baohua I applied this patch(https://gerrit.hyperledger.org/r/#/c/30721/) on cello's current code to setup a network on 1.4.0. I am got the same error
TASK [Create the certificate files] *********************************************************************************************************************************************************
fatal: [cloud]: FAILED! => {"changed": false, "cmd": "/home/ansible-dev/cello/src/operator-dashboard/agent/ansible/roles/deploy_k8s/../../vars/bc1st/bin/cryptogen generate --config crypto-config.yml --output keyfiles", "msg": "[Errno 8] Exec format error", "rc": 8}
to retry, use: --limit @/home/ansible-dev/cello/src/operator-dashboard/agent/ansible/setupfabric.retry
@baohua I applied this patch(https://gerrit.hyperledger.org/r/#/c/30721/) on cello's current code to setup a network on 1.4.0. I am got the same error
TASK [Create the certificate files] *********************************************************************************************************************************************************
fatal: [cloud]: FAILED! => {"changed": false, "cmd": "/home/ansible-dev/cello/src/operator-dashboard/agent/ansible/roles/deploy_k8s/../../vars/bc1st/bin/cryptogen generate --config crypto-config.yml --output keyfiles", "msg": "[Errno 8] Exec format error", "rc": 8}
to retry, use: --limit @/home/ansible-dev/cello/src/operator-dashboard/agent/ansible/setupfabric.retry
@baohua I applied this patch(https://gerrit.hyperledger.org/r/#/c/30721/) on cello's current code to setup a network on 1.4.0. I am got the same error
TASK [Create the certificate files] *********************************************************************************************************************************************************
fatal: [cloud]: FAILED! => {"changed": false, "cmd": "/home/ansible-dev/cello/src/operator-dashboard/agent/ansible/roles/deploy_k8s/../../vars/bc1st/bin/cryptogen generate --config crypto-config.yml --output keyfiles", "msg": "[Errno 8] Exec format error", "rc": 8}
to retry, use: --limit @/home/ansible-dev/cello/src/operator-dashboard/agent/ansible/setupfabric.retry
Could you please share your thoughts.
[ ](https://chat.hyperledger.org/channel/cello?msg=kELto2AQ8yjr4Q5Bn) @nitishbhardwaj19 @tongli may help have a look?
Has joined the channel.
@baohua any inputs on this issue https://jira.hyperledger.org/browse/CE-513?
I see ppc64le and s390x release jobs are failing since months..
Has joined the channel.
Has joined the channel.
Has joined the channel.
Hi guys! Can someone please help me deploying cello? I've been trying over and over again by following the doc, but I can't access either the operator nor the user dashboard. For the operator dasboard, after loging in keycloack, I'm redirected to a blank page. Here are the container's log:
```
cello-operator-dashboard | [2019-04-28 10:54:11 +0000] [23] [ERROR] Socket error processing request.
cello-operator-dashboard | Traceback (most recent call last):
cello-operator-dashboard | File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/async.py", line 62, in handle
cello-operator-dashboard | six.reraise(*sys.exc_info())
...
cello-operator-dashboard | File "/usr/local/lib/python3.6/site-packages/eventlet/greenio/base.py", line 50, in socket_checkerr
cello-operator-dashboard | raise socket.error(err, errno.errorcode[err])
cello-operator-dashboard | ConnectionRefusedError: [Errno 111] ECONNREFUSED
```
For the user dashboard, I just get "Internal Server Error, real status: 500" in the browser with no error log on the container.
I'm trying to deploy everything locally; here's my configuration:
```
ubuntu 18.04.2 LTS
docker version: 18.09.5
docker-compose version 1.13.0, build 1719ceb
```
I'm using the latest sources; I've been struggling with this for several days now. What am I missing?
Hi guys! Can someone please help me deploy cello? I've been trying over and over again by following the doc, but I can't access either the operator nor the user dashboard. For the operator dasboard, after loging in keycloack, I'm redirected to a blank page. Here are the container's log:
```
cello-operator-dashboard | [2019-04-28 10:54:11 +0000] [23] [ERROR] Socket error processing request.
cello-operator-dashboard | Traceback (most recent call last):
cello-operator-dashboard | File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/async.py", line 62, in handle
cello-operator-dashboard | six.reraise(*sys.exc_info())
...
cello-operator-dashboard | File "/usr/local/lib/python3.6/site-packages/eventlet/greenio/base.py", line 50, in socket_checkerr
cello-operator-dashboard | raise socket.error(err, errno.errorcode[err])
cello-operator-dashboard | ConnectionRefusedError: [Errno 111] ECONNREFUSED
```
For the user dashboard, I just get "Internal Server Error, real status: 500" in the browser with no error log on the container.
I'm trying to deploy everything locally; here's my configuration:
```
ubuntu 18.04.2 LTS
docker version: 18.09.5
docker-compose version 1.13.0, build 1719ceb
```
I'm using the latest sources; I've been struggling with this for several days now. What am I missing?
Has joined the channel.
[ ](https://chat.hyperledger.org/channel/cello?msg=Si3pRHwT827r9hdSf) @rameshthoomu I think it may be because we are re-arching after the v0.9.0 release. There might need some change to the ppc64/s390x dockerfile.
@hightall , hi, I just run user-dashboard at my host, then i find the first page is blank when i login localhost:8081. I check user-dashboard sources and get some problems, the npm start application with load "index.tpl", index.tpl load ”“ , but i can't find any "static" folder in user-dashboard project. Is it the reason for the first blank page? How to fix it ? Thanks
[ ](https://chat.hyperledger.org/channel/cello?msg=H4SvGRsRShn9ZeHSZ) I find the static link define in the config.default.js, it would be the fact
[ ](https://chat.hyperledger.org/channel/cello?msg=EYLqNAufGxj2JjYRo) @Gaurang @Vanitha Could you guys please help me with this? Seems like you had the same kind of issues. Thanks
Has joined the channel.
Hi, so i am setting up a demo instance of cello.. uptill now i am successful in the configurtion of master and worker nodes and i can access the user and operator dashboard. I am also able to create host in operator dashboard but when i create the chain in operation dashboard it shows the status 'deleting' and health 'failed'. When i digged more into the detail so i found out that it initially creates the fabric containers but then they exit. Before exiting the containers have this error "ERRO 021 Cannot run peer because error when setting up MSP of type bccsp from directory /etc/hyperledger/fabric/msp: could not load a valid signer certificate from directory /etc/hyperledger/fabric/msp/signcerts: stat /etc/hyperledger/fabric/msp/signcerts: no such file or directory". Now i am following all the steps mentioned in the guidelines and tutorial. I dont understand why these cryptographic sign certs are missing. Did anyone else faced the same issue too? if yes how to resolve this?
[ ](https://chat.hyperledger.org/channel/cello?msg=7Bi6sLynRnhAnMDYm) Hi, so i am setting up a demo instance of cello.. uptill now i am successful in the configurtion of master and worker nodes and i can access the user and operator dashboard. I am also able to create host in operator dashboard but when i create the chain in operation dashboard it shows the status 'deleting' and health 'failed'. When i digged more into the detail so i found out that it initially creates the fabric containers but then they exit. Before exiting the containers have this error "*ERRO 021 Cannot run peer because error when setting up MSP of type bccsp from directory /etc/hyperledger/fabric/msp: could not load a valid signer certificate from directory /etc/hyperledger/fabric/msp/signcerts: stat /etc/hyperledger/fabric/msp/signcerts: no such file or directory*". Now i am following all the steps mentioned in the guidelines and tutorial. I dont understand why these cryptographic sign certs are missing. Did anyone else faced the same issue too? if yes how to resolve this?
.
chains.PNG
chains.PNG
chains.PNG
Another question that i have related to this is that i am not being asked to provide my own crypto certificates at dashboard. would these certificates be hardcoded then in the new blockchain? Or am i missing a way to provide my own configuration of crypto certificates and then spawn the blockchain according to them? Could not find anything related to this in the documentation..
.
chain error.PNG
Another question that i have related to this is that i am not being asked to provide my own crypto certificates at dashboard. would these certificates be hardcoded then in the new blockchain? Or am i missing a way to provide my own configuration of crypto certificates and then spawn the blockchain according to them? Could not find anything related to this in the documentation..
Has joined the channel.
Has joined the channel.
Hello Everyone, i'm new here, and i need your help.
Thank you before my questions.
How can i change the cello-engine bind port from 80 to others ??
[ ](https://chat.hyperledger.org/channel/cello?msg=DSoaT6oR4kT5Xpi6o) @Ulysses If you start cello using the make cmd, then the port mapping is specified in the docker-compose.yml file.
And you can learn some knowledge on the docker-compose file here: https://docs.docker.com/compose/compose-file/
Nice and thanks very much
Has joined the channel.
Hi everybody,
I'm deploying cello chain with fabric 1.2, but i'm facing with this issue
`Cannot run peer because error when setting up MSP of type bccsp from directory /etc/hyperledger/fabric/msp: could not load a valid signer certificate from directory /etc/hyperledger/fabric/msp/signcerts: stat /etc/hyperledger/fabric/msp/signcerts: no such file or directory`
Could anybody help me for this?
Has joined the channel.
Hello team, I want to create fabric network in an on-premise k8s cluster. So can anyone please confirm if hat can be done with the help of Cello ? If yes, please share the reference material/documentation for the same. Thanks in advance !
Hello team, I want to create fabric network (v1.3) in an on-premise k8s cluster. So can anyone please confirm if hat can be done with the help of Cello ? If yes, please share the reference material/documentation for the same. Thanks in advance !
@HLFPOC please take a look at this doc.
you should be able to do that in few minutes.
```https://github.com/hyperledger/cello/blob/master/docs/worker_ansible_docker.md```
please follow the instructions in that doc, and you should have your network in few minutes.
if you have to use 1.3, then please change the image tag accordingly.
Thanks for sharing it @tongli . Will go through it and will let you know in case of any issues.
I see many people including myself stuck at this.. why is nobody helping :/
`*Cannot run peer because error when setting up MSP of type bccsp from directory /etc/hyperledger/fabric/msp: could not load a valid signer certificate from directory /etc/hyperledger/fabric/msp/signcerts: stat /etc/hyperledger/fabric/msp/signcerts: no such file or directory*`
Has joined the channel.
I had met the same problem. And I solved it by copy the directory 'src/operator-dashboard/agent/docker/_compose_files/fabric-1.0' to '/opt/cello/fabric-1.0'
Does anyone encounter the problem:"TypeError: Cannot read property 'key' of null". When entering "http://localhost:8081/#/chain/info/Pmbl2gUGh1Z7WDPxM4PTnqDJtAvM30dk"
i'm using Fabric 1.2, your solution not working :(
Not met before. Could you help open an jira issue with details? Thanks!
Hi, thanks for your reply.. your solution works for all fabric versions but only on consensus plugin solo.. it doesnt work with kafka.. :/
Has joined the channel.
Is it possible set up faric1.4 network with cello ?
Has joined the channel.
Has joined the channel.
:tools: If you can blindly trust other parties to have your infrastructure and centralize it for the whole network, then a blockchain is not what you need.
I think this article is great for blockchain adoption:
https://medium.com/worldsibu/forma-the-baas-killer-61f470fcbc78
Yes! 1.4 is the major LTS
@YutaoSong yes, if you use ansible agent you can setup fabric network using any fabric release including 1.4.1
and use raft.
Has joined the channel.
Has joined the channel.
Has joined the channel.
Hi...I am running a network created by cello over kubernetes cluster in azure...I enabled TLS to true in bc1st.yaml file and restarted the network. I am getting the following error: ```2019-06-11 06:38:13.767 UTC [grpc] handleRawConn -> DEBU 76be6 grpc: Server.Serve failed to complete security handshake from "10.244.0.1:59187": read tcp 10.244.0.40:7050->10.244.0.1:59187: i/o timeout
2019-06-11 06:38:13.979 UTC [core.comm] ServerHandshake -> ERRO 76be7 TLS handshake failed with error EOF {"server": "Orderer", "remote address": "10.240.0.18:51267"}
2019-06-11 06:38:13.979 UTC [grpc] handleRawConn -> DEBU 76be8 grpc: Server.Serve failed to complete security handshake from "10.240.0.18:51267": EOF
2019-06-11 06:38:14.319 UTC [core.comm] ServerHandshake -> ERRO 76be9 TLS handshake failed with error EOF {"server": "Orderer", "remote address": "10.240.0.9:51024"}
2019-06-11 06:38:14.320 UTC [grpc] handleRawConn -> DEBU 76bea grpc: Server.Serve failed to complete security handshake from "10.240.0.9:51024": EOF ```
Hi...I am running a network created by cello over kubernetes cluster in azure...I enabled TLS to true in bc1st.yaml file and restarted the network. I am getting the following error at the orderer: ```2019-06-11 06:38:13.767 UTC [grpc] handleRawConn -> DEBU 76be6 grpc: Server.Serve failed to complete security handshake from "10.244.0.1:59187": read tcp 10.244.0.40:7050->10.244.0.1:59187: i/o timeout
2019-06-11 06:38:13.979 UTC [core.comm] ServerHandshake -> ERRO 76be7 TLS handshake failed with error EOF {"server": "Orderer", "remote address": "10.240.0.18:51267"}
2019-06-11 06:38:13.979 UTC [grpc] handleRawConn -> DEBU 76be8 grpc: Server.Serve failed to complete security handshake from "10.240.0.18:51267": EOF
2019-06-11 06:38:14.319 UTC [core.comm] ServerHandshake -> ERRO 76be9 TLS handshake failed with error EOF {"server": "Orderer", "remote address": "10.240.0.9:51024"}
2019-06-11 06:38:14.320 UTC [grpc] handleRawConn -> DEBU 76bea grpc: Server.Serve failed to complete security handshake from "10.240.0.9:51024": EOF ```
Do I need to change any other settings anywhere? I am not getting any errors at the peer end.
I am getting the following error in the peer container: ```2019-06-11 06:44:01.822 UTC [core.comm] ServerHandshake -> ERRO 13ca12 TLS handshake failed with error read tcp 10.244.2.61:7051->10.240.0.16:51981: i/o timeout {"server": "PeerServer", "remote address": "10.240.0.16:51981"}
2019-06-11 06:44:01.822 UTC [grpc] handleRawConn -> DEBU 13ca13 grpc: Server.Serve failed to complete security handshake from "10.240.0.16:51981": read tcp 10.244.2.61:7051->10.240.0.16:51981: i/o timeout
2019-06-11 06:44:01.844 UTC [core.comm] ServerHandshake -> ERRO 13ca14 TLS handshake failed with error read tcp 10.244.2.61:7052->10.240.0.13:52500: i/o timeout {"server": "ChaincodeServer", "remote address": "10.240.0.13:52500"}
2019-06-11 06:44:01.844 UTC [grpc] handleRawConn -> DEBU 13ca15 grpc: Server.Serve failed to complete security handshake from "10.240.0.13:52500": read tcp 10.244.2.61:7052->10.240.0.13:52500: i/o timeout
2019-06-11 06:44:01.982 UTC [core.comm] ServerHandshake -> ERRO 13ca16 TLS handshake failed with error EOF {"server": "ChaincodeServer", "remote address": "10.240.0.18:62455"}
2019-06-11 06:44:01.982 UTC [grpc] handleRawConn -> DEBU 13ca17 grpc: Server.Serve failed to complete security handshake from "10.240.0.18:62455": EOF
2019-06-11 06:44:02.337 UTC [core.comm] ServerHandshake -> ERRO 13ca18 TLS handshake failed with error EOF {"server": "PeerServer", "remote address": "10.240.0.11:57352"}
2019-06-11 06:44:02.337 UTC [grpc] handleRawConn -> DEBU 13ca19 grpc: Server.Serve failed to complete security handshake from "10.240.0.11:57352": EOF```
Even after enabling tls, its making grpc calls, I guess. @tongli @baohua Could you please help.
Hey guys, how are you? Just wanted to know, is there any support for fabric 1.4?
NVM, just saw the answer few messages above, ty guys have a wonderfull day!
Has joined the channel.
Has joined the channel.
Hi...how do I check if chaincode containers are getting created in the pods in kubernetes cluster. Executing this command - ```kubectl get pod
Has joined the channel.
Has joined the channel.
hi experts here, what is the chaincode install path in peer? Thank you.
the default chaincode install path
maybe this is set by fabric sdk, not cell. sorry
@jarvis26 hi, I am traveling this week. Will be home next week. I would like to know how you setup your network and very mich like to help.
Hi, what's the differences between `make start-old` and `make start-next`?
what should I use when start master service ?
`make start-next` is to use the latest code (on-going).
Thank you @baohua I am trying to start the master service on my virtual machine. The command was `SERVER_PUBLIC_IP=x.x.x.x make start` ,but when I tried to open the dashboard with link x.x.x.x:8080. It redirected to ` localhost/auth/realms/protocal/openid-conection/auth? ... ` .
Thank you @baohua I am trying to start the master service on my virtual machine. The command was `SERVER_PUBLIC_IP=x.x.x.x make start` ,but when I tried to open the dashboard with link x.x.x.x:8080. It redirected to ` localhost/auth/realms/protocal/openid-conection/auth? ... ` . Do you know how to solve this problem ?
@baohua Hi..I am trying to setup new cello project and the chaincode deployment fails gracefully. One thing to notice is that all the peer, orderer and ca pods are being appended with `-0` in the end which is probably hindering with namespace resolution. How do I fix this?
@baohua Hi..I am trying to setup new cello project and the chaincode deployment fails gracefully. One thing to notice is that all the peer, orderer and ca pods are being appended with `-0` like `peer1st-orga-0` etc in the end which is probably hindering with namespace resolution. How do I fix this?
which version are you using?
I cloned the latest code in the master branch
oh, it's on-going now, not finished yet. recently we will make some demo, please watch the meeting~~
oo..ok
could you please provide me with a link to a stable version that works with TLS enabled
sorry, i remember the last release does not claim tls as a feature.
ok..I was trying to set that up but I keep getting this error: ```TLS handshake failed with error EOF``` I made a lot of changes in the code and was able to make it work till the instantiation of chaincode on a channel. TLS handshake fails between peer and the chaincode container while executing an invoke. Do you have any pointers here?
hmmm, for tls, there should be three things to consider: 1. config all nodes to enable TLS; 2. set correct tls cert/key files to right places. 3. set tls ca info of each org, which will be stored in the channel config.
I believe I did that
I need to debug more on this, I guess.
Just a quick question, How do I check the grpc version which gets installed on the pods
Also how do I get the invite/link to join the meeting?
hey guys, when doing make start i get the folowing error:
```
ERROR: for 27fd77ba5e4f_cello-initial-keycloak-server Cannot start service keycloak: OCI runtime create failed: container_linux.go:345: starting container process caused "process_linux.go:430: container init caused \"rootfs_linux.go:58: mounting \\\"/home/vagrant/cello/thirdparty/keycloak/resources/img/keycloak-logo-text.png\\\" to rootfs \\\"/var/lib/docker/overlay2/fd95a173cc57cf3087862e98872e885631e44cbf9ad7f74a0aba0c46ce59c6f4/merged\\\" at \\\"/var/lib/docker/overlay2/fd95a173cc57cf3087862e98872e885631e44cbf9ad7f74a0aba0c46ce59c6f4/merged/opt/jboss/keycloak/themes/keycloak/login/resources/img/keycloak-logo-text.png\\\" caused \\\"not a directory\\\"\"": unknown: ARecreating cello-watchdog ... done
```
ok, just had to copy those files
Please see the project wiki and maillist, thanks!
Has joined the channel.
Hi Guys, Please help me!!! I try run hyperledger Cello v0.9.0 but the docker hyperledger/cello-user-dashboard breaks.
I have this error
`
2019-07-11 19:18:54,609 INFO 154 [egg-multipart] stream mode enable
/node_modules/egg/lib/application.js:70
throw e;
^
AssertionError [ERR_ASSERTION]: [egg-passport-oauth2] config.passportOauth2.key required
at module.exports.app (/node_modules/egg-passport-oauth2/app.js:10:3)
at Hook.configDidLoad (/node_modules/egg-core/lib/lifecycle.js:92:9)
at Lifecycle.triggerConfigDidLoad (/node_modules/egg-core/lib/lifecycle.js:143:14)
at Lifecycle.triggerConfigWillLoad (/node_modules/egg-core/lib/lifecycle.js:137:10)
at AppWorkerLoader.loadCustomApp (/node_modules/egg-core/lib/loader/mixin/custom.js:37:20)
at AppWorkerLoader.load (/node_modules/egg/lib/loader/app_worker_loader.js:33:10)
at new Application (/node_modules/egg/lib/application.js:66:19)
at Object.
Has joined the channel.
Has joined the channel.
Hello, I'm struggling to get Cello up and running. On multiple machines, it says "cello-initial | python: can't open file '/app/initial.py': [Errno 2] No such file or directory" as the final error when running SERVER_PUBLIC_IP=XXX.XX.XXX.XX make start
Hi All, Can any one help me how to deploy hyperledger fabric(Production ready) using Cello services? @tongli @baohua
@sureshtedla did you look at this doc? https://github.com/hyperledger/cello/blob/master/docs/worker_ansible_docker.md
couple of steps to get it going if you have a k8s cluster.
@HarryLovesCode Were u using latest code ? Cello will do keycloak initialization before start master service. This is the initialization docker-compose file https://github.com/hyperledger/cello/blob/master/bootup/docker-compose-files/docker-compose-initial.yml#L26
the docker-compose file will add a volume `- $ROOT_PATH/src/operator-dashboard:/app` , but `src/operator-dashboard` not longer exists in latest code...
@HarryLovesCode After I reset the commit level to `57f12abd9ae9a8c1997d791fa5de2525ce8530e9` and start master service with `SERVER_PUBLIC_IP=x.x.x.x ROOT_PATH=${cello_root_path} make start` , the cello service start up successfully now .
@HarryLovesCode After I reset the commit level to `57f12abd9ae9a8c1997d791fa5de2525ce8530e9` and start master service with `SERVER_PUBLIC_IP=x.x.x.x ROOT_PATH=${cello_root_path} make start` , the cello service can start up successfully now .
Hi Tong Li I am very new Hyperledger Cello service's can you explain in detail?
Has joined the channel.
Hi Team,When will cello 1.0 be released?
Has joined the channel.
hi tongli ... i couldnt able to found the much documentation regarding cello ... even i need some simple/layman explanation regarding configuration and integrating with fabric
thanks
hi am trying to setup the cello for my project .. just want to know am trying 0.8.0 setup .... do we need to change the default.env file to any updated values or we can leave as it is ?
does anybody getting this error while doing make start command and checking the logs for make logs user-dashboard
AssertionError [ERR_ASSERTION]: [egg-passport-oauth2] config.passportOauth2.key required
at module.exports.app (/node_modules/egg-passport-oauth2/app.js:10:3)
at Hook.configDidLoad (/node_modules/egg-core/lib/lifecycle.js:92:9)
at Lifecycle.triggerConfigDidLoad (/node_modules/egg-core/lib/lifecycle.js:143:14)
at Lifecycle.triggerConfigWillLoad (/node_modules/egg-core/lib/lifecycle.js:137:10)
at AppWorkerLoader.loadCustomApp (/node_modules/egg-core/lib/loader/mixin/custom.js:37:20)
at AppWorkerLoader.load (/node_modules/egg/lib/loader/app_worker_loader.js:33:10)
at new Application (/node_modules/egg/lib/application.js:66:19)
at Object.
Hi team , what is the key and secret that we have fill in hyperledger cello egg-passport-oauth2?
is it okay to use 0.8.0 cello version ?
hi meru cheppinatu nen post chesanuu
cello group loo
waiting for the hyperledger team response
Has joined the channel.
Hi..I am getting the following error when I am trying to spin the network up -
```2019/08/01 15:22:13 [DEBUG] TLS is enabled
2019/08/01 15:22:13 [DEBUG] Closing server DBs
Error: File specified by 'tls.keyfile' does not exist: /etc/hyperledger/fabric-ca-server-config/keyfiles/orga/tlsca/tlsca_private.key ```
The fabric ca containers are crashing.. All the other steps are running just fine
@jarvis26 show me your network spec file,
@tongli network spec file is the bc1st.yml right?
right.
jarvis26 - Thu Aug 01 2019 23:29:46 GMT+0530 (India Standard Time).txt
@jarvis26 this may not be the cause of the problem, but you should not do this.
logging_level: "grpc=debug:debug",
that should be just word DEBUG
I set that to debug my tls errors
helper_tag: "amd64-0.4.8",
that should also be changed to
helper_tag: "amd64-1.4.1",
all these things should not have been the cause of the CA crashing.
it seems to me that you are doing in on Azure?
can you do this
yes, I am doing it in Azure
kubectl exec -it
try to exec to a peer node which is up running.
once you log in,
cd to this directory
go here /etc/hyperledger/artifacts/keyfiles
then try to locate the tls file.
yes, let me try.. but its not able to find the keyfile... one difference from my earlier setup is msp paths are started by `keyfiles\...` as compared to previous implementation which was `{{fabricworkdir}}\keyfiles`
or, you can unzip the vars/certs.tgz file.
that file should be somewhere on your local machine where you run ansible script.
there should be a tgz file which contains every certs.
you can also do this
```kubectl get pvc```
and see if the pvc was created ok.
i can find all the files in this directory `/etc/hyperledger/artifacts/keyfiles`
checked pvc too.. created correctly
great.
now check the permission of these files.
0775
not sure what that means,
is it saying read for the world?
the file is there but for some reason the CA container can not find it.
this is very strange.
it has read write permissions
I have not tried this on Azure.
oo..ok
it seems to me that things should have worked.
is it probably due to cange in msp paths?
change in msp path?
what do you mean by that?
in your vars directory or subdir, you should have a file named fabric-pod.yml ,
now the commit has msp path starting by keyfiles\... as compared to previous implementation which was {{fabricworkdir}}\keyfiles
no, it should not.
the ca container setting explicitly configured the path.
copy that fabric-pod.yml file here, so that I can take a look.
ok
jarvis26 - Thu Aug 01 2019 23:56:56 GMT+0530 (India Standard Time).txt
not the template.
the one actually was used to set up your cluster.
you sent me the template.
oo..ok..
that is one from the source.
simply copy and paste here so that I do not need to download it.
should be in the keyfiles/run directory.
or keyfiles/fabric, I am not exactly sure.
its not there.. i did a find as well - `find . -name 'fabric-pod.yml'`
ah?
it should be on your local machine.
vars directory.
or sub directory.
found it..
The file is large so its not pasting directly.
jarvis26 - Fri Aug 02 2019 00:19:26 GMT+0530 (India Standard Time).txt
does your cluster have enough resources for all the pods to be running?
yes.. i have 14 vms, each with 7 gb memory
oh, ok.that should be enough.
so you have 3 ca nodes,
all of them doing the same thing?
yes..
one for each org..
can you try this command
```create deployment myca --image=hyperledger/fabric-ca:amd64-1.4.1```
this simply create a dep and I like to see what happens.
ok
`deployment.apps/myca created`
yeah. that means your cluster can run the image without problem.
oo..ok
I think I know what the problem is.
oo really..what
you have these lines
``` - { name: "FABRIC_CA_SERVER_DB_TYPE", value: "mysql" }
- { name: "FABRIC_CA_SERVER_DB_DATASOURCE", value: "root:password@tcp(ca1st-orgb-db-mysql:3306)/fabriccadb?parseTime=true" }
```
you changed something to have these lines in.
nope..just added those to have mysql as db for fabric ca
where did you make that change?
I added this section for creating a pod to run mysql and reference it later in the fabric ca pod - around line 107
i c.
well, I suggest you do this
just use the file network spec file that you have, stand up fabric network and see if everything is ok
once everything works as it should, you can go ahead make that change.
I am not quite sure what else you might have changed and where.
it is hard for me to debug the issue without knowing what changes that you have made and when and where.
ok..let me try it once
ok. yeah. I will be online and let me know how it goes.
I ran the setup again.. Now all the pods/containers are up properly.. But instantiation of chaincode is failing..
```2019-08-01 19:36:56.635 UTC [grpc] newHTTP2Transport -> DEBU 5242 grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams failed to receive the preface from client: read tcp 10.244.12.83:7050->10.240.0.16:56135: read: connection reset by peer"
2019-08-01 19:36:56.715 UTC [grpc] newHTTP2Transport -> DEBU 5243 grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams failed to receive the preface from client: read tcp 10.244.12.83:7050->10.240.0.10:52652: read: connection reset by peer"
2019-08-01 19:36:56.737 UTC [grpc] newHTTP2Transport -> DEBU 5244 grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams failed to receive the preface from client: read tcp 10.244.12.83:7050->10.240.0.4:63893: read: connection reset by peer"
2019-08-01 19:36:56.762 UTC [grpc] newHTTP2Transport -> DEBU 5245 grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams failed to receive the preface from client: read tcp 10.244.12.83:7050->10.240.0.11:63288: read: connection reset by peer"
2019-08-01 19:36:56.770 UTC [grpc] newHTTP2Transport -> DEBU 5246 grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams failed to receive the preface from client: read tcp 10.244.12.83:7050->10.240.0.13:63291: read: connection reset by peer"
2019-08-01 19:36:56.777 UTC [grpc] newHTTP2Transport -> DEBU 5247 grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams failed to receive the preface from client: read tcp 10.244.12.83:7050->10.240.0.9:52660: read: connection reset by peer"
2019-08-01 19:36:56.787 UTC [grpc] newHTTP2Transport -> DEBU 5248 grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams failed to receive the preface from client: read tcp 10.244.12.83:7050->10.240.0.17:52759: read: connection reset by peer"
2019-08-01 19:36:56.793 UTC [grpc] newHTTP2Transport -> DEBU 5249 grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams failed to receive the preface from client: read tcp 10.244.12.83:7050->10.240.0.5:52728: read: connection reset by peer"
2019-08-01 19:36:56.798 UTC [grpc] newHTTP2Transport -> DEBU 524a grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams failed to receive the preface from client: read tcp 10.244.12.83:7050->10.240.0.12:50255: read: connection reset by peer"
2019-08-01 19:36:57.054 UTC [grpc] newHTTP2Transport -> DEBU 524b grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams failed to receive the preface from client: read tcp 10.244.12.83:7050->10.240.0.7:51226: i/o timeout" ```
2019-08-01 19:36:57.219 UTC [grpc] newHTTP2Transport -> DEBU 524c grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams failed to receive the preface from client: read tcp 10.244.12.83:7050->10.240.0.8:57170: read: connection reset by peer"
```2019-08-01 19:36:56.635 UTC [grpc] newHTTP2Transport -> DEBU 5242 grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams failed to receive the preface from client: read tcp 10.244.12.83:7050->10.240.0.16:56135: read: connection reset by peer"
2019-08-01 19:36:56.715 UTC [grpc] newHTTP2Transport -> DEBU 5243 grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams failed to receive the preface from client: read tcp 10.244.12.83:7050->10.240.0.10:52652: read: connection reset by peer"
2019-08-01 19:36:56.737 UTC [grpc] newHTTP2Transport -> DEBU 5244 grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams failed to receive the preface from client: read tcp 10.244.12.83:7050->10.240.0.4:63893: read: connection reset by peer"
2019-08-01 19:36:56.762 UTC [grpc] newHTTP2Transport -> DEBU 5245 grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams failed to receive the preface from client: read tcp 10.244.12.83:7050->10.240.0.11:63288: read: connection reset by peer"
2019-08-01 19:36:56.770 UTC [grpc] newHTTP2Transport -> DEBU 5246 grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams failed to receive the preface from client: read tcp 10.244.12.83:7050->10.240.0.13:63291: read: connection reset by peer"
2019-08-01 19:36:56.777 UTC [grpc] newHTTP2Transport -> DEBU 5247 grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams failed to receive the preface from client: read tcp 10.244.12.83:7050->10.240.0.9:52660: read: connection reset by peer"
2019-08-01 19:36:56.787 UTC [grpc] newHTTP2Transport -> DEBU 5248 grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams failed to receive the preface from client: read tcp 10.244.12.83:7050->10.240.0.17:52759: read: connection reset by peer"
2019-08-01 19:36:56.793 UTC [grpc] newHTTP2Transport -> DEBU 5249 grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams failed to receive the preface from client: read tcp 10.244.12.83:7050->10.240.0.5:52728: read: connection reset by peer"
2019-08-01 19:36:56.798 UTC [grpc] newHTTP2Transport -> DEBU 524a grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams failed to receive the preface from client: read tcp 10.244.12.83:7050->10.240.0.12:50255: read: connection reset by peer"
2019-08-01 19:36:57.054 UTC [grpc] newHTTP2Transport -> DEBU 524b grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams failed to receive the preface from client: read tcp 10.244.12.83:7050->10.240.0.7:51226: i/o timeout"
2019-08-01 19:36:57.219 UTC [grpc] newHTTP2Transport -> DEBU 524c grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams failed to receive the preface from client: read tcp 10.244.12.83:7050->10.240.0.8:57170: read: connection reset by peer" ```
```2019-08-01 19:36:56.798 UTC [grpc] newHTTP2Transport -> DEBU 524a grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams failed to receive the preface from client: read tcp 10.244.12.83:7050->10.240.0.12:50255: read: connection reset by peer"
2019-08-01 19:36:57.054 UTC [grpc] newHTTP2Transport -> DEBU 524b grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams failed to receive the preface from client: read tcp 10.244.12.83:7050->10.240.0.7:51226: i/o timeout"
2019-08-01 19:36:57.219 UTC [grpc] newHTTP2Transport -> DEBU 524c grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams failed to receive the preface from client: read tcp 10.244.12.83:7050->10.240.0.8:57170: read: connection reset by peer" ```
This is the error at the orderer
you've made good progress.
it seems to me that your peer may not be able to create chaincode container.
not quite sure. it could be security issues.
there is one thing we can try to confirm that.
you can remove this network,
I mean fabric network by running the destroy command.
once your cluster is clean, you can change the tls: true in your network spec file to false.
just to try the same thing and see if you can instantiate.
Has joined the channel.
Thanks..
This network was setup with TLS disabled
:'(
Thanks
I started this network wit tls disabled
hi team does anybody faced the same above issue ?
mallikarjunasai995 - Fri Aug 02 2019 18:03:06 GMT+0530 (India Standard Time).txt
does anybody faced the same cello-engine-container ... api_engine module not found error ?
this was the precise error that am facing not sure why this even though set the path using PYTHONPATH
File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module
cello-engine-tasks | return _bootstrap._gcd_import(name[level:], package, level)
cello-engine-tasks | File "
@hightall the initContainer for peer pod, what image do you use for it? fabric-tools?
Has joined the channel.
Hi, will be on trip and absent for this week's meeting. Add one items for @Luke_Chen on the coming TSC quarterly report: https://wiki.hyperledger.org/display/cello/2019-08-09
Has joined the channel.
Hi...In the latest code, I see that a couple of k8s specific configurations are removed -such as `mem_req, mem_limit, cpu_req, cpu_limit`. How is the space within the cluster handled now?
Has joined the channel.
@jarvis26 which part did you refer to?
In k8s settings within vars/bc1st.yml
@jarvis26 you will find a resource.yml file at the same directory, where you can specify resource limits.
more flexible.
by component type
Has joined the channel.
Hi guys,
I have a problem with installing HF Cello.
My system:
Ubuntu 18.04.3 LTS
Docker: 19.03.1
Docker Compose: 1.24.1
Procedures I made:
- Got v0.9 from Github (https://github.com/hyperledger/cello/archive/v0.9.0.zip)
- run make setup-master (2-times)
- run chmod 777 -R /opt/cello
- run SERVER_PUBLIC_IP=192.168.1.10 make start
- first docker ps output:
`make[1]: Leaving directory '/home/user/cello-0.9.0'
user@ubuntu:~/cello-0.9.0$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
89af693d7726 itsthenetwork/nfs-server-alpine:9 "/usr/bin/nfsd.sh" 15 seconds ago Up 13 seconds 0.0.0.0:2049->2049/tcp cello-nfs
b15b36990189 hyperledger/cello-user-dashboard "/bin/sh -c 'ln -sf …" 20 seconds ago Up 17 seconds 0.0.0.0:8081->8081/tcp cello-user-dashboard
1dbe7fbbb436 hyperledger/cello-engine "python restserver.py" 27 seconds ago Up 21 seconds 0.0.0.0:80->80/tcp cello-engine
ad7ae1a93cc5 mongo:3.4.10 "docker-entrypoint.s…" 27 seconds ago Up 20 seconds 27017/tcp cello-dashboard_mongo
e00d4b9f638b mongo:3.4.10 "docker-entrypoint.s…" 27 seconds ago Up 22 seconds 127.0.0.1:27017-27018->27017-27018/tcp cello-mongo
293e661cc12a hyperledger/cello-operator-dashboard "/bin/sh -c '/etc/in…" 27 seconds ago Up 24 seconds 0.0.0.0:8080->8080/tcp cello-operator-dashboard
f6227f47d182 hyperledger/cello-watchdog "python watchdog.py" 27 seconds ago Up 4 seconds cello-watchdog
1606c55366a6 rabbitmq "docker-entrypoint.s…" 27 seconds ago Up 25 seconds 4369/tcp, 5671-5672/tcp, 25672/tcp cello-dashboard_rabbitmq`
- after a while:
`user@ubuntu:~/cello-0.9.0$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
89af693d7726 itsthenetwork/nfs-server-alpine:9 "/usr/bin/nfsd.sh" 37 seconds ago Up 34 seconds 0.0.0.0:2049->2049/tcp cello-nfs
1dbe7fbbb436 hyperledger/cello-engine "python restserver.py" 49 seconds ago Up 43 seconds 0.0.0.0:80->80/tcp cello-engine
ad7ae1a93cc5 mongo:3.4.10 "docker-entrypoint.s…" 49 seconds ago Up 42 seconds 27017/tcp cello-dashboard_mongo
e00d4b9f638b mongo:3.4.10 "docker-entrypoint.s…" 49 seconds ago Up 44 seconds 127.0.0.1:27017-27018->27017-27018/tcp cello-mongo
293e661cc12a hyperledger/cello-operator-dashboard "/bin/sh -c '/etc/in…" 49 seconds ago Up 45 seconds 0.0.0.0:8080->8080/tcp cello-operator-dashboard
f6227f47d182 hyperledger/cello-watchdog "python watchdog.py" 49 seconds ago Restarting (1) Less than a second ago cello-watchdog
1606c55366a6 rabbitmq "docker-entrypoint.s…" 49 seconds ago Up 46 seconds 4369/tcp, 5671-5672/tcp, 25672/tcp cello-dashboard_rabbitmq`
- container cello-user-dashboard, cello-watchdog crashed immediately - cello-operator-dashboard is not available at port 8080
Logs are attached.
I have no idea what I did wrong. I cannot start the master. V0.8 and the latest commit are also not working.
I hope you could help me, thanks.
log_cello-user-dashboard.txt
log_cello-watchdog.txt
hey guys, on doing make start Im getting the following error:
```
WARNING: The DOCKER_HOST variable is not set. Defaulting to a blank string.
Pulling api-engine (hyperledger/cello-api-engine:)...
ERROR: The image for the service you're trying to recreate has been removed. If you continue, volume data could be lost. Consider backing up your data before continuing.
Continue with the new image? [yN]y
Pulling api-engine (hyperledger/cello-api-engine:)...
ERROR: pull access denied for hyperledger/cello-api-engine, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
Makefile:208: recipe for target 'start-docker-compose' failed
make: *** [start-docker-compose] Error 1
```
hey guys, on doing make start Im getting the following error:
```
if [ "docker-compose" = "docker-compose" ]; then \
make start-docker-compose; \
else \
make start-k8s; \
fi
make[1]: Entering directory '/home/vagrant/cello'
docker-compose -f bootup/docker-compose-files/docker-compose.yml up -d --force-recreate
WARNING: The DOCKER_HOST variable is not set. Defaulting to a blank string.
Pulling api-engine (hyperledger/cello-api-engine:)...
ERROR: The image for the service you're trying to recreate has been removed. If you continue, volume data could be lost. Consider backing up your data before continuing.
Continue with the new image? [yN]y
Pulling api-engine (hyperledger/cello-api-engine:)...
ERROR: pull access denied for hyperledger/cello-api-engine, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
Makefile:208: recipe for target 'start-docker-compose' failed
make[1]: *** [start-docker-compose] Error 1
make[1]: Leaving directory '/home/vagrant/cello'
Makefile:211: recipe for target 'start' failed
make: *** [start] Error 2
Makefile:208: recipe for target 'start-docker-compose' failed
make: *** [start-docker-compose] Error 1
```
Hey, hello again, on the cello dashboard, when adding an agent, there is a file with the configuration, what are the posible parameters? I cant find any reference on the documentation, Im working with the latest
@tongli Hi.. If I want to add a new peer in an existing organization, how should I do it? Is it possible after the network is already up with the certificates generated via cryptogen?
@jarvis26 currently ansible agent does not support adding new node
You can set up a peer node, then use fabric config update to add new node
Has joined the channel.
Hi Guys, what is recomended for performance in a rest service with high troughput. Create a single gateway and contract instance from application startup or create for every request?
@tongli ok..thanks
How do I ensure prevention of data loss. Currently, the chain data resides in production directory which gets created within the docker container within the pod. Should the storage be mapped outside the container so as to ensure persistence?
@jarvis26 that is right. In the new k8s operator, we create stateful set for that. Coming up in couple of weeks
Has joined the channel.
@tongli this is a security fix, it needs tested and merged: https://gerrit.hyperledger.org/r/#/c/cello/+/33299/
also, please see https://gerrit.hyperledger.org/r/#/c/cello-analytics/+/33298/ and https://gerrit.hyperledger.org/r/#/c/cello/+/33297/
this is all to support the move from Gerrit to GitHub and Jenkins to... something
So, that would entail kafka and zookeeper nodes also be created as stateful sets?
:cello:
https://gerrit.hyperledger.org/r/#/c/cello/+/33173/
@baohua could you grab my last three commits? https://gerrit.hyperledger.org/r/q/status:open+owner:ry%2540linux.com
@baohua could you grab my last three commits? https://gerrit.hyperledger.org/r/q/status:open+owner:ry%2540linux.com please?
+2, and merged one with tong's support together.
+2, and merged one with tong's support together. @rjones
https://gerrit.hyperledger.org/r/#/c/cello-analytics/+/33298/
Has left the channel.
@rjones interested to know how to make the cello icon in msg?
:cello:
`:cello:`
there is one for each project now
oh. cool.
:fabric: :indy:
:cello:
Cool!!
Has left the channel.
Has joined the channel.
https://lgtm.com/projects/g/hyperledger/cello/?mode=list
https://gerrit.hyperledger.org/r/#/c/cello/+/33441
here the hyperledger community will migrate from the gerrit CI to the github one. Please pay attention to the latest progress at #cicd channel.
@here the hyperledger community will migrate from the gerrit CI to the github one. Please pay attention to the latest progress at #cicd channel.
https://gerrit.hyperledger.org/r/#/c/cello-analytics/+/33747/
Has left the channel.
@baohua Hello all. Some help please. I have a 2 org 4 node network running on Kubernetes
# peer channel list
2019-09-30 12:51:35.509 UTC [channelCmd] InitCmdFactory -> INFO 001 Endorser and orderer connections initialized
Channels peers has joined:
mychannel
# peer chaincode instantiate -C mychannel -n koinearthcc -v 1.2 -c '{"Args":["init"]}' -P "$POLICY 7050"
2019-09-30 12:43:50.551 UTC [chaincodeCmd] InitCmdFactory -> INFO 001 Retrieved channel (mychannel) orderer endpoint: orderer1-koinearth.koinearth:7050
2019-09-30 12:43:50.553 UTC [chaincodeCmd] checkChaincodeCmdParams -> INFO 002 Using default escc
2019-09-30 12:43:50.553 UTC [chaincodeCmd] checkChaincodeCmdParams -> INFO 003 Using default vscc
Error: got unexpected status: NOT_FOUND -- channel does not exist
cannot figure this out. I have checked channel joining status on all 4 nodes and all the peer channel list commands show that the channel has been joined
what am I doing wrong
@tongli
R u using ansible agent doing the setting up?
@tongli I have a couple cello changes I need merged: https://gerrit.hyperledger.org/r/q/%2522%255Bin-68%255D%2522+status:open
I'm trying to help get you guys pointed in the right direction for the migration
Here is an example fix of the move to AZP: https://jira.hyperledger.org/browse/CE-693 https://gerrit.hyperledger.org/r/c/cello/+/33843 https://dev.azure.com/Hyperledger/Cello/_build/results?buildId=554&view=results
@rjones thx
@tongli thank you. Let me know when you're ready to move forward on https://jira.hyperledger.org/browse/CE-703 - https://gerrit.hyperledger.org/r/c/ci-management/+/33856 is part of that work
@rjones for some reason I can only do +1 for this https://gerrit.hyperledger.org/r/c/ci-management/+/33856
is it because I am not the committer for the repo?
if we merge this patch, does it mean we are starting building cello on Azure?
yes, please do not merge that patch. please read the JIRA - there is an order to do this
@tongli
ok, @rjones
@baohua please review this dockerfile for building k8s fabric operator agent on docker hub. https://gerrit.hyperledger.org/r/c/cello/+/33889
@rjones good afternoon, I need to build two new images for cello. can you let me know what process should I follow?
+2!
@tongli are you ready to move to github or not? Until cello and cello-analytics move to GitHub, use your current process.
You will be building technical debt but the world is full of trade offs. @baohua @tongli
Building debt as you make changes to Jenkins I mean
@rjones we need to still resolve these security warning before we move , right?
I do not expect that all the builds will be working perfectly once we move over, but I think we can probably move then gradually making changes to the build scripts.
@tongli no, the security warnings can wait until you move. those issues are orthogonal.
@rjones in that case, can we now move over?
Next couple of days I can watch and try to fix any issues we might have.
we do not have a lot of patches since many of the contributors are in China and just coming back from their national holidays.
OK. @tongli please merge or abandon any in progress patches; also, please merge the change in cello-analytics
https://gerrit.hyperledger.org/r/c/cello-analytics/+/33747
@tongli @baohua I have moved everything to github I have not enabled CI yet please don't use Gerrit thanks
@tongli @baohua I have moved everything to github I have enabled CI yet please don't use Gerrit thanks
@rjones oh, this is a big change, i wanna to let everyone know that immediately. @hightall @Luke_Chen @all
@rjones oh, this is a big change, want to let everyone notice that immediately. @hightall @Luke_Chen @all
@rjones oh, this is a big change, would like to let everyone notice that immediately. @hightall @Luke_Chen @all
yeah. let's take couple of days, I will let everyone know in wechat channel as well, not to check in anything.
@rjones Do you know if there's any guideline for the new code contribution and ci usage process?
@baohua it is in GitHub now - https://github.com/hyperledger/cello/
@rjones so we can start doing the pull request to the repo?
(and pretend that gerrit never existed?)
@rjones seems all our patch have to be reviewed by Baohua, is that how it was setup currently? I tried one pull request, it says it required a code owner to review it.
@rjones got some errors, may need your help.
I did not create fork but used the web ui created the pull request, but some how it produced a new branch.
I could not push to that branch since it gives me error like this.
```To https://github.com/hyperledger/cello.git
! [remote rejected] litong01-patch-1 -> litong01-patch-1 (protected branch hook declined)
error: failed to push some refs to 'https://github.com/hyperledger/cello.git'```
and I could not remove the branch and could not get rid of the pull request,
it seems that I am stuck with it, can you provide a bit help? Thanks.
@tongli fixed
```mba:sawtooth-pbft ry$ git remote -v
GH git@github.com:hyperledger/sawtooth-pbft.git (fetch)
GH git@github.com:hyperledger/sawtooth-pbft.git (push)
origin git@github.com:5daa1fac63ae83fd1b3fca8f219a6851/sawtooth-pbft.git (fetch)
origin git@github.com:5daa1fac63ae83fd1b3fca8f219a6851/sawtooth-pbft.git (push)
mba:sawtooth-pbft ry$ git fetch GH master && git rebase -i FETCH_HEAD && git push -f origin
From github.com:hyperledger/sawtooth-pbft
* branch master -> FETCH_HEAD
Successfully rebased and updated refs/heads/master.
Everything up-to-date
mba:sawtooth-pbft ry$
``` this is how I update things personally
```mba:indy-node ry$ git fetch GH master && git rebase -i FETCH_HEAD && git push -f origin
remote: Enumerating objects: 4, done.
remote: Counting objects: 100% (4/4), done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 4 (delta 0), reused 1 (delta 0), pack-reused 0
Unpacking objects: 100% (4/4), done.
From github.com:hyperledger/indy-node
* branch master -> FETCH_HEAD
a1148a49..067be83e master -> GH/master
Successfully rebased and updated refs/heads/master.
Enumerating objects: 4, done.
Counting objects: 100% (4/4), done.
Delta compression using up to 4 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 850 bytes | 850.00 KiB/s, done.
Total 3 (delta 1), reused 1 (delta 0)
remote: Resolving deltas: 100% (1/1), completed with 1 local object.
To github.com:5daa1fac63ae83fd1b3fca8f219a6851/indy-node.git
+ cc5e43c8...0d258b0e master -> master (forced update)
mba:indy-node ry$
``` this is how I update things personally
@rjones thanks very much.
@tongli this part was not obvious:
```mba:indy-node ry$ git remote -v
GH git@github.com:hyperledger/indy-node.git (fetch)
GH git@github.com:hyperledger/indy-node.git (push)
origin git@github.com:5daa1fac63ae83fd1b3fca8f219a6851/indy-node.git (fetch)
origin git@github.com:5daa1fac63ae83fd1b3fca8f219a6851/indy-node.git (push)
mba:indy-node ry$
```
@tongli i summarize my understanding of the new process, feel free to revise: https://github.com/hyperledger/cello/wiki/Contribution-Process
@rjones we figure out a pretty good way to do this, I think we are good now. Thx
@rjones good afternoon, since we’ve moved over, what is the process getting our images built and posting them onto docker hub ?
Do we need you to create docker hub repo for each image first?
@tongli I don't think it will be any different - we can just add the auto build stuff like existed before, right?
@tongli check out:
https://cloud.docker.com/u/hyperledger/repository/docker/hyperledger/cello-dashboard/builds
@here Created the meeting agenda page for this week's meeting, feel free to add items: https://wiki.hyperledger.org/display/cello/2019-10-18
Thanks!
Has left the channel.
Has joined the channel.
HI
anybody help me to understand how to setup the fabric network using hyperledger cello
Has joined the channel.
@AbhijeetSamanta would you like to read the doc of cello? https://cello.readthedocs.io/en/master/
Has left the channel.
Has joined the channel.
Hello! Do the cello support fabric 1.4.3?
Has joined the channel.
@baohua I went through the docs https://cello.readthedocs.io/en/master/. I found the section where it says under Getting started section that For beginners, it is highly recommended to read the Setup first. When i went to setup docs, the links are not present for 2nd Steps step
Has left the channel.
Oh, there missed some docs. Would you like to create a github issue for this? I think we should add it soon.
@hightall @tongli @Luke_Chen https://github.com/hyperledger/cello/pull/44
@baohua +2
https://github.com/hyperledger/cello/pull/57
https://wiki.hyperledger.org/display/cello/2019-11-01
Checked the community calendar, and set a doodle for meeting time voting. Feel free to let me know if there's other suggestion. Thanks! https://doodle.com/poll/k8bmnm5kmrsah5pv
Updated the contribution process, now add the sync operation, too: https://github.com/hyperledger/cello/wiki/Contribution-Process
Hi All,
Can we add new org or peer nodes into existing consortium through cello ?
https://github.com/hyperledger/cello/pull/67
Please help review and comments: https://wiki.hyperledger.org/display/HYP/2019+Q4+Hyperledger+Cello
I have created the issue. Here is the link https://github.com/hyperledger/cello/issues/68
https://docs.google.com/document/d/1Dw6cEKaul6FenORNkDcxvPDDKwpl0A5EmcJBlqAWJoU/edit#heading=h.eorq7j6q432u
Hello everyone, I am really sorry for not being able to attend the meeting today. There was some misinterpretation from my side regarding the time of the meeting. I would like to demo the work I have done in the next meeting or any earlier if possible. Thanks
Thanks, manak!
Welcome, Sorry again for missing the meeting.
Welcome, Sorry again for missing the meeting. :slight_smile:
Has joined the channel.
Hi @tongli Which k8s version will support to Cello?
Hi All Which k8s version will support to Cello?
Hi All,
Hi All, I am new to hyperledger cello. I am trying to implement it to our new project in production. However I am confused with some step like 1) which k8s version cello support as k8s 1.12 is deprecated 2) how will I connect with worker node with master node in k8s .
Hi, I have new query as I am trying to pull docker images though command "make dockerhub-pull", however its show some images not available in docker hub, please let me know how we can fixed the issue.
screenshot of log
@AbhijeetSamanta if you are using k8s to stand up your fabric network and using ansible agent, then the k8s 1.1 or newer are all supported.
@AbhijeetSamanta looks that you are using the cello dashboard, which currently is not using ansible agent.
please see the doc here about ansible agent.
https://github.com/hyperledger/cello/blob/master/docs/agents/ansible.md
Thanks let me try to deploy it
@AbhijeetSamanta if you have any issues, please contact me here.
I wonder what k8s cluster do you have.
IKS, Azure or aws?
I am using google cloud
ok.
should work.
I tested awhile back. but now I do not have paid account so that I can not actually try it.
yes sure thanks for help I am going though document now
I remember that the default PVC on google cloud does not allow READWRITEMULTI.
you probably can setup a PV class like NFS to allow PVC to be READWRITE by multiple containers
if I will use aws then it will work fine right?
things may have been changed on google cloud , but I am not sure.
ok
I had a lot of issues with AWS k8s,
stich with google first probably
stick
ok same here I also faced it aws k8s
once you get a handle on this, you can try on different k8s.
it worked great ok IKS.
IBM k8s.
ok but I don't have access to it
but IBM K8S mostly just like Google k8s. the only difference is the default PV restriction on google cloud
but that was almost like a year ago. things may have been changed.
also I don't know pricing of IKS
yeah, if you have google, try it first.
we can get it through.
ok thanks
thanks for your replies and help
:thumbsup:
the only thing I am aware of is the PV restriction. even that we can resolve by adding our own pv class if the restriction is still there.
ok
---
fabric: {
# options are "solo", "kafka", "etcdraft" (etcdraft support from 2.0.0)
# for consensus_type, default is kafka
consensus_type: "etcdraft",
# options are "goleveldb", "CouchDB", default is goleveldb
peer_db: "CouchDB",
# options are "true", "false", "client" default is true
tls: false,
generate_certs: true,
logging_level: "DEBUG",
metrics: false,
k8s: {
exposeserviceport: False,
storageclass: "default",
storagecapacity: "15Gi",
shared_storage: false
},
# Peer environment variables to override core.yaml values
# variables should be in the peer readable format like shown below
peersettings: {
CORE_LEDGER_STATE_COUCHDBCONFIG_WARMINDEXESAFTERNBLOCKS: "10"
},
# Orderer configuration and environment variables to override
# configtx.yaml and orderer.yaml values. ordererenv will take the
# environment variables in the orderer readable format for example:
# ORDERER_GENERAL_TLS_ENABLED=true
orderersettings: {
config: {
BatchTimeout: 2s,
BatchSize: {
MaxMessageCount: 500,
AbsoluteMaxBytes: "98 MB",
PreferredMaxBytes: "512 KB"
}
},
ordererenv: {
}
},
kafkasettings: {
MESSAGE_MAX_BYTES: "103809024",
MAX_REQUEST_SIZE: "104857600",
REPLICA_FETCH_MAX_BYTES: "103809024",
UNCLEAN_LEADER_ELECTION_ENABLE: "true",
MIN_INSYNC_REPLICAS: "2",
DEFAULT_REPLICATION_FACTOR: "3"
},
raftsettings: {
Options: {
TickInterval: "500ms",
ElectionTick: 10,
HeartbeatTick: 1,
MaxInflightBlocks: 5,
SnapshotIntervalSize: "100 MB"
}
},
# The following section defines how the fabric network is going to be made up
# cas indicates certificate authority containers
# peers indicates peer containers
# orderers indicates orderer containers
# kafka indicates kafka containers
# all names must be in lower case. Numeric characters cannot be used to start
# or end a name. Character dot (.) can only be used in names of peers and orderers.
network: {
fabric001: {
cas: ["ca1st.orga"],
peers: ["anchor@peer1st.orga", "anchor@peer1st.orgb"],
orderers: ["orderer1st.orgc", "orderer1st.orgd"]
},
fabric002: {
cas: ["ca1st.orgb"],
peers: ["worker@peer2nd.orga", "worker@peer2nd.orgb"],
orderers: ["orderer2nd.orgc", "orderer2nd.orgd"]
},
fabric003: {
cas: ["ca1st.orgc", "ca1st.orgd"],
peers: ["worker@peer3rd.orga", "worker@peer3rd.orgb"],
orderers: []
}
},
# the container tag for main fabric components such as orderer, peer,
# tools and chaincode. If left blank, then fabric components will be built
# from the latest fabric source code with the combination of the refspec
# specified above. If it is not empty, then its values must be the full
# container tag like the following:
# x86_64-1.1.0-alpha # alpha release
# DAILY_STABLE # daily stable
# DEV_STABLE # dev stable
# s390x-0.4.2 # release 0.4.2 for s390x
# 048c91eb47812582f36665fbea3d7e0e68e396bf # commit build
# x86_64-1.0.0-snapshot-8d3275f # snapshot build
baseimage_tag: "amd64-1.4.3",
# The container tag for accessory fabric components such as ca, kafka,
# zookeeper and couchdb. It has to be the full tag just like the
# baseimage_tag
helper_tag: "amd64-1.4.3",
ca: { admin: "admin", adminpw: "adminpw", image_tag: "amd64-1.4.3" },
# Only use this field when you have a docker repository.
repo: {
# the url defines docker hub project access point
# official hub - url: "hyperledger/",
# fabric build hub - url: "nexus3.hyperledger.org:10001/hyperledger/",
url: "hyperledger/",
username: "",
password: "",
# locations to download fabric binaries. This url should point to a
# compressed such as tar or zip file which contains necessary binaries
# such as configtxgen, configtxlator, cryptogen etc.
bin: "https://nexus.hyperledger.org/content/repositories/releases/org/hyperledger/fabric/hyperledger-fabric/linux-amd64-1.4.3/hyperledger-fabric-linux-amd64-1.4.3.tar.gz"
}
}
keep me posted.
yes sure
hi @tongli could you please let me know how much memory and CPU need for master and worker node for run Cello with K8s
@AbhijeetSamanta just to have them running without doing a lot of txs, you can give 0.5 CPU and 1GB ram.
Hi @tongli @baohua I am trying to implement the Cello in google cloud with one master and 2 worker node. I am running the below command on master node(ubuntu 16.04 lts)) however I am getting error as "docker run --rm -v $(pwd)/vars:/opt/agent/vars hyperledger/cello-agent-ansible \ ansible-playbook -i hosts -e "mode=apply env=bc1st" setupfabric.yml" . Please help me to understand what wrong I am doing here. P.S : I have change the bc1st.yaml file. please see the below file bc1st.yaml.
Hi @tongli @baohua I am trying to implement the Cello in google cloud with one master and 2 worker node. I am running the below command on master node(ubuntu 16.04 lts)) however I am getting error as "docker run --rm -v $(pwd)/vars:/opt/agent/vars hyperledger/cello-agent-ansible \ ansible-playbook -i hosts -e "mode=apply env=bc1st" setupfabric.yml" . Please help me to understand what wrong I am doing here. P.S : I have change the bc1st.yaml file. please see the attached file bc1st.yaml.
---
fabric: {
consensus_type: "etcdraft",
peer_db: "CouchDB",
tls: false,
generate_certs: true,
logging_level: "DEBUG",
metrics: false,
k8s: {
exposeserviceport: False,
storageclass: "default",
storagecapacity: "15Gi",
shared_storage: false
},
peersettings: {
CORE_LEDGER_STATE_COUCHDBCONFIG_WARMINDEXESAFTERNBLOCKS: "10"
},
orderersettings: {
config: {
BatchTimeout: 2s,
BatchSize: {
MaxMessageCount: 500,
AbsoluteMaxBytes: "98 MB",
PreferredMaxBytes: "512 KB"
}
},
ordererenv: {
}
},
kafkasettings: {
MESSAGE_MAX_BYTES: "103809024",
MAX_REQUEST_SIZE: "104857600",
REPLICA_FETCH_MAX_BYTES: "103809024",
UNCLEAN_LEADER_ELECTION_ENABLE: "true",
MIN_INSYNC_REPLICAS: "2",
DEFAULT_REPLICATION_FACTOR: "3"
},
raftsettings: {
Options: {
TickInterval: "500ms",
ElectionTick: 10,
HeartbeatTick: 1,
MaxInflightBlocks: 5,
SnapshotIntervalSize: "100 MB"
}
},
network: {
fabric001: {
cas: ["ca1st.orga"],
peers: ["anchor@peer1st.orga", "anchor@peer1st.orgb"],
orderers: ["orderer1st.orgc", "orderer1st.orgd"]
},
fabric002: {
cas: ["ca1st.orgb"],
peers: ["worker@peer2nd.orga", "worker@peer2nd.orgb"],
orderers: ["orderer2nd.orgc", "orderer2nd.orgd"]
},
fabric003: {
cas: ["ca1st.orgc", "ca1st.orgd"],
peers: ["worker@peer3rd.orga", "worker@peer3rd.orgb"],
orderers: []
}
},
baseimage_tag: "amd64-1.4.3",
helper_tag: "amd64-1.4.3",
ca: { admin: "admin", adminpw: "adminpw", image_tag: "amd64-1.4.3" },
repo: {
url: "hyperledger/",
username: "",
password: "",
bin: "https://nexus.hyperledger.org/content/repositories/releases/org/hyperledger/fabric/hyperledger-fabric/linux-amd64-1.4.3/hyperledger-fabric-linux-amd64-1.4.3.tar.gz"
}
}
---
fabric: {
# options are "solo", "kafka", "etcdraft" (etcdraft support from 2.0.0)
# for consensus_type, default is kafka
consensus_type: "etcdraft",
# options are "goleveldb", "CouchDB", default is goleveldb
peer_db: "CouchDB",
# options are "true", "false", "client" default is true
tls: false,
generate_certs: true,
logging_level: "DEBUG",
metrics: false,
k8s: {
exposeserviceport: False,
storageclass: "default",
storagecapacity: "15Gi",
shared_storage: false
},
# Peer environment variables to override core.yaml values
# variables should be in the peer readable format like shown below
peersettings: {
CORE_LEDGER_STATE_COUCHDBCONFIG_WARMINDEXESAFTERNBLOCKS: "10"
},
# Orderer configuration and environment variables to override
# configtx.yaml and orderer.yaml values. ordererenv will take the
# environment variables in the orderer readable format for example:
# ORDERER_GENERAL_TLS_ENABLED=true
orderersettings: {
config: {
BatchTimeout: 2s,
BatchSize: {
MaxMessageCount: 500,
AbsoluteMaxBytes: "98 MB",
PreferredMaxBytes: "512 KB"
}
},
ordererenv: {
}
},
kafkasettings: {
MESSAGE_MAX_BYTES: "103809024",
MAX_REQUEST_SIZE: "104857600",
REPLICA_FETCH_MAX_BYTES: "103809024",
UNCLEAN_LEADER_ELECTION_ENABLE: "true",
MIN_INSYNC_REPLICAS: "2",
DEFAULT_REPLICATION_FACTOR: "3"
},
raftsettings: {
Options: {
TickInterval: "500ms",
ElectionTick: 10,
HeartbeatTick: 1,
MaxInflightBlocks: 5,
SnapshotIntervalSize: "100 MB"
}
},
# The following section defines how the fabric network is going to be made up
# cas indicates certificate authority containers
# peers indicates peer containers
# orderers indicates orderer containers
# kafka indicates kafka containers
# all names must be in lower case. Numeric characters cannot be used to start
# or end a name. Character dot (.) can only be used in names of peers and orderers.
network: {
fabric001: {
cas: ["ca1st.orga"],
peers: ["anchor@peer1st.orga", "anchor@peer1st.orgb"],
orderers: ["orderer1st.orgc", "orderer1st.orgd"]
},
fabric002: {
cas: ["ca1st.orgb"],
peers: ["worker@peer2nd.orga", "worker@peer2nd.orgb"],
orderers: ["orderer2nd.orgc", "orderer2nd.orgd"]
},
fabric003: {
cas: ["ca1st.orgc", "ca1st.orgd"],
peers: ["worker@peer3rd.orga", "worker@peer3rd.orgb"],
orderers: []
}
},
# the container tag for main fabric components such as orderer, peer,
# tools and chaincode. If left blank, then fabric components will be built
# from the latest fabric source code with the combination of the refspec
# specified above. If it is not empty, then its values must be the full
# container tag like the following:
# x86_64-1.1.0-alpha # alpha release
# DAILY_STABLE # daily stable
# DEV_STABLE # dev stable
# s390x-0.4.2 # release 0.4.2 for s390x
# 048c91eb47812582f36665fbea3d7e0e68e396bf # commit build
# x86_64-1.0.0-snapshot-8d3275f # snapshot build
baseimage_tag: "amd64-1.4.3",
# The container tag for accessory fabric components such as ca, kafka,
# zookeeper and couchdb. It has to be the full tag just like the
# baseimage_tag
helper_tag: "amd64-1.4.3",
ca: { admin: "admin", adminpw: "adminpw", image_tag: "amd64-1.4.3" },
# Only use this field when you have a docker repository.
repo: {
# the url defines docker hub project access point
# official hub - url: "hyperledger/",
# fabric build hub - url: "nexus3.hyperledger.org:10001/hyperledger/",
url: "hyperledger/",
username: "",
password: "",
# locations to download fabric binaries. This url should point to a
# compressed such as tar or zip file which contains necessary binaries
# such as configtxgen, configtxlator, cryptogen etc.
bin: "https://nexus.hyperledger.org/content/repositories/releases/org/hyperledger/fabric/hyperledger-fabric/linux-amd64-1.4.3/hyperledger-fabric-linux-amd64-1.4.3.tar.gz"
}
}
error screenshot
hmmm. you can change the baseimage_tag just to be this
baseimage_tag: "1.4.3"
and ca also to be the same thing.
can you also put the more logs here?
@AbhijeetSamanta from the error, it looks like that the peer container in the peer pod can not access the dind container which is how the chaincode container will be created. It seems to me you are not experiences the pvc issues, if indeed that is the case, that is really good.
can you confirm that?
not quite sure why there is permission denied error.
but want to point it out that when you use etcdraft as consesus method, you have to use tls to be true. tls to be false wont work for etcdraft.
so here is a network spec file that you can try.
```---
peer_db: "goleveldb"
tls: true
logging_level: "ERROR"
exposeMetricsPort: true
consensus_type: "etcdraft"
k8s:
exposeserviceport: True
storageclass: "default"
storagecapacity: "20Gi"
shared_storage: False
mem_req: "1Gi"
mem_limit: "8Gi"
cpu_req: "1"
cpu_limit: "8"
raftsettings:
Options:
TickInterval: "500ms"
ElectionTick: 10
HeartbeatTick: 1
MaxInflightBlocks: 5
SnapshotIntervalSize: "20 MB"
orderersettings:
config:
BatchTimeout: 1s
BatchSize:
MaxMessageCount: 500
AbsoluteMaxBytes: "98 MB"
PreferredMaxBytes: "256 KB"
ordererenv:
ORDERER_GENERAL_PROFILE_ENABLED: false
peersettings:
CORE_PEER_PROFILE_ENABLED: false
network:
orderers:
- orderer1.ordererorg
- orderer2.ordererorg
- orderer3.ordererorg
peers:
- worker@peer1.org0
- worker@peer2.org0
- worker@peer1.org1
- worker@peer2.org1
baseimage_tag: "1.4.3"
helper_tag: "0.4.15"
ca:
admin: "admin"
adminpw: "adminpw"
image_tag: "1.4.3"
repo:
url: "hyperledger/"
username: ""
password: ""
bin: "https://nexus.hyperledger.org/content/repositories/releases/org/hyperledger/fabric/hyperledger-fabric/linux-amd64-1.4.3/hyperledger-fabric-linux-amd64-1.4.3.tar.gz"
resourceQuota:
# Each section can be any valid k8s resource limit parameters
peer:
requests:
memory: "1Gi"
cpu: "1"
limits:
memory: "8Gi"
cpu: "8"
orderer:
requests:
memory: "1Gi"
cpu: "1"
limits:
memory: "8Gi"
cpu: "8"```
here is the modified bc1st.yml file. I run it in IBM IKS, everyting worked fine.
```fabric: {
consensus_type: "etcdraft",
peer_db: "CouchDB",
tls: true,
generate_certs: true,
logging_level: "DEBUG",
metrics: false,
k8s: {
exposeserviceport: false,
storageclass: "default",
storagecapacity: "20Gi",
shared_storage: false
},
peersettings: {
CORE_LEDGER_STATE_COUCHDBCONFIG_WARMINDEXESAFTERNBLOCKS: "10"
},
orderersettings: {
config: {
BatchTimeout: 2s,
BatchSize: {
MaxMessageCount: 500,
AbsoluteMaxBytes: "98 MB",
PreferredMaxBytes: "512 KB"
}
},
ordererenv: {}
},
kafkasettings: {
MESSAGE_MAX_BYTES: "103809024",
MAX_REQUEST_SIZE: "104857600",
REPLICA_FETCH_MAX_BYTES: "103809024",
UNCLEAN_LEADER_ELECTION_ENABLE: "true",
MIN_INSYNC_REPLICAS: "2",
DEFAULT_REPLICATION_FACTOR: "3"
},
raftsettings: {
Options: {
TickInterval: "500ms",
ElectionTick: 10,
HeartbeatTick: 1,
MaxInflightBlocks: 5,
SnapshotIntervalSize: "100 MB"
}
},
network: {
fabric001: {
cas: ["ca1st.orga"],
peers: ["anchor@peer1st.orga", "anchor@peer1st.orgb"],
orderers: ["orderer1st.orgc", "orderer1st.orgd"]
},
fabric002: {
cas: ["ca1st.orgb"],
peers: ["worker@peer2nd.orga", "worker@peer2nd.orgb"],
orderers: ["orderer2nd.orgc", "orderer2nd.orgd"]
},
fabric003: {
cas: ["ca1st.orgc", "ca1st.orgd"],
peers: ["worker@peer3rd.orga", "worker@peer3rd.orgb"],
orderers: []
}
},
baseimage_tag: "1.4.3",
helper_tag: "0.4.14",
ca: { admin: "admin", adminpw: "adminpw", image_tag: "1.4.3" },
repo: {
url: "hyperledger/",
username: "",
password: "",
bin: "https://nexus.hyperledger.org/content/repositories/releases/org/hyperledger/fabric/hyperledger-fabric/linux-amd64-1.4.3/hyperledger-fabric-linux-amd64-1.4.3.tar.gz"
}
}```
BTW, with what you have, you are actually setting up 4 orgs total, one orderer org, and 3 peer orgs.
each peer org has two peers
with limited resources on your k8s cluster, I think you probably can remove few nodes , you actually will have 4 orderer nodes.
I think you can simply remove orderers under orderers fabric002
leave orderers under fabric002 empty just like orderers under fabric003.
or you can look at the vb1st.yml file as example to have less number of various nodes.
also use goleveldb will reduce number of resources needed.
thank you very much for your help, otherwise can I can increase the resource if yes how much cpu and memory need for master node and 2 worker node?
ok thank otherwise can I can increase the resource if yes how much cpu and memory need for master node and 2 worker node?
well, I suggest that you go through the simple network setup and make sure everything works, then you can start running some txs and see how it will performe.
the resource.yaml file actually limit the resource requirements for each type of node.
you can manupulate that if you do not have a lot of resources.
but the error does not look like you run out resources.
ok
I have one query regarding first step as K8s configuration file, to get access to a k8s cluster, how can I get this k8s config file?
Hi @tongli , I have one query regarding first step as K8s configuration file, to get access to a k8s cluster, how can I get this k8s config file?
according to document https://github.com/hyperledger/cello/blob/master/docs/agents/ansible.md
Hi @tongli I got same error while update the bc1st.yml file with less resource with 2 org (a,b) and each org have 2 peer including anchor peer and 2 orderer and 2 ca's. I have doubt about my k8s config file, as per document which you have shared me in Create configuration files section step 1 I have to create K8s configuration file, to get access to a k8s cluster, could you please tell me how I can do it?
Hi @tongli I got same error while update the bc1st.yml file with less 2 peer including anchor peer and 2 orderer and 2 ca's. I have doubt about my k8s config file, as per document which you have shared me in Create configuration files section step 1 I have to create K8s configuration file, to get access to a k8s cluster, could you please tell me how I can do it?
Hi @tongli I have create new k8s cluster and I didn't get that error however I am facing new error. Please find attached screenshot of error.
error while run the ansible-agent
You did not place the kubeconfig file in the right place
The error just said that. You need to place your new cluster kubeconfig file in the right place just like what you did previously
Hi @tongli thanks for advise, I have stuck in one thing actually I tried two way to implement the Cello 1st way I created 3 vm instances(ubuntu) [1master and 2 worker ] and create cluster using k8s. In this approach I have kubeconfig file, however I got error as `_"TASK [Make sure that working directory exists] *********************************
fatal: [cloud]: FAILED! => {"changed": false, "msg": "There was an issue creating /opt/agent/roles/deploy_k8s/../../vars/bc1st as requested: [Errno 13] Permission d
enied: '/opt/agent/roles/deploy_k8s/../../vars/bc1st'", "path": "/opt/agent/roles/deploy_k8s/../../vars/bc1st/run"}
" ` In 2nd approach I have create cluster using google Kubernetes engine but I got error as kubecofig file not found as I don't know where the file created as its done automatically by Kubernetes engine. So could you please help me to understand which approach will good for cello?
Hmmm, your kubeconfig file should be in your user root .kube directory.
You should copy kubeconfig file to your vars directory when running cello
Which describes all this in the readme doc
Hi @tongli Thanks for the reply, however I am getting same issue which I received early asd 1st way
hmmm. did you name your kubeconfig file actually kubeconfig?
and place it in your $(pwd)/vars directory?
yes I have change the file name and place it into $(pwd)/vars directory
Could you please let me know which way will be the best for Cello implementation as I mentioned?
I am not really sure how you setup your k8s cluster, I would think the google one is more reliable.
I am running into the same issue with regard to configuring the persistent volume claim from the ansible playbook. It fails because on DigitalOcean the pvc access mode of ReadWriteMany is not supported, and thus the pvc hangs in the 'pending' state. On DO, only ReadWriteOnce access mode is currently supported for the creation of the persistent volume claim (pvc).
@cmgabriel you can create your own PV like using NFS based storage.
doing this is described here. https://docs.okd.io/latest/install_config/storage_examples/shared_storage.html
Hi @tongli sorry for late reply, can you tell me best way to setup k8s so the cello will work?
Hi @tongli I have made progress to setup of cello with ansible agent and k8s, however I stuck with error. could you please guide me how to resolve it. Please see below attached screenshot of error.
Error screenshot
Has joined the channel.
If there are 4 organisations in a fabric network then should there be a single k8s cluster connecting all of them or 4 k8s clusters talking to each other?
@AbhijeetSamanta you can’t connect to the k8s cluster. Most likely your kubeconfig file was not in right place. Please follow the instructions
Or maybe your kubeconfig file was not created correctly
Has left the channel.
Has joined the channel.
I've completed the export of Gerrit to GitHub, as outlined here: https://lists.hyperledger.org/g/fabric/message/7331
Has joined the channel.
@rjones thanks so much for doing that.
Does anyone have tips for setting up the Fabric Cello for deployment on Raspberry Pi's?
Does anyone have tips for setting up Fabric Cello for deployment on Raspberry Pi's?
Has joined the channel.
Hi there! I've been working with Fabric lately, but I never asked myself if Cello could help setting up that in production. Is it that mature?
@tongli https://jira.hyperledger.org/browse/CE-718
It has been used by several companies in production, and some of them have enhanced the features like the UI part.
Hi @tongli,
Thanks for the starting this project. Cello would really help us to manage our networks. I am new to cello. I just cloned the repo and trying to run it on my system(ubuntu 18.x). Executed _make start_ which spins up all the required containers perfectly. But, I am not able to login to dashboard on _localhost:8081-throws a 400 error_ and couldn't connect to operators dashboard at all at _localhost:8080_(as mentioned in tutorial of cello).
Screenshot from 2019-12-15 17-17-51.png
Has joined the channel.
Has joined the channel.
Greetings Cello community! I'm part of the new Blockchain Automation Framework lab https://github.com/hyperledger-labs/blockchain-automation-framework and a reaching out to see if there are any opportunities for our projects to compliment each other. I've only joined the Blockchain space 6 months back so thank you for your patience. Cheers.
could you please let us know which username and password you are using?
the default which is mentioned in docs: admin/pass. But I have also tried: admin/password, admin/passwrOrd and admin/passw0rd
Welcome!
Has joined the channel.
Hi Cello Community, I'm trying to setup the Cello Master Node and Docker Worker agent using latest v0.9 git repository.
Hi Cello Community, I'm trying to setup the Cello Master Node and Docker Worker agent using latest v0.9 git repository. As per Hyperledger Cello documentation there should be Operator Dashboard running on 8080 and User dashboard on 8081. I can only see dashboard running on 8081, but nothing running on 8080.
I'm following steps mentioned in the link below:
https://hyperledger-cello.readthedocs.io/en/latest/setup/setup_master/
Moreover, the user dashboard running on 8081 doesn't seem to have Smart Contract option mentioned in the standard documentation:
https://hyperledger-cello.readthedocs.io/en/latest/dashboard_user/
Is this expected or am I missing something? Has someone successfully setup the Cello Master and Worker with latest version? Can you please share the steps?
Thank you
Does cello only function with k8s deployments of Fabric? When I looked around the user dashboard, it was the only option.
Has joined the channel.
Hello everyone, I attempted to setup cello for the first time following the getting started documentation. Running 'make setup-master' for v0.8.0 and v0.9.0 informed me that several cello images (cello-mongo, cello-base, cello-operator-dashboard, cello-user-dashboard,etc) were not accessible on docker with the tag x86 64-latest. I'm not entirely sure what steps I should take to resolve this.
Has joined the channel.
Exact same issue here
Is there a solution?
@RaphD @here, 0.9.0 is in bad shape. If you like to simple try to stand up a fabric network on a single node and experience fabric network or even doing some chaincode development, please take a look at this
https://github.com/litong01/minifabric
this will allow you to start it up very quickly without going through tons of work.
Has joined the channel.
Looking at some of the previous documentation and the language in the code, is there a reason behind the shift from clusters to chains on the front end?
Looking at some of the previous documentation and the language in the code, is there a reason behind the shift from clusters to chains on the front end? The change in terminology is a bit odd.
Hi - this should fix the uninteresting CI failures: https://github.com/hyperledger/cello/pull/92
Has joined the channel.
Has joined the channel.
@tongli @baohua any thoughts on this?
The Linux Foundation’s CommunityBridge engineers are working on a tool to measure the health of critical open source projects and one of the key areas identified is QA Testing. They request that our communities provide honest and detailed information on testing tools and methodologies you use in your projects for us to come up with a detailed analysis, which they will share with all respondents and projects. https://www.surveymonkey.com/r/9H5G2GV. It’s only 5 questions long.
Has joined the channel.
https://github.com/hyperledger/cello/pull/105
Has joined the channel.
Has joined the channel.
Howdy Contributors and Maintainers! Are you wondering about tapping into Developer marketing for your group or project? Do you have a blog post idea? An awesome announcement? Please attend our Contributor/marketing meeting! https://wiki.hyperledger.org/display/Marketing/2020-02-19+Meeting+notes
Are you wondering about tapping into Developer marketing for your group or project? Do you have a blog post idea? An awesome announcement? Please attend TOMORROW! https://wiki.hyperledger.org/display/Marketing/2020-02-19+Meeting+notes
Has joined the channel.
Is this project dead?
No commits to the master for more than 20 days :grimacing:
@abu, not dead, most of the contributors are in the coronavirus effected zone.
@abu If you have a need to stand up fabric network, take a look at this project https://github.com/litong01/minifabric
Too bad. What a disaster man :disappointed_relieved:
Is there any casualty?
@abu, so far there are more than 2000 people dead due to this virus. I do say very serious
Ok I know that. I mean is there any among the contributers?
Not that I know of
Has joined the channel.
Has joined the channel.
@tongli hi, just wonder why the current docker build is not passed?
[ ](https://chat.hyperledger.org/channel/cello?msg=k8Zmbeuqf8MTxGfGd)
Clipboard - February 24, 2020 5:13 AM
I cannot check details since the page returns 404
```Cloning into '.'...
Warning: Permanently added the RSA host key for IP address '140.82.114.3' to the list of known hosts.
Reset branch 'master'
Your branch is up-to-date with 'origin/master'.
Build path does not exist: ./build_image/dockerhub/latest/ansible-agent```
https://hub.docker.com/repository/docker/hyperledger/cello-agent-ansible/builds
https://hub.docker.com/repository/docker/hyperledger/cello-ansible-agent/builds
I disabled builds for https://hub.docker.com/repository/docker/hyperledger/cello-ansible-agent/builds since it hasn't passed in a month - @tongli let me know how you want to go forward
@rjones we had a mixup, we have two hyperledger/cello-agent-ansible and cello-ansible-agent.
let me take a look why it is failing.
@rjones I do not seem have the access to configure the automated builds.
for some reasons, we have two hyperledger/cello-agent-ansible and hyperledger/cello-ansible-agent.
cello-agent-ansible is fine, cello-ansible-agent is not. all the docs we have are based on cello-ansible-agent.
ideally we should just have cello-ansible-agent working and remove cello-agent-ansible. but the builds have issues with the right one, I do not have permission to look into the automated build.
ideally we should just have cello-ansible-agent working and remove cello-agent-ansible. but the builds have issues with the right one, I do not have permission to look into the automated build for either of them.
@tongli can we chat in a bit via screen share?
sure.
I am ready
@rjones ^^^
https://zoom.us/j/2032622322
@tongli
build_image/dockerhub/latest/agent/ansible
https://hub.docker.com/repository/docker/hyperledger/cello-ansible-agent/builds
@tongli do you want me to delete all these tags except latest? https://hub.docker.com/repository/docker/hyperledger/cello-nginx/tags?page=1
that is the only project that has tags other than `latest`
yes, please
OK. When I look here, it looks like it has 3 builds set up: https://hub.docker.com/repository/docker/hyperledger/cello-nginx/builds/edit
if you can't disable them, I can disable the other two builds if you like
let me try.
OK. I'm interested to know, since it's connected already, if you can
no, I cant disable them. same issue. I think because I can not access the auto build config
I can only trigger a build. but cannot disable or make changes to any of them.
OK. So should I disable, or delete, the builds for latest-x86_64 and 080?
(which do you want)
delete them.
please
Gone
Thanks Ry.
you're welcome :)
Has left the channel.
Has joined the channel.
hello cello folks. I am trying to deploy k8s through the ansible script in the repository. I was wondering if you could redirect me to the binary releases that were up on nexus before that was taken down...https://nexus.hyperledger.org/content/repositories/releases/org/hyperledger/fabric/hyperledger-fabric/linux-amd64-1.1.0/hyperledger-fabric-linux-amd64-1.1.0.tar.gz, was where it was referenced
@cmonkeydo https://github.com/hyperledger/fabric/releases/tag/v1.1.1
@cmonkeydo the binary is at the bottom
thanks for the quick response @rjones
in the documentation for working with ansible there is no provcluster.yml file
ansible-playbook -e "mode=apply env=mycloud cloud_type=os" provcluster.yml
My assumption is that this is wrapped up in the fabric_k8s plays in the first step? Is my assumption valid?
@tongli ^^ ?
Hey all, I see in the repo there is kubernetes, ansible and fabric operator agents available. For my ansible deployment I get almost all the way through however when a peerlist pod attempts to pull from a resource eg docker.io/hyperledger/cc-env:1.4.4 it can't resolve it. The context for this is a bare metal system so not in the cloud, however I am not sure that that is an issue or not.
All in another test I am going from the fabric-operator docs. I get to the ca section (with defaults from the repo) and the ca pod never gets Ready. From what I can tell it has to do with the storage which I have added default pvc storage. Currently I have added the default config to have the same storageClass as the pv and yet the pvc does not currently mount. Any help on this or the above question would be greatly appreciated.
I have figured out that the issue was because I didn't have the correct mode of ReadWriteOnce on my pv for each of the different types of systems. This seems like something that should be added to the documentation for the operator. I can write something up if that would be appreciated.
On the documentation for the fabric-operator read me there is r
To run the operator using the cello API or dashboard follow the instructions here :- Fabric Operator Agent README
Which has a link back to the Fabric operator agent readmen (e.g. the same page I am reading) Is there documentation somewhere on integrating the gui with the operator api?
On the documentation for the fabric-operator read me there is r
To run the operator using the cello API or dashboard follow the instructions here :- Fabric Operator Agent README
Which has a link back to the Fabric operator agent readmen (e.g. the same page I am reading) Is there documentation somewhere on integrating the gui with the operator agent?
docker pull hyperledger/fabric-ccenv:1.4
`docker pull hyperledger/fabric-ccenv:1.4`
https://hub.docker.com/repository/docker/hyperledger/fabric-ccenv/tags?page=1
very much appreciated if you can!
link?
https://github.com/hyperledger/cello/issues/116 is the pr I did for the PersistentVolume...it probably is only relevant in a bare metal setup as cloud providers probably have solved this already...https://github.com/hyperledger/cello/tree/master/src/agent/fabric-operator all the way at the bottom is the link
using export DEPLOY_METHOD=k8s and running the start script did install things on the local k8s node, however I am unable to access the dashboard from the 8081 or 8080 ports....not sure if nginx is the issue on this or not
OK. You'll need to fix the DCO and squash your commits
https://wiki.hyperledger.org/display/BESU/DCO
alrighty I will look into that...probably should have done that first
you can force push to your local repo after a git squash and it will update the PR
k
Has left the channel.
Any plans to support Fabric 1.4.x?
1.4 is already supported in v0.9. Now we are redesigning the architecture.
Has joined the channel.
Hi Team,
We are facing issue while setting up cello master node.
To setup the master node, we run the following command.
make setup-master
Getting the following error while executing the " make setup-master" command.
Download required Docker images for Cello Services...
Downloading the docker images for Cello services: VERSION=latest ARCH=x86_64
Check node:9.2 image.
Pulling hyperledger/cello-dashboard:x86_64-latest from dockerhub
Error response from daemon: manifest for hyperledger/cello-dashboard:x86_64-latest not found: manifest unknown: manifest unknown
Error response from daemon: No such image: hyperledger/cello-dashboard:x86_64-latest
Pulling hyperledger/cello-nginx:x86_64-latest from dockerhub
Error response from daemon: manifest for hyperledger/cello-nginx:x86_64-latest not found: manifest unknown: manifest unknown
Error response from daemon: No such image: hyperledger/cello-nginx:x86_64-latest
Pulling hyperledger/cello-api-engine:x86_64-latest from dockerhub
Error response from daemon: manifest for hyperledger/cello-api-engine:x86_64-latest not found: manifest unknown: manifest unknown
Error response from daemon: No such image: hyperledger/cello-api-engine:x86_64-latest
Note:
After executing "make setup-master" command, I'm able to open the user dashboard on port 8081 but operator dashboard not working. As per the documentation, operator dashboard should work on port 8080 by default.
Please help.
Looking at the dockerhub repo of hyperledger/cello* it do es not seem that they are multi arch builds. You can look at https://hub.docker.com/r/hyperledger/cello-api-engine/tags to see what I mean. The :x96_64 is not one of the tags available.
Good luck
https://github.com/hyperledger/cello/pull/120
Has joined the channel.
Has joined the channel.
https://github.com/hyperledger/cello/pull/122
@baohua https://github.com/hyperledger/cello/issues/125 builds are broken since 120 was merged
Has left the channel.
https://github.com/hyperledger/cello/pull/129
Has joined the channel.
Has joined the channel.
Does this support Fabric 2.x?
After succeed of "make start", what next?
@baohua @tongli merge please? https://github.com/hyperledger/cello/pull/145
@rjones Done. thanks.
thank you!
@spartucus the documentation is pretty good for using cello - I was able to get it up and running without a lot of work on a Fedora VM
Thanks for reply! In "setup" section, no details of how to "Prepare environment", any tips of that?
Has joined the channel.
Hi @rjones , which branch do you use? for a simple setup of cello (both master and worker)?
master
Has joined the channel.
Has joined the channel.
Has joined the channel.
I was going through the cello docs( https://github.com/hyperledger/cello#introduction ), but the tutorial & index links under 'Documentation, Getting started and Develop Guideline' didn't seem to work.
Can you please guide me to the correct path?
I tried going to the ReadTheDocs site for the same ( https://cello.readthedocs.io/en/latest/setup/server/ ), but the documentation doesnt have content under 'Setup/Server'. Also 'Operations/Dashboard' tab doesn't have any information under the sections 'Operator operations', 'Admin operations' and 'User operations'. Same goes for section 'Agent/Ansible' and I couldn't find the sample fabric network spec after it redirected to https://github.com/hyperledger/cello/tree/master/src/operator-dashboard/agent/ansible/vars/bc1st.yml
Has left the channel.
This week's meeting is cancelled, enjoy the vacation!
Hi @spartucus - I noticed all of your commits to Cello are signed with an invalid email address - could you work with me to fix that?
Sure, how can I help?
Has joined the channel.
Has joined the channel.
Hi. I want to know if cello can work for fabric 1.4.4 ?:grinning:
@HunterJW I think it probably can? give it a whirl
Can cello work perfectly with master and worker in one machine and simply use docker without kubernetes? I try it but it doesn't work, I think maybe some problems there.
Does cello work with fabric 2.1
Has joined the channel.
Has joined the channel.
Hi, all
I'm struggle with cello set-up a week.
I can not create Agent in cello.
https://github.com/hyperledger/cello/issues/46
is there any docs or useful link for it ??
Has someone successfully setup the Cello Master and Worker with ANY version? Can you please share the steps ?
Clipboard - June 24, 2020 10:24 AM
I created Agent k8s type.
Clipboard - June 24, 2020 10:25 AM
But, I can not create node.
There is no +New button. How to add Node ? I have log in default admin/pass
Does cello work with fabric 2.1
Clipboard - July 21, 2020 4:18 PM
Clipboard - July 22, 2020 12:43 PM
hi,anyone know how to fix above problem?
Has left the channel.
Has joined the channel.
Hi guys,
I have set up cello from master branch.
I am confused how to create agents and nodes from ui. Can someone guide, there are also lack of docs for cello-dashboard.
Has left the channel.
Has joined the channel.
Has joined the channel.
Has joined the channel.
Has joined the channel.
Can Cello be used for MultiCloud deployment of hyperledger fabric on AWS , GCP and Azure Kubernetes Clsuters?
looks like this channel is not active?
@AshutoshKumar7 most of the developers are in China on China timezone
@AshutoshKumar7 There are support for k8s but for aws, gcp I don't think we have direct support
@AshutoshKumar7 There are supports for k8s & docker but for aws, gcp I don't think we have direct support
Has joined the channel.
HL Cello session: https://pathable.zoom.us/j/92909594210?pwd=SXFSNGt2ODJLNjRPd0VYUkpxUVF5UT09
Is there a recording available?
Hello All,
We are evaluating using Cello to deploy a HLF 2.x network.
Is it supported by Cello? If yes, any latest deployment guide available to setup HLF network using Cello?
HLF 2.x is not yet.
TSC 选举正在进行中,请大家关注 https://wiki.hyperledger.org/display/TSC/TSC+Election+2020-2021。
TSC election is ongoing, more details can be found at https://wiki.hyperledger.org/display/TSC/TSC+Election+2020-2021。
TSC election is ongoing, more details can be found at https://wiki.hyperledger.org/display/TSC/TSC+Election+2020-2021
Has joined the channel.
Has joined the channel.
Has joined the channel.
Has joined the channel.
Has joined the channel.
may I ask when v1.0.0 will be released ?
according to roadmap, it should be released in Oct ?
Has joined the channel.
Has joined the channel.
Has joined the channel.
Has joined the channel.
are the cello images from docker hub?
Hi, all:) Does the cello support transaction info search?
Has joined the channel.
Has joined the channel.
Good morning! I am learning to deploy cello, but I have not found a suitable tutorial or a solution to the deployment failure. Can you help me deploy cello?
Has joined the channel.
Has joined the channel.
Has joined the channel.
hi,I want to ask
Hi,I want to ask where is perfoamance code in cello?
now,I want to use cello to manage gm-fabric
has anyone who familiar with cello?
someone?
@here https://github.com/hyperledger/fabric/discussions/2456
Has joined the channel.
Hi, when the version 1.0 of cello will be released?
The current target is second half of this year.
Has joined the channel.
hi
I'm installing hyperledger cello based on this doc: https://cello.readthedocs.io/en/v0.9.0/setup_master/
in setup master node i get this error
root@hyperledger:~/cello# DEV=True make start
if [ "" = "y" ]; then \
make start-docker-compose; \
else \
make start-k8s; \
fi
make[1]: Entering directory '/root/cello'
make[2]: Entering directory '/root/cello/bootup/kubernetes'
make[3]: Entering directory '/root/cello/bootup/kubernetes'
mkdir -p /root/cello/bootup/kubernetes/nginx
make[3]: Leaving directory '/root/cello/bootup/kubernetes'
make[3]: Entering directory '/root/cello/bootup/kubernetes'
rm -rf /root/cello/bootup/kubernetes/api-engine
mkdir -p /root/cello/bootup/kubernetes/api-engine
if [ "prod" = "dev" ]; then \
envsubst < templates/api-engine/ingress.tmpl > api-engine/ingress.yml; \
fi
make[3]: Leaving directory '/root/cello/bootup/kubernetes'
make[3]: Entering directory '/root/cello/bootup/kubernetes'
mkdir -p /root/cello/bootup/kubernetes/api-engine-tasks
make[3]: Leaving directory '/root/cello/bootup/kubernetes'
make[3]: Entering directory '/root/cello/bootup/kubernetes'
mkdir -p /root/cello/bootup/kubernetes/redis
make[3]: Leaving directory '/root/cello/bootup/kubernetes'
make[3]: Entering directory '/root/cello/bootup/kubernetes'
mkdir -p /root/cello/bootup/kubernetes/postgres
make[3]: Leaving directory '/root/cello/bootup/kubernetes'
make[2]: Leaving directory '/root/cello/bootup/kubernetes'
make[2]: Entering directory '/root/cello/bootup/kubernetes'
The connection to the server localhost:8080 was refused - did you specify the right host or port?
Makefile:89: recipe for target 'start' failed
make[2]: *** [start] Error 1
make[2]: Leaving directory '/root/cello/bootup/kubernetes'
Makefile:221: recipe for target 'start-k8s' failed
make[1]: *** [start-k8s] Error 2
make[1]: Leaving directory '/root/cello'
Makefile:208: recipe for target 'start' failed
make: *** [start] Error 2
this is my make logs output:
------> this is my make logs output:
-----------------------------
root@hyperledger:~/cello# make logs
docker-compose -f bootup/docker-compose-files/docker-compose-dev.yml logs -f --tail=200
WARNING: The CONFIG_API_ENGINE_WEBROOT variable is not set. Defaulting to a blank string.
WARNING: The CONFIG_POSTGRES_USER variable is not set. Defaulting to a blank string.
WARNING: The CONFIG_POSTGRES_PASSWORD variable is not set. Defaulting to a blank string.
WARNING: The CONFIG_API_ENGINE_ADMIN_TOKEN variable is not set. Defaulting to a blank string.
WARNING: The CONFIG_API_ENGINE_ADMIN_USERNAME variable is not set. Defaulting to a blank string.
WARNING: The CONFIG_API_ENGINE_ADMIN_PASSWORD variable is not set. Defaulting to a blank string.
WARNING: The CONFIG_API_ENGINE_ADMIN_EMAIL variable is not set. Defaulting to a blank string.
WARNING: The CONFIG_API_VERSION variable is not set. Defaulting to a blank string.
WARNING: The CONFIG_API_ENGINE_SERVICE_PORT variable is not set. Defaulting to a blank string.
WARNING: The CONFIG_API_DOCKER_HOST variable is not set. Defaulting to a blank string.
WARNING: The CONFIG_DASHBOARD_SERVICE_PORT variable is not set. Defaulting to a blank string.
ERROR: The Compose file './bootup/docker-compose-files/docker-compose-dev.yml' is invalid because:
services.dashboard.ports contains an invalid type, it should be a number, or an object
services.api-engine.ports contains an invalid type, it should be a number, or an object
Makefile:198: recipe for target 'logs' failed
make: *** [logs] Error 1
-------------------------------
i want to take tutorial steps to setup cello cluster
------> after execution of make setup-master these images get error and i download latest version without any problem:
------> after execution of make setup-master these images get error and i download latest version without any problem: (before execution of make start )
Pulling hyperledger/cello-dashboard:x86_64-0.9.0 from dockerhub
Error response from daemon: manifest for hyperledger/cello-dashboard:x86_64-0.9.0 not found: manifest unknown: manifest unknown
Error response from daemon: No such image: hyperledger/cello-dashboard:x86_64-0.9.0
Pulling hyperledger/cello-nginx:x86_64-0.9.0 from dockerhub
Error response from daemon: manifest for hyperledger/cello-nginx:x86_64-0.9.0 not found: manifest unknown: manifest unknown
Error response from daemon: No such image: hyperledger/cello-nginx:x86_64-0.9.0
Pulling hyperledger/cello-api-engine:x86_64-0.9.0 from dockerhub
Error response from daemon: manifest for hyperledger/cello-api-engine:x86_64-0.9.0 not found: manifest unknown: manifest unknown
my os is ubuntu 1.4
my os is ubuntu 18.04
my default run was kubernetes. i changed it to docker compose
Clipboard - March 31, 2021 10:06 AM
i didnt change the default. but it was kubernetes
i dont have add host
Clipboard - March 31, 2021 10:24 AM
these images are used
Clipboard - March 31, 2021 11:29 AM
Clipboard - March 31, 2021 4:17 PM
I want to add a kubernetes cluster
which ip address i should insert? (my kubernets cluster have 3 nodes and all nodes have all role: etcd, control plane and worker)
which ip address i should insert? (my kubernets cluster have 3 nodes and all nodes have all roles: etcd, control plane and worker)
what kind of smart contract can be added in user dashboard?
can Ethereum smart contract can be run in it?
Has joined the channel.
Has joined the channel.
Has joined the channel.
Has joined the channel.
Hello team
getting error while setup cello in docker - 404: http://34.136.214.145:8081/api/token-verify
not able to login
cello dashboard
Has joined the channel.
Same here
Has joined the channel.
Has joined the channel.
Has joined the channel.
having the same login problem.
we are having the same problem, has this been solved?
Has joined the channel.
Having the same problem with login
Has joined the channel.
Does cello usable for besu?
As far as I know, Cello still only supports Fabric. If you are interested in deploying a Hyperledger Besu network, you may want to check out #blockchain-automation-framework -- this is a lab that allows you to deploy networks for a number of different frameworks, including Besu
Has joined the channel.
hi team, I have started working in fabric recently, now I want to integrate cello with it.
Please someone point me to related videos for installation with fabric
Has joined the channel.
Hello everyone. I would like to use cello with HLF 2.2 to deploy me network as suggested https://lists.hyperledger.org/g/fabric/topic/41199117#7104
Is this version supported? Also, is this the best way to deploy HLF for production?
I don't only want to deploy HLF but to mange too. For example add a new channel or org
Has joined the channel.
Has joined the channel.
Has joined the channel.
This is a message for chinese who are not proficent in English。
中国的朋友可以加微信:ali-iot, 备注“cello”, 我拉您进微信群。
讨论cello相关的技术话题,此广告长期有效。
Has joined the channel.
hey I was getting redy to try cello in rancher
has anyone done this yet ?
Clipboard - September 29, 2021 11:13 AM
Clipboard - September 29, 2021 11:13 AM
and this part here is telling me maybe I am not accessing the complete code ?
I am running this with newest docker Ubuntu 20.04 x64
clean installs updated, git clone, docker make
E: Package 'python-pip' has no installation candidate
The command '/bin/sh -c apt-get update && apt-get install -y bash curl python-dev sudo python-pip build-essential openssh-client libffi-dev libssl-dev && pip install --upgrade pip ansible pyyaml && groupadd -g ${gid} ${user} && useradd -d /opt/agent -u ${uid} -g ${user} ${user} && usermod -a -G root ${user} && echo "${user} ALL=(ALL) NOPASSWD: ALL"|tee /etc/sudoers.d/${user} && mkdir -p /opt/agent/.ssh && cd /opt/agent/.ssh && echo "host *" > config && echo " StrictHostKeyChecking no" >> config && echo " UserKnownHostsFile /dev/null" >> config' returned a non-zero code: 100
make: *** [Makefile:119: build/docker/agent/ansible/.x86_64-0.9.0-snapshot-8bd36c0] Error 100
shit, wrong room , my apologues,
and this part here is telling me maybe I am not accessing the complete code ?
I am running this with newest docker Ubuntu 20.04 x64
clean installs updated, git clone, docker make
E: Package 'python-pip' has no installation candidate
The command '/bin/sh -c apt-get update && apt-get install -y bash curl python-dev sudo python-pip build-essential openssh-client libffi-dev libssl-dev && pip install --upgrade pip ansible pyyaml && groupadd -g ${gid} ${user} && useradd -d /opt/agent -u ${uid} -g ${user} ${user} && usermod -a -G root ${user} && echo "${user} ALL=(ALL) NOPASSWD: ALL"|tee /etc/sudoers.d/${user} && mkdir -p /opt/agent/.ssh && cd /opt/agent/.ssh && echo "host *" > config && echo " StrictHostKeyChecking no" >> config && echo " UserKnownHostsFile /dev/null" >> config' returned a non-zero code: 100
make: *** [Makefile:119: build/docker/agent/ansible/.x86_64-0.9.0-snapshot-8bd36c0] Error 100
cello, you know somthing
I watched this video last night
on rancher, I have unique project I am launching which finally has a stable capable stack here finally in the openSource community
and in this video , The gentleman makes a very good point about things not working not being maintained , and configurations that should be done
Here is the video, It's funny i know this guy in this video, even if I never met him, or may never hear from him again, you see were all alike :)
And he does this explainer for kubernetes, and does such a good job of pointing out something's about the software that apply also to hyperledgers stack
https://www.youtube.com/watch?v=LK6KbAlQRIg
I mean this gentleman like me, has probley been install Linux since early mid 90's when someone simply dident just install linux
95% of all the software we have all made money off of and built business and empires off off, just worked, really it did, you followed the instruction in the docks and there was no guessing till you got your configs, Apache, CSS , HTML, PHP, NODE, Asterisk, Wordpress, Joomla, GIT, I mean I go on and on. I have yet to use one hyper ledger product that just worked
you followed the instrctions which were simple enough for an engineer to understand or even a noob engineer, and you are set up and running ready to poke away at your own configs and products, Docker and Conatinerd, Work out the get go? Would it be nice if you had to sit there fugit with dockers underlineings before we started our own works, Just some food for thought, was this project not given money for this software to work ?
Has joined the channel.
Has joined the channel.
Hi,
I have deployed Cello following the steps at https://github.com/hyperledger/cello#readme and when accessing the dashboard I am asked my organization name, what shoud I use for the default admin credentials (user: admin / password: pass)?
Has joined the channel.
Has joined the channel.
hi all, I am new to cello. One question I have after some reading on the internet is, is it possible to use Cello web interface to create a new peer in another server (in another organisation/region).
Has joined the channel.
Hi All, quick question: would anyone from the Cello community be willing to help me by filling out a survey[1] for the TSC chat platform task force? I'm happy to do the typing if anyone would prefer to get on a call and just explain themselves instead of filling it out themselves. (I can't make it to the weekly call unfortunately because it's at 1 AM my time)
[1]: https://wiki.hyperledger.org/display/TF/Cello
Has joined the channel.
Has joined the channel.
Clipboard - December 28, 2021 2:39 PM
Hi All, I followed the steps given in https://github.com/hyperledger/cello for setting up cello. I'
m not using kubernetes. I'm able to start the cello service. but while starting dashboard using make start-dashboard i'm getting this error
Clipboard - December 28, 2021 2:39 PM
Hi All, I followed the steps given in https://github.com/hyperledger/cello for setting up cello. I'
m not using kubernetes (using docker-compose). I'm able to start the cello service. but while starting dashboard using make start-dashboard i'm getting this error. Can anyone please help?
Clipboard - December 28, 2021 2:39 PM
Hi All, I followed the steps given in https://github.com/hyperledger/cello for setting up cello. I'```
``` m not using kubernetes. I'm able to start the cello service. but while starting dashboard using make start-dashboard i'm getting the following error
Has joined the channel.
we are also having the same problem, anyoneeee?
Has joined the channel.
Registration successful with error
Hi All, We are doing the cello set up and facing the issues to login . Can anyone guide us how to resolve this ? The screenshots are attached for the reference.
Error 2: If we are doing the login, its showing success, though it's not
Error :Registered success with errors
Any support to resolve this is highly appreciable.
make all-tox
are you able to access the dashboard and perform any actions from the UI?
Hi all,we are trying to setup Hyperledger cello for HLF(2.2). The thing is any action we perform in the dashboard, its routing to the same font-end api,the request is not passing to the api engine. is there any workaround for this/any scripts available to validate our setup?
use "admin@cello.com" as email and "pass" as password
Has joined the channel.
Cello-0.9.0h3c is a version that can be used, I think if main branch Can't run ,maybe,you can try 0.9.0h3c branch
Has joined the channel.
Has joined the channel.
[chat is now over on discord](https://discord.gg/hyperledger)