Request for Proper Build Instruction

[somsak]

Rocket.Chat Version: 0.55.1
Running Instances: 1
DB Replicaset OpLog: 0 (standalone)
Node Version: 4.8.1

I would like to request a proper build instruction for Rocket.Chat server.

We have fixed some issue with SlackBridge and wanted to build our own server base on Docker Official image. Currently we are using Docker official image in production. In the Dockerfile, the docker build pull the pre-compiled Rocket.chat from rocket.chat release space and continue from that. My question is how to build the pre-compiled Rocket.chat release.

Right now we did the dirty quick fix by editing the compiled code in Docker image.

We have tried the following build instruction

export NODE_ENV="production"
meteor npm install --save bcrypt
meteor add rocketchat:internal-hubot meteorhacks:kadira
meteor build --server "https://our.rocket.chat.server" --directory "/path/to/bundle"

It almost works (though the resulting image is much bigger than official Docker image), except the Cordova app. The Cordova app (on Android) stuck after finish loading file (stuck at Loading screen).

[somsak]

Sorry, the actual instructions we used was

export NODE_ENV="production"
meteor npm install --save bcrypt
meteor add rocketchat:internal-hubot meteorhacks:kadira
meteor build --server-only --server "https://our.rocket.chat.server" --directory "/path/to/bundle"

Without --server-only the build process failed during Cordova built due to error with Google plus dependency (it needs REVERSE_CLIENT_ID which we didn't sure what the heck is that).

[geekgonecrazy]

check out our travis script here in the repo for exactly what we do to produce every build artifacts.

[somsak]

Ok we'll try that.

BTW, during the build process, a message warning that Cordova on IOS need OSX to build, along with all those XCode stuff. If we didn't want to modify anything in Cordova apps but would wants the existing apps to still work, can we build on Linux machine?

[geekgonecrazy]

Travis builds on Linux. We also get that warning

[nonth]

Hi @geekgonecrazy have you use --server-only option when build RocketChat ? I saw in travis script it doesn't use that option.

because when I build RocketChat without that option it's try to build Cordova and the build process failed as @somsak mention above (need REVERSE_CLIENT_ID)

[somsak]

We just tried this new which we guess+extract from travis.yml

# there's a warning message at the start regarding bcrypt that instructs to do this
meteor npm install --save bcrypt
# Extract from before_script in travis.yml
meteor npm install -g node-gyp node-pre-gyp
# Or else the build will fail
meteor npm install
# Do we need this? Extract from exampl-build.sh
meteor add rocketchat:internal-hubot meteorhacks:kadira
# Extract from travis.yml
meteor build --headless /path/to/store/bundle

The result is a tar+gz of bundle. We store this on our Docker image, which was base on official docker. The relevant build steps are

RUN tar -Jxf rocketchat-${RC_VERSION}.tar.xz && rm -f rocketchat-${RC_VERSION}.tar.xz \
&& cd bundle/programs/server \
&& npm install

The result Docker image worked with Web-site, Rocket.chat+ Native app, and Electron desktop app. However, Cordova app stuck during the loading, just before it was trying to display the log-in prompt.

Note that, after we switched to the new docker image, Cordova app noticed that there's an update. If we cancel that, the app would still work. If we update the app or start fresh (Clear data on Android), the app stuck after downloading all 118 files with the three waiting dot, just before the username/password prompt, forever.

Please note that we build the meteor app on Linux. Does the released Rocket.chat built on Linux?

[geekgonecrazy]

@somsak every single release of rocket.chat comes out of travis which runs linux. So yes for sure. As to the mobile client not connecting... not sure.

From your build script. You don't need: meteor add rocketchat:internal-hubot meteorhacks:kadira unless you just want it.

Other then that everything looks right.

@rodrigok any idea on their build not working with the cordova app?

[somsak]

What if we want the internal rocket.cat hubot? Do we need to add it?

Note that we add a dummy mobile-config.js. https://github.com/inoxth/Rocket.Chat/blob/0.55.1-hotfix/mobile-config.js.

How can we debug the Cordova App? If we know how to debug it might help.

[geekgonecrazy]

Our Cordova app isn't built using this code base. It's essentially a shell that will load in multiple servers. So specific code like you linked there is usually done in the cordova app.

You'll need to build the cordova app to do debugging. Inside that app is where we set the push details..etc

Maybe come hang out in #dev on our demo server. Maybe someone there will have ideas to help.

[somsak]

So is it safe to pass --server-only to meteor build? We have tried that before but Cordova app stuck the same.

[somsak]

Let me explain a bit more about why we want to build our own release.

We have fix some bugs in Slackbridge and would want to extend Slackbridge a bit (like support for sending file from RC to Slack, better reply/quote display in Slack, Fav, etc..). So we just want to build a server that includes all this fixes while still using the same old Cordova app.

So far we have fix the problem of sending file from Slack to RC.

inoxth@e178690

We wish to test things out for a while before creating pull request.

[somsak]

BTW, after detail checking, it seems that the different in release size is mainly because of phantomjs binary. Can we delete the bin/phantomjs file from all node_modules? Is this would be use only for unit test?

[somsak]

Ok I found a way to workaround this issue.

After I built the bundle with the instruction above. I copied over bundle/programs/web.cordova from official release (0.55.1) and then overwrite the directory in my own build. This way the Cordova app on Android still working.

[somsak]

We have been building docker image by using above method for a while. The result docker image run perfectly.

This could be close for a while, though I think there's still something weird and the build process need more explanations.

[megamaced]

Can this be re-opened?

We currently run a 0.48.2 instance compiled by us which works fine in the mobile app. We recently compiled 0.56.0 in the same way but the Cordova app doesn't work and we had to roll back to 0.48.2

I am afraid @somsak isn't really a solution for us as we like to package up an RPM and doing that workaround in an RPM would be horrible.

We can't use the new Rocket.Chat+ mobile app because it doesn't support ADFS login

[reetp]

I'd love to see some clear instructions on how to build from git source so I can test the dev releases.

I build a rpm installer for Rocket on Koozali SME server and would like to be able to incorporate bug fixes as they arrive (and possibly try to contribute back if I can)

However I have zero experience in this build environment but would like to learn.

[JSzaszvari]

1. Install MeteorJS from https://www.meteor.com/
2. Git clone the Rocket.Chat Repo
3. run 'meteor npm start' in the folder where you cloned the repo.

Make sure you are not the root user and it will tell you anything else it needs from there.

Thats it, Anything unexpected that comes up can usually be solved with a 'npm install' or quick google if its something specific to your environment.

[JSzaszvari]

I auto-build every release with GitLab CI and my whole build-pipline file consists of

   - curl https://install.meteor.com/ | sh
   - meteor npm install
   - mkdir /tmp/build
   - meteor build /tmp/build 

I build in a Docker Image, Specifically 'node:latest'

[reetp]

Thanks!
Will that build a tar.gz as per releases, or install direct on the machine?
My build server is a separate vm so need the compiled tar.gz (rather than installed directly) so I can build a rpm from it.

[JSzaszvari]

@reetp

The 'meteor build' command will output a tar.gz

'meteor build /tmp/build '

Will output a tar.gz into /tmp/build

Here is the CI script in full, I cut some off before. This seems to work reliably without fail

build:
  stage: build
  script:
   - apt-get upgrade && apt-get install -y curl
   - curl https://install.meteor.com/ | sh
   - mkdir /tmp/build
   - meteor build /tmp/build --allow-superuser
   - mkdir artifacts
   - mv /tmp/build/rocketchat_server.tar.gz artifacts/
    paths:
     - artifacts/*.gz

If you run into issues let me know and I should be able to step you around them.

[reetp]

Many thanks!
Am away til middle of the week but will try when back and let you know.

[reetp]

@JSzaszvari
Installed meteor
made dir /tmp/build
ran the build

error :
Error: ENOENT: no such file or directory, scandir 'packages/rocketchat-katex/node_modules/katex/dist/fonts/'

I have the packages/rocketchat-katex dir but it has no node_modules sub dir. Most likely something completely stupid on my part as I am a node/npm virgin :-)

[JSzaszvari]

@reetp

Thats interesting, I actually started getting that on a few different environments the other week when trying to build.

Whilst I cant tell you why it's failing (Only starting learning Meteor/Node when I fell in love with Rocket.Chat) - I can offer a fix/workaround (I hope)

Essentially what we need to do is jump into that directory and force the module to pull down the missing dependancy

So what I posted above would change to this

curl https://install.meteor.com/ | sh
meteor npm install
cd packages/rocketchat-katex
npm install
cd ../..
mkdir /tmp/build
meteor build /tmp/build --allow-superuser

Note the 3 extra steps, go into the katex plugin folder, run a 'npm install' whilst sitting in that folder. Then go back 2 levels to where we ran the original 'npm install' - then it should build fine.

Let me know how you go! :)

[reetp]

Well after a lot of hacking about I seem to be stuck on building bcrypt.

I then tried to install/build manually and get this:

[root@www ~]# npm install bcrypt --g

bcrypt@1.0.2 install /usr/lib/node_modules/bcrypt
node-pre-gyp install --fallback-to-build

node-pre-gyp ERR! Tried to download(404): https://github.com/kelektiv/node.bcrypt.js/releases/download/v1.0.2/bcrypt_lib-v1.0.2-node-v46-linux-x64.tar.gz
node-pre-gyp ERR! Pre-built binaries not found for bcrypt@1.0.2 and node@4.8.3 (node-v46 ABI) (falling back to source compile with node-gyp)
gyp WARN EACCES user "root" does not have permission to access the dev dir "/root/.node-gyp/4.8.3"
gyp WARN EACCES attempting to reinstall using temporary dev dir "/usr/lib/node_modules/bcrypt/.node-gyp"
make: Entering directory /usr/lib/node_modules/bcrypt/build' make: *** No rule to make target ../.node-gyp/4.8.3/include/node/common.gypi', needed by Makefile'. Stop. make: Leaving directory /usr/lib/node_modules/bcrypt/build'
gyp ERR! build error
gyp ERR! stack Error: make failed with exit code: 2
gyp ERR! stack at ChildProcess.onExit (/usr/lib/node_modules/node-gyp/lib/build.js:285:23)
gyp ERR! stack at emitTwo (events.js:87:13)
gyp ERR! stack at ChildProcess.emit (events.js:172:7)
gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:211:12)
gyp ERR! System Linux 2.6.32-696.1.1.el6.x86_64
gyp ERR! command "/usr/bin/node" "/usr/lib/node_modules/node-gyp/bin/node-gyp.js" "build" "--fallback-to-build" "--module=/usr/lib/node_modules/bcrypt/lib/binding/bcrypt_lib.node" "--module_name=bcrypt_lib" "--module_path=/usr/lib/node_modules/bcrypt/lib/binding"
gyp ERR! cwd /usr/lib/node_modules/bcrypt
gyp ERR! node -v v4.8.3
gyp ERR! node-gyp -v v3.6.1
gyp ERR! not ok
node-pre-gyp ERR! build error
node-pre-gyp ERR! stack Error: Failed to execute '/usr/bin/node /usr/lib/node_modules/node-gyp/bin/node-gyp.js build --fallback-to-build --module=/usr/lib/node_modules/bcrypt/lib/binding/bcrypt_lib.node --module_name=bcrypt_lib --module_path=/usr/lib/node_modules/bcrypt/lib/binding' (1)
node-pre-gyp ERR! stack at ChildProcess. (/usr/lib/node_modules/bcrypt/node_modules/node-pre-gyp/lib/util/compile.js:83:29)
node-pre-gyp ERR! stack at emitTwo (events.js:87:13)
node-pre-gyp ERR! stack at ChildProcess.emit (events.js:172:7)
node-pre-gyp ERR! stack at maybeClose (internal/child_process.js:862:16)
node-pre-gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:222:5)
node-pre-gyp ERR! System Linux 2.6.32-696.1.1.el6.x86_64
node-pre-gyp ERR! command "/usr/bin/node" "/usr/lib/node_modules/bcrypt/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd /usr/lib/node_modules/bcrypt
node-pre-gyp ERR! node -v v4.8.3
node-pre-gyp ERR! node-pre-gyp -v v0.6.32
node-pre-gyp ERR! not ok
Failed to execute '/usr/bin/node /usr/lib/node_modules/node-gyp/bin/node-gyp.js build --fallback-to-build --module=/usr/lib/node_modules/bcrypt/lib/binding/bcrypt_lib.node --module_name=bcrypt_lib --module_path=/usr/lib/node_modules/bcrypt/lib/binding' (1)
npm ERR! Linux 2.6.32-696.1.1.el6.x86_64
npm ERR! argv "/usr/bin/node" "/usr/bin/npm" "install" "bcrypt" "--g"
npm ERR! node v4.8.3
npm ERR! npm v3.10.9
npm ERR! code ELIFECYCLE

npm ERR! bcrypt@1.0.2 install: node-pre-gyp install --fallback-to-build
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the bcrypt@1.0.2 install script 'node-pre-gyp install --fallback-to-build'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the bcrypt package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! node-pre-gyp install --fallback-to-build
npm ERR! You can get information on how to open an issue for this project with:
npm ERR! npm bugs bcrypt
npm ERR! Or if that isn't available, you can get their info via:
npm ERR! npm owner ls bcrypt
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR! /root/npm-debug.log

Odd that it reports this as even when I chmod the directory to 0777

gyp WARN EACCES user "root" does not have permission to access the dev dir "/root/.node-gyp/4.8.3"

Any thoughts appreciated as I am well and truly stuck

[reetp]

Any ideas anybody ?

[megamaced]

@somsak I have deployed Rocketchat using your idea of overwriting web.cordova folder with the prebuild tgz version, although I only replace program.json and copy 8b30c08d68fc362b382342c0e204430d16399153.js

The mobile app works but now I noticed the logo asset doesn't work. It doesn't natter if using a custom logo or reverting to default. The mobile app just displays a small square with broken picture icon where the real logo should be. Do you get the same issue?

[somsak]

@megamaced yep the logo is also missing on my site (just noticed it, thanks!).

[alanadiwardana]

thanks a lot. This post was very helpful. We've had this same issue and overwriting web cordova folder brakes logo asset either. Anyone has clues about this behaviour?

[reetp]

Last night I finally got an error free build but not tested it yet.
I'll post my notes later, and I have a number of questions too.... stuff I did but don't really understand!

[reetp]

I took a look in the file
example-build.sh

With that I did the following:

curl https://install.meteor.com/ | sh
meteor npm install
cd ~/git/Rocket.Chat/packages/rocketchat-katex
npm install
cd ../..
mkdir ~/rocketbuild
meteor add rocketchat:internal-hubot
meteor build --server https://chat.myserver.co.uk --directory /home/me/gittest/build

That installed the katex fonts packages and then did a build but it complained about 'Unable to resolve some modules'

I went through each and tried to install them. These are the ones it seemed to want one way or another:

katex postcss-smart-import postcss-cssnext caniuse-db moment toastr highlight.js babel-runtime

I noticed that after I had installed the katex package into the Rocket.Chat/packages subdir and then installed again via

npm install -g katex postcss-smart-import postcss-cssnext caniuse-db moment toastr highlight.js babel-runtime jquery

I got a warning like this:

npm WARN The package postcss-cssnext is included as both a dev and production dependency.
npm WARN The package postcss-smart-import is included as both a dev and production dependency

The question is how do you install packages for building - globally, locally, and if so how to determine what is what and where ? What is dev and what is production ?

I am sure I have multiple copies all over the show now.

Also, where are the build options noted ? Do I need the --server and --directory directives ? I have seen there is also a --headless directive as well ? If I am running it as server only (no local clients,) do I need headless or server ?

This should all be documented somewhere - it's fine if you know and understand, but for those of who don't, and want to learn, it is VERY difficult.

[reetp]

This builds a tar.gz file (finally !):
meteor build /home/me/gittest/build

This builds the code in directories - same structure as the extracted tar.gz I think:
meteor build --server https://cat.myserver.com --directory /home/me/gittest/build

Still like to know about the build directives. I just found this
https://stackoverflow.com/questions/41942329/meteor-build-server-vs-root-url

So --server adds the URL into the code for Android devices. That means the build is specific to the server

If you do not use --server you have to specify the URL at startup of the server in your config file eg

export ROOT_URL=http://foo.com

So when building, unless you are JUST building for one server then leave out the --server directive

https://docs.meteor.com/commandline.html#meteorbuild (all about directives there)

--directory option will produce a bundle directory instead of the tarball.

[reetp]

Ahhh OK, bit more. I built it but when I ran it the server complains of missing modules.

Add this to find missing modules in the build directory:
https://www.npmjs.com/package/npm-install-missing

npm install -g npm-install-missing

Within your project directory:
npm-install-missing

I think it is all well and good if you have a fully setup node build environment. You do not have to think about dependencies and modules or whatever as it is all 'just there' (probably the same if you develop in PHP or Perl and you have all the right packages installed since wayback when and don't then about it)

For those of use wanting to just build Rocket and want create a build environment just for this, it is trickier.

I'll see if the next build works :-)

[geekgonecrazy]

To build and run:

git pull
meteor npm i # meteor npm install is valid too
meteor build --headless --directory /tmp/build 
mv /tmp/build/bundle /your/desired/path
cd /your/desired/path
cd programs/server
npm install
cd ../..
node main.js

Once you have the bundle instructions are exact same as any of our instructions.

[reetp]

@geekgonecrazy Thanks for that. Note that this is to build for running on a different server, not on my build server ( I have a box I use just for building packages). So I need a tar.gz and then to copy that elsewhere (or build into an RPM)

Ran your commands.

[me@buildserver Rocket.Chat]$ meteor npm -i

Usage: npm

where is one of:
'blah'

npm@4.6.1 /home/me/.meteor/packages/meteor-tool/.1.5.0.8sw1so++os.linux.x86_64+web.browser+web.cordova/mt-os.linux.x86_64/dev_bundle/lib/node_modules/npm

meteor build --headless ~/build

[me@builder Rocket.Chat]$ meteor build --headless ~/build
Error: ENOENT: no such file or directory, scandir 'packages/rocketchat-katex/node_modules/katex/dist/fonts/'

cd Rocket.Chat/packages

npm install

chromedriver@2.30.1 install /home/john/gittest/Rocket.Chat/node_modules/chimp/node_modules/chromedriver
node install.js

module.js:327
throw err;
^
Error: Cannot find module 'json-stable-stringify'

npm ERR! chromedriver@2.30.1 install: node install.js
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the chromedriver@2.30.1 install script 'node install.js'.
npm ERR! This is most likely a problem with the chromedriver package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! node install.js
npm ERR! You can get information on how to open an issue for this project with:
npm ERR! npm bugs chromedriver

Tried npm-install-missing and that seemed happy

OK, lets try another build

native: ENOENT: no such file or directory, scandir 'packages/rocketchat-katex/node_modules/katex/dist/fonts/'

Grrrr.

cd packages/rocketchat-katex; npm install katex

And again. Same error. BASH HEAD. Had enough for one night. It HAS to be easier than this.......

Desperate roll of the dice

npm install -g katex

Build again. Success.

Copy to chat server, cd Rocket.Chat/programs/server, npm install and rocket chat start and it starts !

➔ System ➔ startup
➔ +-----------------------------------------------------+
➔ | SERVER RUNNING |
➔ +-----------------------------------------------------+
➔ | |
➔ | Rocket.Chat Version: 0.57.0-develop |
➔ | NodeJS Version: 4.8.3 - x64 |
➔ | Platform: linux |
➔ | Process Port: 3000 |
➔ | Site URL: https://chat.myserver.com |
➔ | ReplicaSet OpLog: Disabled |
➔ | Commit Hash: 5cb30b4 |
➔ | Commit Branch: develop |

But seems to hang on the initial screen with the 3 little dots with no login page generated.

I give up :-(

[vfseanm]

Bringing this back, I'm having similar issues to what @reetp was having, across a variety of OS's.

Can someone please provide instructions on how to build the .tar.gz from source code??

I want to contribute but all the tutorials teach you is how to run the pre-built release code. I'm surprised this is missing from the documentation - are there guides reetp and I are missing?

[sampaiodiego]

@vfseanm the build instructions should be simple like this:

$ git clone https://github.com/RocketChat/Rocket.Chat.git
$ cd Rocket.Chat
$ meteor npm install
$ meteor build /tmp

if you have any issues running that, please open a new issue describing as much as you can so we can help you.

[reetp]

Just trying this on my existing build system now. Will report back.

[reetp]

Trying to build headless on CentOS 6

meteor build --headless /tmp
[john@builder Rocket.Chat]$ meteor build --headless /tmp

/home/john/.meteor/packages/meteor-tool/.1.5.2.nfo91l++os.linux.x86_64+web.browser+web.cordova/mt-os.linux.x86_64/dev_bundle/bin/node: /lib64/libc.so.6: version `GLIBC_2.15' not found (required by /home/john/.meteor/packages/meteor-tool/.1.5.2.nfo91l++os.linux.x86_64+web.browser+web.cordova/mt-os.linux.x86_64/dev_bundle/bin/node)

/home/john/.meteor/packages/meteor-tool/.1.5.2.nfo91l++os.linux.x86_64+web.browser+web.cordova/mt-os.linux.x86_64/dev_bundle/bin/node: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /home/john/.meteor/packages/meteor-tool/.1.5.2.nfo91l++os.linux.x86_64+web.browser+web.cordova/mt-os.linux.x86_64/dev_bundle/bin/node)

/home/john/.meteor/packages/meteor-tool/.1.5.2.nfo91l++os.linux.x86_64+web.browser+web.cordova/mt-os.linux.x86_64/dev_bundle/bin/node: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.15' not found (required by /home/john/.meteor/packages/meteor-tool/.1.5.2.nfo91l++os.linux.x86_64+web.browser+web.cordova/mt-os.linux.x86_64/dev_bundle/bin/node)