ashatora/openlayers
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Destroyed Moving to NPM and Grunt (markdown)

Tim Schaub
2017-09-25 23:28:35 -07:00
parent e39e384279
commit 1d47869a6f
1 changed files with 0 additions and 225 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Moving-to-NPM-and-Grunt.md
-225
@@ -1,225 +0,0 @@
**Outdated!**
This is a collection of information about, and hopefully decisions for, a move away from pake/build.py to nodejs for the cli development tool chain.
@tschaub is the primary motivator for this work as stated near the top of [this thread](https://groups.google.com/forum/#!topic/ol3-dev/FeE5dTFgel8), in summary
> I'm hoping this doesn't get taken the wrong way, but my motivation for node-ifying ol3 is that I see more promise in .js based tools for managing JavaScript projects. Things like gjslint are hard dependencies now, but I want to be positioned to leverage other tools as the .py ones become outdated.
The following is based on the assumption that there is general agreement to move the cli development tools to NodeJS, at least where possible, and from the various discussions there hasn't been a lot of push back on moving in this direction.
## Approach
The primary tool for managing dependencies and invoking various development tasks will be `npm`, which is included with all recent NodeJS versions.
The actual development tasks will be run via [Grunt](gruntjs.com). Grunt requires a grunt-cli module to be installed. To avoid another dependency the user has to install, it can be installed as a direct dependency via npm and various grunt tasks can be invoked by npm via run-scripts. This will keep some consistency in the command line descriptions (npm install, npm test, npm build etc) and keep the prerequisites to a minimum.
### npm run-scripts
npm run-scripts are scripts that are listed in the `scripts` section of the `package.json` file and are invoked on the cli via `npm <script-name>`.
npm currently has a limitation with run-scripts discussed in npm issues [3494](https://github.com/isaacs/npm/issues/3494) and [3313](https://github.com/isaacs/npm/issues/3313) in that you cannot pass arbitrary arguments to a run-script. This is a minor inconvenience at this point and they are actively working on changes that would allow this. What this means is we cannot have a single run-script target handle multiple things so it increases documentation tasks somewhat and perhaps has a less nice feel to using it on the command line, i.e.
npm test
npm integration-test
npm watch
versus
npm test
npm test --integration
npm test --watch
Initially, then, only the most obviously useful targets will be defined for things like `build`, which will build all the permutations, to simplify the target list. These would eventually evolve to include finer grained control once npm supports flags.
@elemoine has expressed some concerns about using npm run-scripts to wrap grunt tasks. If everyone agrees that asking a user to `npm install -g grunt-cli` is not a problem then the build system can easily be documented as grunt tasks.
## Using npm for installation
### From Source
`git clone https://github.com/openlayers/ol3.git`
`cd ol3`
`npm install` (depending on how install/check are used by other targets this may not be strictly necessary)
### From NPM
_assuming we publish an ol3 package with npm_
`npm install ol3`
### Steps
Both `npm install` and `npm install ol3` will process the `package.json` dependencies (and devDependencies if the environment variable `NODE_ENV` is **not** set to `production`). The `postinstall` script can be used to run an initial build if desired, this is what Tim has currently set up in his [use-node PR](https://github.com/openlayers/ol3/pull/804) - it runs `build.py build test-deps`
## Build Targets
### Current Situation
**WIP** - this is an attempt to map what build.py currently does for various targets
* build - build `ol.css`, `ol.js`, `ol-simple.js` and `ol-whitespace.js`
* check - runs `lint`, builds `ol.css`, `ol-all.js`, runs `test`
* todo/fixme - greps code for TODO and FIXME comments
* integration-test - runs `lint`, `build`, `build-all`, `test`, `build-examples`, `check-examples`, `doc`
* build-all - builds `ol-all.js`
* build-examples - runs `examples`, `EXAMPLES_COMBINED`
* examples - ultimately runs bin/exampleparser.py
* lint - runs gjslint on recently modified src, generated and requires files and a whitespace check
* plovr - installs plovr
* doc - builds documentation
* test-deps
* test
* checkdeps - checks for the existence of executables that cannot be managed by ol3, i.e. java, python, git, gjslint, jsdoc, phantomjs (NOTE we can pare this down with npm)
* reallyclean
* serve - runs `plover`, `test-deps` and `examples`
* serve-integration-test - runs `plovr`, `test-deps`, `INTERNAL_SRC` and runs plovr on ol-all.json and test.json.
* host-resources - cleans build/hosts/<branch>/resources and deploys new resources there
* host-examples - runs `build`, `host-resources`, `examples`
## NPM scripts
* postinstall - grunt install
* test
* what Tim did!
* check [https://github.com/karma-runner/karma-closure/](https://github.com/karma-runner/karma-closure/)
* watch
* run tests in watch mode, automatically re-running tests on file changes
* not sure if this detects changes in source or just the tests
* probably should consider other uses of this, like running lint in watch mode to report errors as files are saved
* integration-test
* run an integration test against the examples to ensure that exports and requires are working
* build - grunt build
* doc - grunt doc
* lint - grunt lint
* clean - grunt clean
* reallyclean - grunt reallyclean
* todo/fixme - grunt todo
* serve - grunt serve
### Grunt Tasks
what grunt tasks are required (based on npm script targets) and how we can potentially satisfy them
* install - install / update dependencies
* status: not started
* [http://dl.google.com/closure-compiler/compiler-latest.zip](http://dl.google.com/closure-compiler/compiler-latest.zip)
* [http://closure-library.googlecode.com/files/closure-library-20130212-95c19e7f0f5f.zip](http://closure-library.googlecode.com/files/closure-library-20130212-95c19e7f0f5f.zip) - **or** `git clone https://code.google.com/p/closure-library/ build/closure-library`
* it would be nice to pick up environment variables that would use existing versions that a developer might install elsewhere and avoid downloading/installing dependencies
* how do we make sure the dependencies are updated? Is there an automatic way to do this or is it the responsibility of the developer to periodically do something (clean perhaps?) to trigger an update
* perhaps install always updates the dependencies to the current version (if not overridden by env variables) and the check task only installs if missing?
* (@probins) it may be possible to avoid a local java install for casual use if we used Google's web service - see https://github.com/weaver/scribbles/tree/master/node/google-closure/
* (@twpayne) glsl-unit latest npm package does not work, use https://code.google.com/p/glsl-unit/ instead
* check - checks for dependencies, runs lint and test
* status: not started
* check for the existence of a dependency
* other tasks use this to ensure dependencies are available
* should differentiate between dependencies we can install and ones we can't
* may possibly install missing dependencies
* might collapse with install task
* todo - outputs a list of TODO and FIXME comments in the source code
* status: complete
* uses [https://npmjs.org/package/grunt-todos](https://npmjs.org/package/grunt-todos)
* lint - runs gjslint on the source code
* status: complete
* uses [https://npmjs.org/package/grunt-gjslint](https://npmjs.org/package/grunt-gjslint) which uses [https://github.com/jmendiara/node-closure-linter-wrapper](https://github.com/jmendiara/node-closure-linter-wrapper)
* build - builds ol.js, ol-simple.js, ol-whitespace.js (touch ol.css)
* status: not started
* uses [https://npmjs.org/package/grunt-closure-compiler](https://npmjs.org/package/grunt-closure-compiler)
* doc - build documentation
* status: complete
* uses [https://npmjs.org/package/grunt-jsdoc](https://npmjs.org/package/grunt-jsdoc)
* reallyclean - this runs a git command to remove all untracked items from the working folder
* status: complete
* serve - starts a local web server that serves the examples, related static files and the js files in various modes to support debugging the examples
* status: not started
## Notes and Related Information
### Packages to investigate/watch
from @probins:
* there is a node port of calcdeps.py https://github.com/bramstein/calcdeps I have been trying this and am getting error msgs when running with ol3, but these may be due to my setup. It would be interesting to know if others also see these errors
* there is also an effort to get jsdoc to run under node https://github.com/jsdoc3/jsdoc/issues/93
### Dependencies
Java and Python will likely continue to be dependencies of the system configuration required to run the build tools because of the use of the closure compiler (java) and gjslint (python).
Need to consider the distinction between build tool dependencies and build dependencies within the system.
Need to ensure the build system is efficient, i.e. minimum number of steps to accomplish a task using only changed files where possible for instance.
### Email threads
[https://groups.google.com/forum/#!topic/ol3-dev/B2GUz9xvtUE](https://groups.google.com/forum/#!topic/ol3-dev/B2GUz9xvtUE) - Tim's initial Node and ol3 thread
[http://groups.google.com/forum/#!topic/ol3-dev/FeE5dTFgel8](http://groups.google.com/forum/#!topic/ol3-dev/FeE5dTFgel8) - Node and ol3 thread that got separated from the initial thread
### Related issues and PRs
* Plovr-free build (@twpayne) [#456](https://github.com/openlayers/ol3/pull/456)
* use-node (@tschaub) [#804](https://github.com/openlayers/ol3/pull/804)
* build should use more robust javascript parsing (@twpayne) [#308](https://github.com/openlayers/ol3/issues/308)
* Examples and Plovr (@elemoine) [#241](https://github.com/openlayers/ol3/issues/241)
### Notes from @twpayne
* Plovr provides a single file download that contains a known version the compiler and the compiler library (and more). Without Plovr, we'll have to download these another way.
* Plovr allows us to not repeat ourselves with our configurations (all our .json Plovr config files inherit frombase.json).
* Plovr is typically faster than running the compiler's calcdeps.py and closurebuilder.py scripts, notably because it avoids JVM start-up in server mode, and that the accessing the Closure library source files packed into the Plovr JAR is much faster than seeking for the files on disk.
* Plovr uses the compiler's Java API to invoke the compiler, which allows many more options to be set than can be set via the compiler's command line.
* Plovr generally makes using the Closure tools much easier than using the individual scripts.
* I think that building the examples with the compiler is an extremely useful validation step and should remain part of the integration tests.
### Replacing Plovr
These things will need to be downloaded:
* [http://closure-linter.googlecode.com/files/closure_linter-latest.tar.gz](http://closure-linter.googlecode.com/files/closure_linter-latest.tar.gz)
* [http://dl.google.com/closure-compiler/compiler-latest.zip](http://dl.google.com/closure-compiler/compiler-latest.zip)
* [http://closure-library.googlecode.com/files/closure-library-20130212-95c19e7f0f5f.zip](http://closure-library.googlecode.com/files/closure-library-20130212-95c19e7f0f5f.zip) - **or** `git clone https://code.google.com/p/closure-library/ build/closure-library`
## Wiki Pages
* [https://github.com/openlayers/ol3/wiki/Build-Tools-Specs](https://github.com/openlayers/ol3/wiki/Build-Tools-Specs)
* [https://github.com/cedricmoullet/ol3/wiki/My-HOWTO](https://github.com/cedricmoullet/ol3/wiki/My-HOWTO)
* [https://github.com/openlayers/ol3/wiki/Developer-Guide](https://github.com/openlayers/ol3/wiki/Developer-Guide)