New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactoring of Notebook JavaScript #7784
Comments
From what I have started to read, react also does not play nicely with external css. |
I might play around from the other side, (notebook structure) since I know nothing about JS text-editors. |
My understanding is that things like React are about efficient updates to the UI. We are really performing two completely separate sync operations:
Is it possible for things like React to not talk to the DOM at all? It seems like that's not the right model for getting content into codemirror, etc., though I'm not actually sure. |
What do you mean? Do you want to address "model to server" using something like React? As you correctly said, React is about encapsulating elements that talk efficiently to the DOM. That is the point of the view in the MVC paradigm in which React works as the view. I would argue against using Polymer in the near future. Polymer proposes a very intriguing idea but it is still in its infancy and currently works only as a proof of concept as it relies heavily on polyfills to achieve its goal. Furthermore, web standards are notoriously slow to be accepted. If you are thinking about refactoring the Javascript code in the near future (months or even one or two years), then it is very unlikely that Polymer is going to be production-ready. React is very nice. As other suggestions, AngularJS is very popular, Ember is also popular but very idiosyncratic. By the way, I noticed you have Backbone as dependency but I don't think you're using it to manage the models. Is this correct? |
Re alternatives: Jason mentioned mithril in this context. I don't know what
the advantages/disadvantages are.
|
@minrk @rsmith31415 Regarding React, what I meant was concerned that React forces you to have a single model object for the whole document, which it renders to the DOM with every change to this object. It does allow components to be stateful, but it seems to maintain a global "state map" (see http://blog.circleci.com/local-state-global-concerns/) Given that react seems to be pretty flexible, I think it will probably be able to work with whatever we do. @rsmith31415 Regarding Polymer, these are good points (btw I work at Google so that might give me a different perspective). We definitely do want things working now. @takluyver Regarding Mithril, looks very similar to React, but I'm assuming React is overall a better choice because it's a bigger project. Not dismissing it but for now I would favor React. Given all this, I think it makes sense to start prototyping with React (and maybe Omniscient) and see where that gets us. Even if we don't choose this framework, it's probably the fastest to prototype with, so we can learn something about what the constraints are. |
Is it worth doing quick prototypes with two or three of the available
frameworks? This is a really important decision, and there will be some
time to play with them while we're splitting the repos before any of this
refactoring can really happen in the IPython codebase.
|
Sorry internet was off yesterday :-( I'll respond to all later. I think one of the main advantage of things like react is that actually you don't have to bind The main problem right now is that the current notebook UI is a state machine which has only one entry point, So you need to describe visually the app purely as a function of the model; Le 14 févr. 2015 à 10:23, Kester Tong notifications@github.com a écrit :
|
I like mithril, as well as the newly-rereleased riot. Both are really cut-down, and have a lower barrier to entry than a big framework like Angular/React, despite the far smaller communities. But you then still have to solve the OT thing. Likely the current comms model is able to support this, but that's mostly outside my ken. However, I'll throw out derbyjs as a maturing nodejs/browser (i.e. isomorphic) framework that handles OT from front to back (under the covers is ShareJS and racer). Presumably, a given "room" in nodejs would start a kernel against a notebook, so there would still only be one kernel. It maintains a virtual DOM like react and friends, but because it also runs on the server, it can serve up the exact expected state of the DOM for someone just showing up when using its own templating syntax. It is possible to use any other javascript library as well, such as this codemirror example, but obviously it's not going to be pushing an already-evented Codemirror over the wire. The big win of derby OT is that everything is handled entirely at an evented model level, where a model is big chunk of JSON-like data: thus it maps very cleanly to the notebook format. No messages, no callbacks, no pubsub. This should feel comfortable for someone who has worked in Backbone, as the models react like There are reactive model queries, etc. as well, but these would not be used for the baseline notebook interface... but would make the file browser, for example, much easier. In addition to the model, there are additional namespaces like Of additional interest is derby-standalone (demo) which wraps up the whole framework into a single file. This could be a very exciting way to deploy a notebook, as many interactive features could be maintained working against the in-memory model. A little prototype-off sounds pretty interesting! |
@KesterTong that is how om works (a clojurescript wrapper over react), but not how React has to work, or how it generally works in practice. Components maintain local state for things they There it some skill involved in designing any large js app to be performant, but I have found that React makes things much easier than any other framework. My experience: writing notablemind, which works as an alternate frontend to ipython, and handles large documents/notebooks (such as the tutorial) with relative ease, and that's without putting much effort into optimizing for speed (there's a lot of low-hanging fruit there, made simple by react). |
Another huge plus for me with React is that it really lets you have modularity. Declaring a component just in javascript (instead of split between js+handlebars/angularbars) buys you so much. In my experience, Angular projects are much harder to maintain at scale. Ember has some good conventions to make things work for large teams, but all around TooMuchMagic for my tastes. |
thanks for the info @jaredly. Reading the docs I got the impression that all the state tends to be stored in the top level component and passed down through the tree as properties. I'm still not quite clear on how stateful components interact. Can you tell me more about this in general and for notablemind? I can imagine two patterns. The first is that the parent component keeps refs for the child component, which it uses whenever it needs to query state from the child component. The second is that the child component's state is managed by a model object that lives outside the child component's state, and whose changes are listened for by the component. The second pattern seems to be a requirement when we use external APIs like Google Drive or derby.js which define their own model objects. I guess the second pattern equates to using React as the View only. |
Good talk yesterday, sorry I had to dip out before we got down to the nitty gritty. Derby components are the same, but different, from/to react: both have an virtual dom. however, with derby one still has a split between html and javascript... or any compile-to-js/html languages. Here are the components for derby-notebook. Here's how the model chain stretches back to the server:
What I like about this is that there is only the one metaphor for interacting with data: /store|route|component/.model./get|set|.../(path, value, function(err){}) |
Some status: I've broken out markdown cells as a specific thing to implement the render-edit loop, as well as capture the concept of "I want this rendered" while "that other person doesn't". Works pretty good!
I've also started thinking about user identity, activity, presence, and chat, as in discussing with colleagues it seems pretty clear that a robust (and preferrably undoable (and preferrably computable)) history is needed: look away, and who knows what happened! At first, I'll likely just let the
is really not very far away. The computable part would be being able to access this kind of history in the notebook itself, which is terrifyingly meta. We're thinking voting widgets, brainstorming with word clouds, etc. Additionally, I've added a little documentation of how my data model represents the stateful notebook. If none of my other stuff lives past my research here, perhaps this can! |
@bollwyvl you might want to check out the coLaboratory Chrome App to get some ideas on identity and presence. We also use google-docs style comments internally, where comments are attached to a particular cell. |
I haven't checked out coLaboratory... I should definitely give it a shot! I Sure, i see "cell" (annotation) as one of the key axes of messages, though Actually, easy uploading and annotation of pdf is one of the must-have I look forward to the continuing discussion... Has there been any more
|
ot.js is much cleaner than share.js. It has an implementation in Javascript and Python and CodeMirror support. I'm not familiar with derbyjs but I would rather have separate implementations for the UI and OT. In my case, I already have an OT server that is not using node.js (or sharejs). |
@benbro we're trying various prototypes at various levels here. I suppose the discussion will probably end up being about the one The Jupyter Realtime Notebook, which as much as anything will be driven by the modularity at each layer, but i think experimentation is the order of the day. But yeah, we will have to pick an stack for what is eventually shipped. What are your needs and goals? What is missing from the comms spec that you want for your integration? We're definitely going to need to support different rt/ot systems, and the simpler, more standards-based, multiple-implementation, the better. I had looked at substance and ot.js some time back, i'll have to circle back around and see what has developed. Can you point us at any specific highlights? The derby prototype is not affiliated with ipython/jupyter, aside from this discussion, and is at least partially in support of creating a more complex open source example of a derby application... One can only look at so many todo examples. However, i feel like it may indeed not be the right long term answer because it is so dominated by one major contributor. That being said, it works pretty good, and suits the work that i am doing. It actually is quite modular, as while sharejs is sharejs, racer (the model layer) and derby (the ui layer) can be used independently (derby-standalone). |
Hi Everyone, I just wanted to update everyone with our current thinking on frameworks. This is a consensus based on discussion in the IPython developers meeting in March, and meetings with @ellisonbg @Carreau @jasongrout @sccolbert @SylvainCorlay and myself.
|
@KesterTong is there any info about phosphor like a website, blog, talk or online examples? |
There are examples I don't remember where. |
https://github.com/phosphorjs/phosphor/tree/master/examples The phosphor-ui and phosphor-core repos have been merged into the phosphor On Thu, Mar 26, 2015 at 8:07 PM, Matthias Bussonnier <
|
Hey, I've been working on a project (still in very early stages) to make writing IPython notebook-like apps easier. I'm using AngularJS - I think I read elsewhere you may have already discounted angular, but maybe some of my code could still be useful. You can find it at https://github.com/dustyrockpyle/ipyng. I have a few novelties working with it, and getting a notebook app working is on my to-do list. Let me know if you have trouble with the build system or if you have any questions. |
Thanks for the pointer @dustyrockpyle. |
@Carreau @sccolbert @jasongrout The latest experiment with using phosphor to create a notebook interface is at https://github.com/KesterTong/phoshpor-notebook. My main goal was to transfer my knowledge of virtual-dom usage to @Carreau and others. The separation of model and view in the code is not very good. @Carreau can you coordinate with @jasongrout who already has a typescript notebook model? @sccolbert has been very helpful with phosphor related issues, so ping him with questions about the framework. |
btw https://github.com/KesterTong/phoshpor-notebook is just a fork of phosphor itself. I didn't figure out a way to use phosphor as a dependency except by following the same pattern as phosphor examples. To run, follow the instructions in the readme, then build the notebook with |
One big hesitation for me about phosphor: Is anyone using it? It has 3 On Wed, Apr 1, 2015 at 9:54 AM Kester Tong notifications@github.com wrote:
|
Technically, phosphor hits a sweet spot in that we can use the virtual dom when want it, but also mix in manual dom manipulation. Another advantage is being written in typescript. @ellisonbg initially noticed phosphor because of it's widgets/panels. These provide layout of multiple panes in js similar to a tiling window manager. I'm not very familiar with this side of things, but I think @ellisonbg feel strongly that Jupyter needs this kind of layout manager. Given that we would be using these widget/panels for the large scale layout, it seems natural to use phosphor for the virtual dom too if we can. Regarding the community support for the project, the following things ameliorate this problem:
Up til now we were only aware of React and Phosphor, so we should look into mithril and mercury. Are you aware if either of these has better support for typescript? |
Yes internally. It has 3
More, but repo was moved, changed to typescript and made public recently.
True, for test but typescript is annoying enough that you cot your test everytime you compile :-)
|
I share similar reservations about phosphor: testing, documentation, lack I am generally in favor of using a higher-order compile-to-JS language, and However, my initial introduction to typescript has a very NIH feel to it. I am +1 on mithril http://lhorie.github.io/mithril/index.html: you can RiotJS http:///is similar. It has its own template language, which is As I mentioned at length earlier, Derby is also worth considering: it can |
We can follow on phosphorjs/phosphor#8 |
Regarding Mithril, has anyone looked at the source code? I have. It's impenetrable and undocumented.
What? Gulp and Node.js are tenuous now? What's wrong with Stylus? Would you be okay with Less or Sass? What difference does any of this make? It builds and it works.
You must have taken a superficial look. It's been in development for years by Microsoft and they have deployed million+ SLOC projects which it. Angular 2.0 is written in TypeScript. etc... The API documentation for Phosphor is very thorough. Yes, there are no website/tutorials yet but that's because I spend my time making the library better. Phosphor provides much more than just a virtual DOM, see the discussions from yesterday: |
Let's move the phosphor discussion to the phosphor repo. On Wed, Apr 1, 2015 at 10:07 AM, S. Chris Colbert notifications@github.com
Brian E. Granger |
To elucidate a bit on what others have said: we did consider a range of JS frameworks at an in-person dev meeting in Berkeley a few weeks ago, and the concerns about Phosphor being a young and relatively unknown project were brought up. However, there are the following points in its favour:
There are also two pieces involved in this discussion:
(My understanding may be incorrect - other people who were in the discussions, please correct me if so) I wanted to write this here to explain why we believe Phosphor makes sense for IPython. As others have suggested, let's take the discussion of phosphor itself to phosphorjs/phosphor#8. |
@ellisonbg We should have our own issue to discuss this. I don't think it is fair to pollute @sccolbert's repository with discussions that are more focused on the use of phosphor in Ipython than on the merits of phosphor as an independent project. |
@rsmith31415 yes, I agree it would be useful to have a separate ipython On Wed, Apr 1, 2015 at 10:24 AM, rsmith31415 notifications@github.com
Brian E. Granger |
@ellisonbg Agreed. That's why I posted there the parts that are related only to phosphor. |
As @ellisonbg points out, these are concerns about phosphor-in-the-notebook-buildchain, and the developers that will have to use it, and may not represent my personal views :)
Like i said, breaks on travis. Could have been related to the DDoS of github over the weekend.
I love stylus! My happiest prototyping environment is a single
See travis.
Very superficial indeed. Mainly just the type package manager rubbed me the wrong way. I've worked a touch with it in the past on webcola, and ran into several tooling issues if you are outside the visual studio mothership, but that was a year ago.
We're here to help, if this is what is chosen! But it does make it hard to evaluate: the generated doc is... thorough... but doesn't help me get an intuitive feel for what I should be doing. As we think about try.jupyter.org we could certainly consider using that as a sample case. |
@bollwyvl moved to: phosphorjs/phosphor#8 (comment) |
Back on a much earlier topic: mesh.js looks like a nice, backend/platform agnostic way to wrap up evented models. The good ol' Todos is quite concise... needs the good ol' Codemirror example, though! I could see this being a nice way to keep downstream code from being too vendor-specific: one adaptor for, say GDRT, one for some lightweight, shippable tornado app. |
Thanks for the pointer. I'm learning Phosphor/Typescript by re-writing slowly the notebook frontend here also, while we are posting links. |
A lot of good discussion on this issue, thanks everyone for participating. With jupyterlab being written atop phosphor, as I understand it, there is no longer an open question left here, so I'm going to go ahead and close this. Happy hacking! |
Hi Everyone,
I'm opening this issue as a place to discuss the planned refactoring of the notebook JavaScript code, that @Carreau and myself will start to investigate now. I think the main choice that has to be made is the JavaScript framework(s). We don't have to decide now, but we do have to decide what to prototype first. Here are my thoughts on various frameworks (possibly incorrect)
React: Very highly praised, but might require a data model that doesn't work for us. E.g. React + omniscient.js will do very efficient diffs of the virtual DOM, but still has to do this with every change. Can we do this every few keystrokes for realtime collaboration?
Polymer: Based on Shadow DOM, a new (proposed?) standard for encapsulating DOM components. In comparison, React is a workaround this lack of encapsulation. However Polymer still forces your to manually manipulate the DOM to build your components.
Others: I've probably left out a lot.
The text was updated successfully, but these errors were encountered: