News about Eclipse Freemarker

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

News about Eclipse Freemarker

angelozerr
Hi Daniel, Greg

For your informaiton, I have restarted from scratch the Freemarker Eclipse https://github.com/angelozerr/Freemarker-Eclipse-DLTK/tree/master/poc

Why?

My goal is to use Eclipse WTP HTML feature as based of the editor to mix HTML and Freemarker like PDT (Eclipse PHP) have done https://github.com/eclipse/pdt. They mix DLTK and WTP to manage their PHP editor and I would like to do the same thing to benefit with HTML features of WTP.

For the moment, my focus is to manage highlight (highlight HTML elements and Freemarker elements). To do that PDT has developped their own grammar (jflex) by modyfing the existing XMLTokenizer.jflex of WTP. I have started doing the same thing https://github.com/angelozerr/Freemarker-Eclipse-DLTK/blob/master/poc/org.eclipse.freemarker.core/devel/FMTokenizer.jflex

and for the moment I can highlight just HTML elements. I discover jflex so I must understand how it works.I try to use https://bitbucket.org/bazaarvoice/opengrok-patches/src/77be5bfb310b/freemarker-support

If you can help me about this topic, it should be very cool.

Why JFlex? Because WTP uses that and the XMLTokenizer.jflex are very robust, so PDT have done this choice. I try to do the same thing than PDT.

If you have time to help me about JFlex topic, it should be very cool. Many thank's.

Regards Angelo

------------------------------------------------------------------------------
November Webinars for C, C++, Fortran Developers
Accelerate application performance with scalable programming models. Explore
techniques for threading, error checking, porting, and tuning. Get the most
from the latest Intel processors and coprocessors. See abstracts and register
http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk
_______________________________________________
FreeMarker-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-devel
Reply | Threaded
Open this post in threaded view
|

Re: News about Eclipse Freemarker

Daniel Dekany
Hi,

It's great that something is going on again, and needless to say, in a
template language mixing in a HTML highlighting is very useful. If how
much time can I find for helping in it, we will see.

I wonder what will happen when the template is not for generating
HTML. Is there a hope that JFlex tokenizers can be generated by
re-using the common part, the FTL tokenizer? So then we could at least
have a FTL+HTML and an FTL+JustText tokenizer, without maintaining two
FTL tokenizers. (Or the coolest thing would be if the real tokenizer of
FreeMarker is based on the same JFlex tokenizer... and I will have to
rewrite the lexer anyway.)

Another thing... An issue that was probably present in all plugins so
far is that they hard-wire the directives and built-ins into the
lexer/parser. This misconception comes from that most languages has a
small and hardly ever changing set of statement-like stuff, like Java
has "if", "for", etc., and then they have a changing standard API. FTL
is different, as there these two things are one and the same. Anyway,
the situation is in big part the mistake of FreeMarker, as it doesn't
provide a way to query the available core directives and built-ins,
and because the directive syntax is not very regular. Like #list has
an "as" in it, #visit has a "using", etc. It doesn't feel like a
"standard API" that way. But that's something I intend to change in
the future (well, if will I ever get there given the huge FreeMarker
backlog I have), like the recommended form of <#import "foo.ftl" as x>
would be <#import x="foo.ftl">, of <#visit node using ns> would be
<#visit node, using=ns>, etc. It's the syntax of <@...>, except that
you don't have to use "/>" where it's known that the directive can't
have nested content. The old forms will have to remain for
backward-compatibility, and I think the special syntax of #list will
not even be deprecated by the generic one (that would be
<#list xs; x>, which I'm afraid is too scary for newcomers). But
otherwise the plan is that new directives will always follow the
regular syntax. So the lexer/parser should just look for any <#...>
and </#...> tags, then it should query FreeMarker if there's a
directive with that name, and if so, what form it expects inside
(reguar, #list-like, both, etc.), and if it can have a nested content
or it's implicitly closed. The same goes for the built-ins (the ?foo
things). Luckily, the syntax there is regular, but the legal names
(can already be queried in 2.3.20!), and number of accepted parameters
should be queried from FreeMarker again.

If the plugin works like that, updating freemarker.jar is all that's
needed to support new directives and built-ins. Plus, this can be
useful even to check user-defined directive calls (like macro calls,
function calls) in the future.

--
Thanks,
 Daniel Dekany


Friday, November 8, 2013, 12:14:53 PM, Angelo zerr wrote:

> Hi Daniel, Greg
>
> For your informaiton, I have restarted from scratch the Freemarker
> Eclipse
> https://github.com/angelozerr/Freemarker-Eclipse-DLTK/tree/master/poc
>
> Why?
>
> My goal is to use Eclipse WTP HTML feature as based of the editor
> to mix HTML and Freemarker like PDT (Eclipse PHP) have done
> https://github.com/eclipse/pdt. They mix DLTK and WTP to manage
> their PHP editor and I would like to do the same thing to benefit with HTML features of WTP.
>
> For the moment, my focus is to manage highlight (highlight HTML
> elements and Freemarker elements). To do that PDT has developped
> their own grammar (jflex) by modyfing the existing
> XMLTokenizer.jflex of WTP. I have started doing the same thing
> https://github.com/angelozerr/Freemarker-Eclipse-DLTK/blob/master/poc/org.eclipse.freemarker.core/devel/FMTokenizer.jflex
>
> and for the moment I can highlight just HTML elements. I discover
> jflex so I must understand how it works.I try to use
> https://bitbucket.org/bazaarvoice/opengrok-patches/src/77be5bfb310b/freemarker-support
>
> If you can help me about this topic, it should be very cool.
>
> Why JFlex? Because WTP uses that and the XMLTokenizer.jflex are
> very robust, so PDT have done this choice. I try to do the same thing than PDT.
>
> If you have time to help me about JFlex topic, it should be very cool. Many thank's.
>
> Regards Angelo


------------------------------------------------------------------------------
November Webinars for C, C++, Fortran Developers
Accelerate application performance with scalable programming models. Explore
techniques for threading, error checking, porting, and tuning. Get the most
from the latest Intel processors and coprocessors. See abstracts and register
http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk
_______________________________________________
FreeMarker-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-devel
Reply | Threaded
Open this post in threaded view
|

Re: News about Eclipse Freemarker

angelozerr
Hi Daniel,


2013/11/9 Daniel Dekany <[hidden email]>
Hi,

It's great that something is going on again, and needless to say, in a
template language mixing in a HTML highlighting is very useful. If how
much time can I find for helping in it, we will see.

It should be very cool! I'm trying to understand JFlex and I will try to create some JUnits with the generated FMTokenizer that I'm developping.
 

I wonder what will happen when the template is not for generating
HTML. Is there a hope that JFlex tokenizers can be generated by
re-using the common part, the FTL tokenizer? So then we could at least
have a FTL+HTML and an FTL+JustText tokenizer, without maintaining two
FTL tokenizers.

My idea is to modify the WTP XMLTokenizer to manage FM too like PDT has done for PHP. So we will have FMTokenizer which manages HTML+FM both.
If you wish to manage other mix language like Python +FM (ex : write some Python  template with FM), it will not work. But the common use of FM is for HTML.
 
(Or the coolest thing would be if the real tokenizer of
FreeMarker is based on the same JFlex tokenizer... and I will have to
rewrite the lexer anyway.)

No it will not possible, because if you see the FMTokenizer https://github.com/angelozerr/Freemarker-Eclipse-DLTK/blob/master/poc/org.eclipse.freemarker.core/devel/FMTokenizer.java, you will see that it depends on some Eclipse classes. 

Another thing... An issue that was probably present in all plugins so
far is that they hard-wire the directives and built-ins into the
lexer/parser. This misconception comes from that most languages has a
small and hardly ever changing set of statement-like stuff, like Java
has "if", "for", etc., and then they have a changing standard API. FTL
is different, as there these two things are one and the same. Anyway,
the situation is in big part the mistake of FreeMarker, as it doesn't
provide a way to query the available core directives and built-ins,
and because the directive syntax is not very regular. Like #list has
an "as" in it, #visit has a "using", etc. It doesn't feel like a
"standard API" that way. But that's something I intend to change in
the future (well, if will I ever get there given the huge FreeMarker
backlog I have), like the recommended form of <#import "foo.ftl" as x>
would be <#import x="foo.ftl">, of <#visit node using ns> would be
<#visit node, using=ns>, etc. It's the syntax of <@...>, except that
you don't have to use "/>" where it's known that the directive can't
have nested content. The old forms will have to remain for
backward-compatibility, and I think the special syntax of #list will
not even be deprecated by the generic one (that would be
<#list xs; x>, which I'm afraid is too scary for newcomers). But
otherwise the plan is that new directives will always follow the
regular syntax. So the lexer/parser should just look for any <#...>
and </#...> tags, then it should query FreeMarker if there's a
directive with that name, and if so, what form it expects inside
(reguar, #list-like, both, etc.), and if it can have a nested content
or it's implicitly closed. The same goes for the built-ins (the ?foo
things). Luckily, the syntax there is regular, but the legal names
(can already be queried in 2.3.20!), and number of accepted parameters
should be queried from FreeMarker again.

My first goal is to highlight the current syntax of FM. I have seen that PDT can manage several version of PHP.
To do that the main PHPTokenizer https://github.com/eclipse/pdt/blob/5bfa09e18cdc7814431d9fcc7bab4c58a7aa408f/plugins/org.eclipse.php.core/Resources/parserTools/highlighting/PHPTokenizer.jflex (which comes from XMLTokenizer) uses a PhpLexer according version. So we could do that, but at this time I want to manage just one syntax.  

If the plugin works like that, updating freemarker.jar is all that's
needed to support new directives and built-ins. Plus, this can be
useful even to check user-defined directive calls (like macro calls,
function calls) in the future.

Yes it should be cool too.

Regards Angelo
 
--
Thanks,
 Daniel Dekany


Friday, November 8, 2013, 12:14:53 PM, Angelo zerr wrote:

> Hi Daniel, Greg
>
> For your informaiton, I have restarted from scratch the Freemarker
> Eclipse
> https://github.com/angelozerr/Freemarker-Eclipse-DLTK/tree/master/poc
>
> Why?
>
> My goal is to use Eclipse WTP HTML feature as based of the editor
> to mix HTML and Freemarker like PDT (Eclipse PHP) have done
> https://github.com/eclipse/pdt. They mix DLTK and WTP to manage
> their PHP editor and I would like to do the same thing to benefit with HTML features of WTP.
>
> For the moment, my focus is to manage highlight (highlight HTML
> elements and Freemarker elements). To do that PDT has developped
> their own grammar (jflex) by modyfing the existing
> XMLTokenizer.jflex of WTP. I have started doing the same thing
> https://github.com/angelozerr/Freemarker-Eclipse-DLTK/blob/master/poc/org.eclipse.freemarker.core/devel/FMTokenizer.jflex
>
> and for the moment I can highlight just HTML elements. I discover
> jflex so I must understand how it works.I try to use
> https://bitbucket.org/bazaarvoice/opengrok-patches/src/77be5bfb310b/freemarker-support
>
> If you can help me about this topic, it should be very cool.
>
> Why JFlex? Because WTP uses that and the XMLTokenizer.jflex are
> very robust, so PDT have done this choice. I try to do the same thing than PDT.
>
> If you have time to help me about JFlex topic, it should be very cool. Many thank's.
>
> Regards Angelo



------------------------------------------------------------------------------
November Webinars for C, C++, Fortran Developers
Accelerate application performance with scalable programming models. Explore
techniques for threading, error checking, porting, and tuning. Get the most
from the latest Intel processors and coprocessors. See abstracts and register
http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk
_______________________________________________
FreeMarker-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-devel
Reply | Threaded
Open this post in threaded view
|

Re: News about Eclipse Freemarker

Daniel Dekany
Yes, now it depends on Eclipse classes, but I thought that since JFlex
is not part of Eclipse, maybe it's possible to create a pure JFlex FTL
scanner and then generate the others (FTL+HTML, etc) from it... but I
really don't know. Obviously, inside FreeMarker we don't need the HTML
parser of WTP, only the FTL parser. OTOH I don't even know if I want
to use JFlex in FM, so I was just thinking loudly.

As of composing an FTL lexer/parser and another one together... this
is really the same problem, as the real parser is just a plainText+FTL
parser. Again, I don't know the technical details, only there's the
thing that as an algorithm problem it's most certainly solvable. When
the "outer" parser (like the one dealing with HTML) hits "<#" or "<@"
or "</#" or "</@" or "${" or "#{", it must enter FTL mode regardless
of where it occurs in the outer language, and then FTL will completely
take over. The "outer" language doesn't mater when it comes to parsing
the inside of the FTL pieces. Hence the parsing of the inside of the
FTL pieces should be reusable. (Where the outer language parser allow
the FTL pieces to start is another question. In theory it should allow
it everywhere, but of course no one expects the outer language parser
to get out of all kind of situations.)

--
Thanks,
 Daniel Dekany


Sunday, November 10, 2013, 12:34:38 AM, Angelo zerr wrote:

> Hi Daniel,
>
>
> 2013/11/9 Daniel Dekany <[hidden email]>
> Hi,
>
> It's great that something is going on again, and needless to say, in a
> template language mixing in a HTML highlighting is very useful. If how
> much time can I find for helping in it, we will see.
>
> It should be very cool! I'm trying to understand JFlex and I will
> try to create some JUnits with the generated FMTokenizer that I'm developping.
>  
>
> I wonder what will happen when the template is not for generating
> HTML. Is there a hope that JFlex tokenizers can be generated by
> re-using the common part, the FTL tokenizer? So then we could at least
> have a FTL+HTML and an FTL+JustText tokenizer, without maintaining two
> FTL tokenizers.
>
> My idea is to modify the WTP XMLTokenizer to manage FM too like PDT
> has done for PHP. So we will have FMTokenizer which manages HTML+FM both.
> If you wish to manage other mix language like Python +FM (ex :
> write some Python  template with FM), it will not work. But the common use of FM is for HTML.
>  
> (Or the coolest thing would be if the real tokenizer of
> FreeMarker is based on the same JFlex tokenizer... and I will have to
> rewrite the lexer anyway.)
>
> No it will not possible, because if you see the FMTokenizer
> https://github.com/angelozerr/Freemarker-Eclipse-DLTK/blob/master/poc/org.eclipse.freemarker.core/devel/FMTokenizer.java,
> you will see that it depends on some Eclipse classes.
>
> Another thing... An issue that was probably present in all plugins so
> far is that they hard-wire the directives and built-ins into the
> lexer/parser. This misconception comes from that most languages has a
> small and hardly ever changing set of statement-like stuff, like Java
> has "if", "for", etc., and then they have a changing standard API. FTL
> is different, as there these two things are one and the same. Anyway,
> the situation is in big part the mistake of FreeMarker, as it doesn't
> provide a way to query the available core directives and built-ins,
> and because the directive syntax is not very regular. Like #list has
> an "as" in it, #visit has a "using", etc. It doesn't feel like a
> "standard API" that way. But that's something I intend to change in
> the future (well, if will I ever get there given the huge FreeMarker
> backlog I have), like the recommended form of <#import "foo.ftl" as x>
> would be <#import x="foo.ftl">, of <#visit node using ns> would be
> <#visit node, using=ns>, etc. It's the syntax of <@...>, except that
> you don't have to use "/>" where it's known that the directive can't
> have nested content. The old forms will have to remain for
> backward-compatibility, and I think the special syntax of #list will
> not even be deprecated by the generic one (that would be
> <#list xs; x>, which I'm afraid is too scary for newcomers). But
> otherwise the plan is that new directives will always follow the
> regular syntax. So the lexer/parser should just look for any <#...>
> and </#...> tags, then it should query FreeMarker if there's a
> directive with that name, and if so, what form it expects inside
> (reguar, #list-like, both, etc.), and if it can have a nested content
> or it's implicitly closed. The same goes for the built-ins (the ?foo
> things). Luckily, the syntax there is regular, but the legal names
> (can already be queried in 2.3.20!), and number of accepted parameters
> should be queried from FreeMarker again.
>
> My first goal is to highlight the current syntax of FM. I have seen
> that PDT can manage several version of PHP.
> To do that the main PHPTokenizer
> https://github.com/eclipse/pdt/blob/5bfa09e18cdc7814431d9fcc7bab4c58a7aa408f/plugins/org.eclipse.php.core/Resources/parserTools/highlighting/PHPTokenizer.jflex
> (which comes from XMLTokenizer) uses a PhpLexer according version.
> So we could do that, but at this time I want to manage just one syntax.
>
> If the plugin works like that, updating freemarker.jar is all that's
> needed to support new directives and built-ins. Plus, this can be
> useful even to check user-defined directive calls (like macro calls,
> function calls) in the future.
>
> Yes it should be cool too.
>
> Regards Angelo
>  
> --
> Thanks,
>  Daniel Dekany
>
>
> Friday, November 8, 2013, 12:14:53 PM, Angelo zerr wrote:
>
>> Hi Daniel, Greg
>>
>> For your informaiton, I have restarted from scratch the Freemarker
>> Eclipse
>> https://github.com/angelozerr/Freemarker-Eclipse-DLTK/tree/master/poc
>>
>> Why?
>>
>> My goal is to use Eclipse WTP HTML feature as based of the editor
>> to mix HTML and Freemarker like PDT (Eclipse PHP) have done
>> https://github.com/eclipse/pdt. They mix DLTK and WTP to manage
>> their PHP editor and I would like to do the same thing to benefit with HTML features of WTP.
>>
>> For the moment, my focus is to manage highlight (highlight HTML
>> elements and Freemarker elements). To do that PDT has developped
>> their own grammar (jflex) by modyfing the existing
>> XMLTokenizer.jflex of WTP. I have started doing the same thing
>> https://github.com/angelozerr/Freemarker-Eclipse-DLTK/blob/master/poc/org.eclipse.freemarker.core/devel/FMTokenizer.jflex
>>
>> and for the moment I can highlight just HTML elements. I discover
>> jflex so I must understand how it works.I try to use
>> https://bitbucket.org/bazaarvoice/opengrok-patches/src/77be5bfb310b/freemarker-support
>>
>> If you can help me about this topic, it should be very cool.
>>
>> Why JFlex? Because WTP uses that and the XMLTokenizer.jflex are
>> very robust, so PDT have done this choice. I try to do the same thing than PDT.
>>
>> If you have time to help me about JFlex topic, it should be very cool. Many thank's.
>>
>> Regards Angelo
>
>


------------------------------------------------------------------------------
November Webinars for C, C++, Fortran Developers
Accelerate application performance with scalable programming models. Explore
techniques for threading, error checking, porting, and tuning. Get the most
from the latest Intel processors and coprocessors. See abstracts and register
http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk
_______________________________________________
FreeMarker-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-devel
Reply | Threaded
Open this post in threaded view
|

Re: News about Eclipse Freemarker

Greg Amerson
In reply to this post by angelozerr
Hey Angelo,

I like the direction you are going with this new plugin.  Let me know once you have something to share and I can help test/debug in my environment.


On Fri, Nov 8, 2013 at 7:14 PM, Angelo zerr <[hidden email]> wrote:
Hi Daniel, Greg

For your informaiton, I have restarted from scratch the Freemarker Eclipse https://github.com/angelozerr/Freemarker-Eclipse-DLTK/tree/master/poc

Why?

My goal is to use Eclipse WTP HTML feature as based of the editor to mix HTML and Freemarker like PDT (Eclipse PHP) have done https://github.com/eclipse/pdt. They mix DLTK and WTP to manage their PHP editor and I would like to do the same thing to benefit with HTML features of WTP.

For the moment, my focus is to manage highlight (highlight HTML elements and Freemarker elements). To do that PDT has developped their own grammar (jflex) by modyfing the existing XMLTokenizer.jflex of WTP. I have started doing the same thing https://github.com/angelozerr/Freemarker-Eclipse-DLTK/blob/master/poc/org.eclipse.freemarker.core/devel/FMTokenizer.jflex

and for the moment I can highlight just HTML elements. I discover jflex so I must understand how it works.I try to use https://bitbucket.org/bazaarvoice/opengrok-patches/src/77be5bfb310b/freemarker-support

If you can help me about this topic, it should be very cool.

Why JFlex? Because WTP uses that and the XMLTokenizer.jflex are very robust, so PDT have done this choice. I try to do the same thing than PDT.

If you have time to help me about JFlex topic, it should be very cool. Many thank's.

Regards Angelo



--
Greg Amerson
Liferay Developer Tools
Liferay, Inc. www.liferay.com

------------------------------------------------------------------------------
DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps
OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access
Free app hosting. Or install the open source package on any LAMP server.
Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native!
http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk
_______________________________________________
FreeMarker-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-devel