Introducing freemarker-introspection

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Introducing freemarker-introspection

Wong, Christopher

I would like to introduce a new project, “freemarker-introspection”. Meant to supplement the core Freemarker project, the purpose of this project is to provide a programmatic way to inspect and modify a Freemarker template. While Freemarker has an object representation of a parsed template, this representation is generally not externally accessible. The primary feature of freemarker-introspection is to expose the parsed Freemarker template as a DOM-like AST object that you can navigate in Java:

 

        Element root = TemplateIntrospector.getRootNode(template);

        for (Element child : root.getChildren()) {

            for (Expr param : child.getParams()) {

                …

            }

        }

 

You can also use a Visitor pattern to walk the template nodes. I implemented 2 classes that serve as consumers of this output.

 

VariableFinder scans the template and returns information on variables that the template will reference from the model:

 

        Element root = TemplateIntrospector.getRootNode(template);

        List<VariableInfo> vars = new VariableFinder(root).seek().getVariableInfo();

 

TemplateEditor lets you replace specific nodes (elements and expressions) with your own content:

 

        Element root = TemplateIntrospector.getRootNode(template);

        Element elem = ...; // find element node in tree to replace

        Expr expr = ...; // find expression node in tree to replace

        String newTemplateText = new TemplateEditor(oldTemplateText)

                .replace(elem, "<#include 'something_else.ftl'>")

                .replace(expr, "123")

                .apply()

                .getModifiedTemplate();

 

You can find freemarker-introspection at:

 

https://github.com/cwong15/freemarker-introspection

 

You will need Gradle to build it. It currently works with Freemarker versions 2.3.19 and 2.3.20. This project is still in its very early stages, so your feedback will be very much welcome.

 

Chris



This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.

------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
FreeMarker-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-user
Reply | Threaded
Open this post in threaded view
|

Re: Introducing freemarker-introspection

Daniel Dekany
Do you plan to work on this against 2.3.21, or if something is missing
from the freemarker.core-package-visible API, then against the trunk?
If I refactor something inside freemarker.core later, it's much easier
to keep freemarker-introspection in sync with it if it relies on
freemkarker.core-package-access rather than on reflection.


Friday, June 13, 2014, 3:03:49 PM, Wong, Christopher wrote:

> I would like to introduce a new project,
> “freemarker-introspection”. Meant to supplement the core Freemarker
> project, the purpose of this project is to provide a programmatic
> way to inspect and modify a Freemarker template. While Freemarker
> has an object representation of a parsed template, this
> representation is generally not externally accessible. The primary
> feature of freemarker-introspection is to expose the parsed
> Freemarker template as a DOM-like AST object that you can navigate in Java:
>  
>         Element root = TemplateIntrospector.getRootNode(template);
>         for (Element child : root.getChildren()) {
>             for (Expr param : child.getParams()) {
>                 …
>             }
>         }
>  
> You can also use a Visitor pattern to walk the template nodes. I
> implemented 2 classes that serve as consumers of this output.
>  
> VariableFinder scans the template and returns information on
> variables that the template will reference from the model:
>  
>         Element root = TemplateIntrospector.getRootNode(template);
>         List<VariableInfo> vars = new
> VariableFinder(root).seek().getVariableInfo();
>  
> TemplateEditor lets you replace specific nodes (elements and
> expressions) with your own content:
>  
>         Element root = TemplateIntrospector.getRootNode(template);
>         Element elem = ...; // find element node in tree to replace
>         Expr expr = ...; // find expression node in tree to replace
>         String newTemplateText = new TemplateEditor(oldTemplateText)
>                 .replace(elem, "<#include 'something_else.ftl'>")
>                 .replace(expr, "123")
>                 .apply()
>                 .getModifiedTemplate();
>  
> You can find freemarker-introspection at:
>  
> https://github.com/cwong15/freemarker-introspection
>  
> You will need Gradle to build it. It currently works with
> Freemarker versions 2.3.19 and 2.3.20. This project is still in its
> very early stages, so your feedback will be very much welcome.
>  
> Chris
>
> This e-mail and files transmitted with it are confidential, and are
> intended solely for the use of the individual or entity to whom this
> e-mail is addressed. If you are not the intended recipient, or the
> employee or agent responsible to deliver it to the intended
> recipient, you are hereby notified that any dissemination,
> distribution or copying of this communication is strictly
> prohibited. If you are not one of the named recipient(s) or
> otherwise have reason to believe that you received this message in
> error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
--
Thanks,
 Daniel Dekany


------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
FreeMarker-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-user
Reply | Threaded
Open this post in threaded view
|

Re: Introducing freemarker-introspection

Wong, Christopher
I agree that the current reflection-based implementation is less maintainable than using the package-access methods you have available starting in 2.3.20. The current approach is only temporary, to allow support for an older version of Freemarker. I will replace it with your recommended approach. This list announcement is to solicit feedback on the public interfaces for freemarker-introspection, which should be independent of the underlying implementation.

Chris

-----Original Message-----
From: Daniel Dekany [mailto:[hidden email]]
Sent: Saturday, June 14, 2014 7:10 AM
To: FreeMarker-user
Subject: Re: [FreeMarker-user] Introducing freemarker-introspection

Do you plan to work on this against 2.3.21, or if something is missing from the freemarker.core-package-visible API, then against the trunk?
If I refactor something inside freemarker.core later, it's much easier to keep freemarker-introspection in sync with it if it relies on freemkarker.core-package-access rather than on reflection.


Friday, June 13, 2014, 3:03:49 PM, Wong, Christopher wrote:

> I would like to introduce a new project, "freemarker-introspection".
> Meant to supplement the core Freemarker project, the purpose of this
> project is to provide a programmatic way to inspect and modify a
> Freemarker template. While Freemarker has an object representation of
> a parsed template, this representation is generally not externally
> accessible. The primary feature of freemarker-introspection is to
> expose the parsed Freemarker template as a DOM-like AST object that
> you can navigate in Java:
>
>         Element root = TemplateIntrospector.getRootNode(template);
>         for (Element child : root.getChildren()) {
>             for (Expr param : child.getParams()) {
>                 ...
>             }
>         }
>
> You can also use a Visitor pattern to walk the template nodes. I
> implemented 2 classes that serve as consumers of this output.
>
> VariableFinder scans the template and returns information on variables
> that the template will reference from the model:
>
>         Element root = TemplateIntrospector.getRootNode(template);
>         List<VariableInfo> vars = new
> VariableFinder(root).seek().getVariableInfo();
>
> TemplateEditor lets you replace specific nodes (elements and
> expressions) with your own content:
>
>         Element root = TemplateIntrospector.getRootNode(template);
>         Element elem = ...; // find element node in tree to replace
>         Expr expr = ...; // find expression node in tree to replace
>         String newTemplateText = new TemplateEditor(oldTemplateText)
>                 .replace(elem, "<#include 'something_else.ftl'>")
>                 .replace(expr, "123")
>                 .apply()
>                 .getModifiedTemplate();
>
> You can find freemarker-introspection at:
>
> https://github.com/cwong15/freemarker-introspection
>
> You will need Gradle to build it. It currently works with Freemarker
> versions 2.3.19 and 2.3.20. This project is still in its very early
> stages, so your feedback will be very much welcome.
>
> Chris
>
> This e-mail and files transmitted with it are confidential, and are
> intended solely for the use of the individual or entity to whom this
> e-mail is addressed. If you are not the intended recipient, or the
> employee or agent responsible to deliver it to the intended recipient,
> you are hereby notified that any dissemination, distribution or
> copying of this communication is strictly prohibited. If you are not
> one of the named recipient(s) or otherwise have reason to believe that
> you received this message in error, please immediately notify sender
> by e-mail, and destroy the original message. Thank You.
--
Thanks,
 Daniel Dekany


------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
FreeMarker-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-user

This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.

------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
FreeMarker-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-user
Reply | Threaded
Open this post in threaded view
|

Re: Introducing freemarker-introspection

revusky
You know, there is a 2.4 codebase which I spent a huge amount of time refactoring in order to make this kind of project technically feasible. THere is all kinds of machinery in there for tree-walking design patterns where you walk the AST and modify things. IN fact, I recall years ago, Attila telling me that some people in the company he was working at (Adeptra?) were already using that to great effect.

I still have no real understanding of why we never moved to that codebase. I was sure it was a very big mistake, and this just reinforces that belief, but, at the time, I was losing energy on the whole thing and didn't push on the whole thing.

By the way, I guess the 2.4 is still in the sourceforge SVN repository. Or is it?


On Mon, Jun 16, 2014 at 2:51 PM, Wong, Christopher <[hidden email]> wrote:
I agree that the current reflection-based implementation is less maintainable than using the package-access methods you have available starting in 2.3.20. The current approach is only temporary, to allow support for an older version of Freemarker. I will replace it with your recommended approach. This list announcement is to solicit feedback on the public interfaces for freemarker-introspection, which should be independent of the underlying implementation.

Chris

-----Original Message-----
From: Daniel Dekany [mailto:[hidden email]]
Sent: Saturday, June 14, 2014 7:10 AM
To: FreeMarker-user
Subject: Re: [FreeMarker-user] Introducing freemarker-introspection

Do you plan to work on this against 2.3.21, or if something is missing from the freemarker.core-package-visible API, then against the trunk?
If I refactor something inside freemarker.core later, it's much easier to keep freemarker-introspection in sync with it if it relies on freemkarker.core-package-access rather than on reflection.


Friday, June 13, 2014, 3:03:49 PM, Wong, Christopher wrote:

> I would like to introduce a new project, "freemarker-introspection".
> Meant to supplement the core Freemarker project, the purpose of this
> project is to provide a programmatic way to inspect and modify a
> Freemarker template. While Freemarker has an object representation of
> a parsed template, this representation is generally not externally
> accessible. The primary feature of freemarker-introspection is to
> expose the parsed Freemarker template as a DOM-like AST object that
> you can navigate in Java:
>
>         Element root = TemplateIntrospector.getRootNode(template);
>         for (Element child : root.getChildren()) {
>             for (Expr param : child.getParams()) {
>                 ...
>             }
>         }
>
> You can also use a Visitor pattern to walk the template nodes. I
> implemented 2 classes that serve as consumers of this output.
>
> VariableFinder scans the template and returns information on variables
> that the template will reference from the model:
>
>         Element root = TemplateIntrospector.getRootNode(template);
>         List<VariableInfo> vars = new
> VariableFinder(root).seek().getVariableInfo();
>
> TemplateEditor lets you replace specific nodes (elements and
> expressions) with your own content:
>
>         Element root = TemplateIntrospector.getRootNode(template);
>         Element elem = ...; // find element node in tree to replace
>         Expr expr = ...; // find expression node in tree to replace
>         String newTemplateText = new TemplateEditor(oldTemplateText)
>                 .replace(elem, "<#include 'something_else.ftl'>")
>                 .replace(expr, "123")
>                 .apply()
>                 .getModifiedTemplate();
>
> You can find freemarker-introspection at:
>
> https://github.com/cwong15/freemarker-introspection
>
> You will need Gradle to build it. It currently works with Freemarker
> versions 2.3.19 and 2.3.20. This project is still in its very early
> stages, so your feedback will be very much welcome.
>
> Chris
>
> This e-mail and files transmitted with it are confidential, and are
> intended solely for the use of the individual or entity to whom this
> e-mail is addressed. If you are not the intended recipient, or the
> employee or agent responsible to deliver it to the intended recipient,
> you are hereby notified that any dissemination, distribution or
> copying of this communication is strictly prohibited. If you are not
> one of the named recipient(s) or otherwise have reason to believe that
> you received this message in error, please immediately notify sender
> by e-mail, and destroy the original message. Thank You.
--
Thanks,
 Daniel Dekany


------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
FreeMarker-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-user

This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.

------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
FreeMarker-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-user


------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
FreeMarker-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-user
Reply | Threaded
Open this post in threaded view
|

Re: Introducing freemarker-introspection

Daniel Dekany
Tuesday, June 17, 2014, 2:46:34 PM, Jonathan Revusky wrote:

> You know, there is a 2.4 codebase which I spent a huge amount of
> time refactoring in order to make this kind of project technically
> feasible. THere is all kinds of machinery in there for tree-walking
> design patterns where you walk the AST and modify things. IN fact, I
> recall years ago, Attila telling me that some people in the company
> he was working at (Adeptra?) were already using that to great effect.
>
> I still have no real understanding of why we never moved to that
> codebase. I was sure it was a very big mistake,

The "mistake" was, simply, not investing a lot of time (including busy
work...) into FM development in the recent several years. Whether it
goes into that cancelled 2.4 (later 3.0) branch or it goes into yet
another branch is a question of details, compared to that. But I can
only call this a mistake in quotation marks because nobody pays you or
Attila or me to do it. And, 2.3.x had to be maintained, or else you
can just delete the whole project. The DLTK fiasco didn't helped 3.0
either. Anyway, I'm still trying to push this thing into something
that can be called 2.4, even if it meant to be different that the
cancelled 2.4/3.0 was.

But let's not discus this here, if there's anything to discuss.

> and this just reinforces that belief, but, at the time, I was losing
> energy on the whole thing and didn't push on the whole thing. By the
> way, I guess the 2.4 is still in the sourceforge SVN repository. Or
> is it?

It has to be.

> On Mon, Jun 16, 2014 at 2:51 PM, Wong, Christopher <[hidden email]> wrote:
> I agree that the current reflection-based implementation is less
> maintainable than using the package-access methods you have
> available starting in 2.3.20. The current approach is only
> temporary, to allow support for an older version of Freemarker. I
> will replace it with your recommended approach. This list
> announcement is to solicit feedback on the public interfaces for
> freemarker-introspection, which should be independent of the underlying implementation.
>
> Chris
>
> -----Original Message-----
> From: Daniel Dekany [mailto:[hidden email]]
> Sent: Saturday, June 14, 2014 7:10 AM
> To: FreeMarker-user
> Subject: Re: [FreeMarker-user] Introducing freemarker-introspection
>
> Do you plan to work on this against 2.3.21, or if something is
> missing from the freemarker.core-package-visible API, then against the trunk?
> If I refactor something inside freemarker.core later, it's much
> easier to keep freemarker-introspection in sync with it if it relies
> on freemkarker.core-package-access rather than on reflection.
>
>
> Friday, June 13, 2014, 3:03:49 PM, Wong, Christopher wrote:
>
>> I would like to introduce a new project, "freemarker-introspection".
>> Meant to supplement the core Freemarker project, the purpose of this
>> project is to provide a programmatic way to inspect and modify a
>> Freemarker template. While Freemarker has an object representation of
>> a parsed template, this representation is generally not externally
>> accessible. The primary feature of freemarker-introspection is to
>> expose the parsed Freemarker template as a DOM-like AST object that
>> you can navigate in Java:
>>
>>         Element root = TemplateIntrospector.getRootNode(template);
>>         for (Element child : root.getChildren()) {
>>             for (Expr param : child.getParams()) {
>>                 ...
>>             }
>>         }
>>
>> You can also use a Visitor pattern to walk the template nodes. I
>> implemented 2 classes that serve as consumers of this output.
>>
>> VariableFinder scans the template and returns information on variables
>> that the template will reference from the model:
>>
>>         Element root = TemplateIntrospector.getRootNode(template);
>>         List<VariableInfo> vars = new
>> VariableFinder(root).seek().getVariableInfo();
>>
>> TemplateEditor lets you replace specific nodes (elements and
>> expressions) with your own content:
>>
>>         Element root = TemplateIntrospector.getRootNode(template);
>>         Element elem = ...; // find element node in tree to replace
>>         Expr expr = ...; // find expression node in tree to replace
>>         String newTemplateText = new TemplateEditor(oldTemplateText)
>>                 .replace(elem, "<#include 'something_else.ftl'>")
>>                 .replace(expr, "123")
>>                 .apply()
>>                 .getModifiedTemplate();
>>
>> You can find freemarker-introspection at:
>>
>> https://github.com/cwong15/freemarker-introspection
>>
>> You will need Gradle to build it. It currently works with Freemarker
>> versions 2.3.19 and 2.3.20. This project is still in its very early
>> stages, so your feedback will be very much welcome.
>>
>> Chris
>>
>> This e-mail and files transmitted with it are confidential, and are
>> intended solely for the use of the individual or entity to whom this
>> e-mail is addressed. If you are not the intended recipient, or the
>> employee or agent responsible to deliver it to the intended recipient,
>> you are hereby notified that any dissemination, distribution or
>> copying of this communication is strictly prohibited. If you are not
>> one of the named recipient(s) or otherwise have reason to believe that
>> you received this message in error, please immediately notify sender
>> by e-mail, and destroy the original message. Thank You.
> --
> Thanks,
>  Daniel Dekany
>
>
> ------------------------------------------------------------------------------
> HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
> Find What Matters Most in Your Big Data with HPCC Systems
> Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
> Leverages Graph Analysis for Fast Processing & Easy Data Exploration
> http://p.sf.net/sfu/hpccsystems
> _______________________________________________
> FreeMarker-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/freemarker-user
>
> This e-mail and files transmitted with it are confidential, and are
> intended solely for the use of the individual or entity to whom this
> e-mail is addressed. If you are not the intended recipient, or the
> employee or agent responsible to deliver it to the intended
> recipient, you are hereby notified that any dissemination,
> distribution or copying of this communication is strictly
> prohibited. If you are not one of the named recipient(s) or
> otherwise have reason to believe that you received this message in
> error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
>
> ------------------------------------------------------------------------------
> HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
> Find What Matters Most in Your Big Data with HPCC Systems
> Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
> Leverages Graph Analysis for Fast Processing & Easy Data Exploration
> http://p.sf.net/sfu/hpccsystems
> _______________________________________________
> FreeMarker-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/freemarker-user
>

--
Thanks,
 Daniel Dekany


------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
FreeMarker-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-user