Why Stores and renditions return InputStream and not resource

[paulcwarren]

Why Stores and renditions return InputStream and not resource

[paulcwarren]

Hi Ladislav,

Thanks for using Spring Content. We also have a port to Spring Boot 2 and will probably push that out in the release. Just trying to decide whether the next release should be 0.0.11 as originally planned. Or 0.1.0. My guess is, like the rest of Spring, we will have customer who stick on with Spring 1.x at least for a while. But that is an internal concern.

We renamed "spring-content-" to "strg-spring-" so maybe in future it will not clash with spring it self.

Spring team are aware of Spring Content. The desire on our part is that Spring Content becomes a part of Spring eventually. I can't speak for the Spring team so I can't say if that will happen for sure but it is what we the Spring Content team are actively working towards. Spring team needs to see an active project with lots of github stars and forks so we are both doing our bit to make that a reality and I thank you for that :-)

"Is there any reason why Stores return InputStream and not Resource?"

ContentStore was our initial effort. Since then we have actually introduced an experimental Store API that comprises of a Store and an AssociativeStore. Store is essentially just a generic Spring ResourceLoader that does exactly as you say, return Spring Resources. AssociativeStore allow you to associate Resources and Entities. I would recommend that you go look at these interfaces. They are almost GA-able. All stores now implement both. AssociativeStore needs a little tweak. But mostly Spring Content REST needs updating to use these, rather than ContentStore.

I will add caching aspect to the renditions

Awesome. Our guess was that we would need the ability to store renditions - not just create them on-demand - so they can be fetched more quickly. We just didn't have anyone asking for them at the time so we did MVP only to get the feature out of the door. More than happy to accept PRs here.

Next one is: <video...> html5 tag

Sounds like a bug fix to me. Again, feel free to PR that.

If you want to PR changes I presume that you need our internal Spring Boot 2 branch made available so that you can move your changes on that? Please let me know.

[LadislavSopko]

Hi Warren, Thx for response ;) Ill check what happen when I do PR due to renamed folders. Ill see, maybe Ill turn back to original structure. I have to check Stores. I have to discus with my friend. And see also our project needs, we changed also RenditionProvider interface, due to our particular needs on transformation redirecting. Soooo many things to check ;) and sooo few time :) .... Ill try to do thing as soon as possible. regards Ladislav Inviato da iPad

…

Il giorno 05 apr 2018, alle ore 17:29, Paul Warren ***@***.***> ha scritto: Hi Ladislav, Thanks for using Spring Content. We also have a port to Spring Boot 2 and will probably push that out in the release. Just trying to decide whether the next release should be 0.0.11 as originally planned. Or 0.1.0. My guess is, like the rest of Spring, we will have customer who stick on with Spring 1.x at least for a while. But that is an internal concern. We renamed "spring-content-" to "strg-spring-" so maybe in future it will not clash with spring it self. Spring team are aware of Spring Content. The desire on our part is that Spring Content becomes a part of Spring eventually. I can't speak for the Spring team so I can't say if that will happen for sure but it is what we the Spring Content team are actively working towards. Spring team needs to see an active project with lots of github stars and forks so we are both doing our bit to make that a reality and I thank you for that :-) "Is there any reason why Stores return InputStream and not Resource?" ContentStore was our initial effort. Since then we have actually introduced an experimental Store API that comprises of a Store and an AssociativeStore. Store is essentially just a generic Spring ResourceLoader that does exactly as you say, return Spring Resources. AssociativeStore allow you to associate Resources and Entities. I would recommend that you go look at these interfaces. They are almost GA-able. All stores now implement both. AssociativeStore needs a little tweak. But mostly Spring Content REST needs updating to use these, rather than ContentStore. I will add caching aspect to the renditions Awesome. Our guess was that we would need the ability to store renditions - not just create them on-demand - so they can be fetched more quickly. We just didn't have anyone asking for them at the time so we did MVP only to get the feature out of the door. More than happy to accept PRs here. Next one is: <video...> html5 tag Sounds like a bug fix to me. Again, feel free to PR that. If you want to PR changes I presume that you need our internal Spring Boot 2 branch made available so that you can move your changes on that? Please let me know. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[LadislavSopko]

Hi Warren,

I saw Store interface. Ok it should work, but why there is getResource with just string instead of Entity it self, so we can hide entity search and ID extraction directly in store instead do it on controller? Will you work on it in the future?
Then mongo store does not implement it at all. Is it just work in progress or you forget to add it there. (other stores implements it). I'm trying to understand how to communicate faster with you, cause I'm in decision processes and we have to decide which way to take in our development. If I'll work with spring-content, I need to understand how much stops or eventual refactors I'll have to do in the future.
I have to make some sort of time table to my customer.

Thank You
Ladislav

[paulcwarren]

Hi Ladislov,
Store and AssociativeStore are currently experimental API. That said we are looking to GA it soon as it seems to work ok. So these will be the next set of stories we will pick up.

On Store we actually have Store.getResource(SID location); i.e. it is generic, not just String and depends on the Store's declared ID type. If that is String then the method will be getResource(String location) (and incidentally it would implement Spring's resource loader interface). However, if the Store was typed to MyCustomId for example then it would be getResource(MyCustomId location). We do have a story on the backlog to add a AssociativeStore.getResource(E entity) along with AssociativeStore.associate(Entity e, Resource r) and AssociativeStore.unassociate(E entity). We just finished implementing Store and AssociativeStore on the Spring Content JPA module. As you say Mongo's Store implementation still needs doing. As do the tweaks to Spring Content REST to use Store and AssociativeStore instead of ContentStore.

Essentially, we think of Stores as generic Spring ResourceLoader's that can hand out fully-functional Spring Resource's that can be read, written to, deleted and that have identity.

Hope that clarifies the situation. It would be very helpful to understand your timelines, the store(s) you are planning to use as within this track of work. We would be more than happy to prioritize our backlog to fit your requirements.

Perhaps a conference call would be appropriate? You can find my email on my github profile. Shoot me an email and we can set up. Probably quicker than using this github issue.

HTH
_Paul

[paulcwarren]

In case it is useful I have pushed our spring-boot2 branch to wip/spring-boot2.

We will probably release this work as 0.1.0.

[LadislavSopko]

Thank you,
Ill try to move my code to other branch, then Ill continue.

Ladislav

[paulcwarren]

Hi Ladislav, thanks for this. I will try and look at this soonest. I am at a conference this week and away from my laptop for the most part so I just want to provide forewarning in case I don't. At first glance the code looks fine. Only thing I spotted so far was unnassociate. It needs to check whether the @contentid is not also the Spring Data @id before nulling out the value otherwise we null out their Id as well which isn't good obviously. But I can add this code. In general it looks good.

I am not sure what it takes to add codacy? I'll look into it.

[LadislavSopko]

Hello Warren,

Can You check my last PR?. It is there few days now, and I'm blocked on my devel.
It pass all tests except Codacy which pop up again.

Then I'd like to change RenditionProvider interface similar to this:

public interface RenditionProvider {

    public String consumes();
    public Boolean consumes(String fromMimeType);
    public String[] produces();
	
    public RenditionCapability isCapable(String fromMimeType, String toMimeType);
	
    @Deprecated
    public InputStream convert(InputStream fromInputSource, String toMimeType);
    public Resource convert(Resource fromResource, String toMimeType);
} 

RenditionCapability is an enum for determine how much I'm capable to do some rendition
Cause there can exists more different renderers but each of them can have different quality.

Can we do some discussion about it?

Regards
Ladislav.

[paulcwarren]

Accepted that last PR. Sorry about that. I missed it for some reason. I think we can do some refactoring of the ContentStore methods to use the AssociativeStore methods but we can get to that anytime. Let's look at renditions first.

On the one hand I like this suggestion. By taking a resource and handing another back this will immediately fit better into the Spring Content REST layer. And in fact probably make it possible to serve byte ranges request of renditions as well as just the actual content. Pretty neat. Not sure how much utility it will have. In reality I have only seen byte range requests used by movie players and adobe reader and custom implementations. But perhaps a word doc gets rendered as a PDF and adobe would use byte range requests to fetch pages at a time. It is nice that it would work for renditions as well as content. Usually the sign of a good abstraction.

On the other hand this does make writing a rendition provider a little harder. I would be curious to see what a sample provider would look like for this? How much extra code is it to write in order to return a resource?

But in general I like the concept of using Resource

[LadislavSopko]

Hello Warren.

I’m on short holidays now, so I’ll write some notes from tablet now.:) We manage consistent number of document records a day, with our solution, and we do almost for each digital attachment text and pdf rendition. So for us ability to render is clue of content storage. And it must be fast as much as possible. So we did POC with apply spring-cache aspect on convert method. And it function with Resource, even with stream in it. (I have example of Cache implementation that handle also streams.) I had made some sort of Super Resource Wrapper which implement Resource and Inputstream interfaces. Cause we need Content entity during some rendition process, so I need move around a bit of more data then just plain content. Then I’m working on particular renderer which will forward rendition request to another copy of content-store in net. so we can handle also many different renderers with horizontal scalability. (the same composition of content-store application just with different renditions buildet in:) All managet trough spring microservice support, this I need study better, but architecture is already on the paper :)
Ill be back in few days.

regards

ladislav

[paulcwarren]

Hey @LadislavSopko. Just checking back in to let you know that 0.1.0 now supports Spring Boot 2 and moves the Store and AssociativeStore APIs in the direction we have discussed.

I have also updated the README to include details for setting up eclipse and intellij IDEs with the relevant code formatting to make PRs a little easier to make and review.

[LadislavSopko]

Hello, I allready rebased my branches to 0.1.0 master :) I have some time stress in other project :( I'll be back with my changes as soon as possible. 2018-06-05 17:24 GMT+02:00 Paul Warren <[email protected]>:

…

Hey @LadislavSopko <https://github.com/LadislavSopko>. Just checking back in to let you know that 0.1.0 now supports Spring Boot 2 and moves the Store and AssociativeStore APIs in the direction we have discussed. I have also updated the README to include details for setting up eclipse and intellij IDEs with the relevant code formatting to make PRs a little easier to make and review. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#21 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALb9BlPSC1ZhQe1v8AYcFaSND9O3XQQ6ks5t5qK9gaJpZM4TGc5e> .

[paulcwarren]

Awesome. Likewise, if I get to it before you I will take a shot at re-vamping the renditions API as we have been discussing here. Talk soon.

[LadislavSopko]

My friend is working on branch with code what I start to use, but it is not at level to be merged in production jet. We had milestone til 31/07. It should be also with functional cache, too. Anyway we continue on code what I propose to You for discussion. Maby We will be able to commit some updates in about 2-3 weeks. Ill do speak with my friend in this days. Ladislav Inviato da Posta per Windows 10 Da: Paul Warren Inviato: mercoledì 6 giugno 2018 06:09 A: paulcwarren/spring-content Cc: Ladislav Sopko; Mention Oggetto: Re: [paulcwarren/spring-content] Why Stores and renditions returnInputStream and not resource (#21) Awesome. Likewise, if I get to it before you I will take a shot at re-vamping the renditions API as we have been discussing here. Talk soon. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[paulcwarren]

Hi @LadislavSopko,

I wanted to check back in with you on this issue and PR. For the most part, I think I have slowly but surely moved spring-content in the direction we agreed upon. Store and AssociativeStore are both now Resource-based and supported by all storage modules. Plus all of the store abstractions are now fairly cleanly supported by the latest spring content REST module.

I think the last thing would be to include some variation of your RenditionProvider. Would you this still be useful to you?

Thanks

[paulcwarren]

Closing due to inactivity. Please feel free to re-open. In the meantime I have added a stories to our backlog to:

• refactor the rendition provider interface to use Resource
• provide the ability to access to the entity in rendition providers