spring-cloud/spring-cloud-function

spring-cloud-function's Issues

Special handling of functions that produce Optional output

A function that maps Foo to Optional<Bar> is like a shorthand for a Flux.filter() on the output of the flux function.

Type detection broken in functions from @ComponentScan

It all works fine with @Bean definitions because we easily have access to the ParametrizedType from the factory method. It's not so easy with @Components (or POFs scanned with @FunctionScan) because we only have the type of the bean which is a Class<?>.

Functional web app doesn't run if it is a Spring Cloud config client

Apparently?

Runtime function compiler doesn't work with Spring Boot fat jar

In the function-sample project , I hope to package a fat jar , so that the sample project can be deployed in Pivotal Cloud Foundry, but I fails. (It works if I run the application in Eclipse)
What I have done is as following:
I try to remove the "spring-boot-thin-layout" dependency from the "spring-boot-maven-plugin" plugin, and then run maven cmd " mvn clean package", and finally I got a fat jar.
But when I run the fat jar (java -jar -Dserver.port=9010 xxxx.jar) , it throws exceptions.

exceptions:

2017-09-26 20:02:46.862  INFO 40852 --- [           main] functions.Application                    : Starting Application v1.0.0.BUILD-SNAPSHOT on DESKTOP-7USSO3N with PID 40852 (G:\GitHub\spring-cloud-function\spring-cloud-function-samples\function-sample-pof\target\function-sample-pof-1.0.0.BUILD-SNAPSHOT-exec.jar started by silencehe09 in G:\GitHub\spring-cloud-function\spring-cloud-function-samples\function-sample-pof\target)
2017-09-26 20:02:46.883  INFO 40852 --- [           main] functions.Application                    : No active profile set, falling back to default profiles: default
2017-09-26 20:02:47.051  INFO 40852 --- [           main] ationConfigEmbeddedWebApplicationContext : Refreshing org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@5ef04b5: startup date [Tue Sep 26 20:02:47 CST 2017]; root of context hierarchy
2017-09-26 20:02:49.359  WARN 40852 --- [           main] ationConfigEmbeddedWebApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'methodValidationPostProcessor' defined in class path resource [org/springframework/boot/autoconfigure/validation/ValidationAutoConfiguration.class]: Unsatisfied dependency expressed through method 'methodValidationPostProcessor' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jsr303Validator' defined in class path resource [org/springframework/boot/autoconfigure/validation/ValidationAutoConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.validation.Validator]: Factory method 'jsr303Validator' threw exception; nested exception is java.lang.NoClassDefFoundError: Could not initialize class org.hibernate.validator.internal.engine.ConfigurationImpl
2017-09-26 20:02:49.369 ERROR 40852 --- [           main] o.s.b.f.s.DefaultListableBeanFactory     : Destroy method on bean with name 'org.springframework.boot.context.properties.ConfigurationPropertiesBindingPostProcessor' threw an exception

java.lang.IllegalStateException: ApplicationEventMulticaster not initialized - call 'refresh' before multicasting events via the context: org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@5ef04b5: startup date [Tue Sep 26 20:02:47 CST 2017]; root of context hierarchy
        at org.springframework.context.support.AbstractApplicationContext.getApplicationEventMulticaster(AbstractApplicationContext.java:404) [spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
        at org.springframework.context.support.ApplicationListenerDetector.postProcessBeforeDestruction(ApplicationListenerDetector.java:97) ~[spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
        at org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:253) ~[spring-beans-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:578) [spring-beans-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:554) [spring-beans-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingleton(DefaultListableBeanFactory.java:961) [spring-beans-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:523) [spring-beans-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingletons(DefaultListableBeanFactory.java:968) [spring-beans-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
        at org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:1033) [spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:555) [spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
        at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:122) [spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
        at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:737) [spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
        at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:370) [spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:314) [spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1162) [spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1151) [spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
        at functions.Application.main(Application.java:28) [classes!/:1.0.0.BUILD-SNAPSHOT]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_131]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_131]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_131]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_131]
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [function-sample-pof-1.0.0.BUILD-SNAPSHOT-exec.jar:1.0.0.BUILD-SNAPSHOT]
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [function-sample-pof-1.0.0.BUILD-SNAPSHOT-exec.jar:1.0.0.BUILD-SNAPSHOT]
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [function-sample-pof-1.0.0.BUILD-SNAPSHOT-exec.jar:1.0.0.BUILD-SNAPSHOT]
        at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51) [function-sample-pof-1.0.0.BUILD-SNAPSHOT-exec.jar:1.0.0.BUILD-SNAPSHOT]

2017-09-26 20:02:49.379 ERROR 40852 --- [           main] o.s.b.f.s.DefaultListableBeanFactory     : Destroy method on bean with name 'org.springframework.boot.autoconfigure.internalCachingMetadataReaderFactory' threw an exception

java.lang.IllegalStateException: ApplicationEventMulticaster not initialized - call 'refresh' before multicasting events via the context: org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@5ef04b5: startup date [Tue Sep 26 20:02:47 CST 2017]; root of context hierarchy
        at org.springframework.context.support.AbstractApplicationContext.getApplicationEventMulticaster(AbstractApplicationContext.java:404) [spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
        at org.springframework.context.support.ApplicationListenerDetector.postProcessBeforeDestruction(ApplicationListenerDetector.java:97) ~[spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
        at org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:253) ~[spring-beans-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:578) [spring-beans-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:554) [spring-beans-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingleton(DefaultListableBeanFactory.java:961) [spring-beans-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:523) [spring-beans-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingletons(DefaultListableBeanFactory.java:968) [spring-beans-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
        at org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:1033) [spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:555) [spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
        at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:122) [spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
        at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:737) [spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]

"Wiretap" or "export" feature

Basic idea: user declares (somehow) an interest in "exporting" items of a given POJO type Foo (say), and framework generates 2 beans: one is a Consumer<Foo> and the other is a Supplier<Flux<Foo>>. The framework already automatically adds the Supplier to the FunctionCatalog, so it can be used to send Foos to an external stream or HTTP client. The Consumer is implemented in such a way that it pushes each Foo into the Flux stream created by the Supplier.

I implemented it here as @EnableWiretap but the name could be changed: https://github.com/dsyer/petflix/blob/master/videos/src/main/java/com/example/VideoApplication.java#L15.

HTTP request for missing function should return a 404

Currently getting 500 (NPE)

Add handling for Message as well as POJO

A Function<Message<Foo>,Message<Bar>> should behave like a Function<Foo,Bar> but with the headers.

Rabbit should be optional dependency

Spring Cloud Stream Rabbit depends on the Rabbit binder. Probably just for demo purposes. It should be optional (or absent).

Add Gradle configuration to samples

The challenge at the minute is this: spring-gradle-plugins/dependency-management-plugin#184.

Cannot build project on windows: JavaFileManager issue

hi I need some help with running this project on windows
here is my environment: jdk 1.8.0_111
always get a error like this:

return (Function<String,String> & java.io.Serializable) s -> s.toUpperCase();

package org.springframework.cloud.function.compiler;
import java.util.*;
import java.util.function.*;
import reactor.core.publisher.Flux;
public class CompiledUppercaseFunctionFactory implements FunctionFactory {
 public Function<String,String> getResult() {
  return (Function<String,String> & java.io.Serializable) s -> s.toUpperCase();
 }
}

2017-07-14 15:09:23.611  INFO 58748 --- [           main] o.s.c.f.c.java.RuntimeJavaCompiler       : compiler com.sun.tools.javac.api.JavacTool@d8305c2
2017-07-14 15:09:23.611  INFO 58748 --- [           main] o.s.c.f.c.java.RuntimeJavaCompiler       : Compiling source for class org.springframework.cloud.function.compiler.CompiledUppercaseFunctionFactory using compiler com.sun.tools.javac.api.JavacTool
2017-07-14 15:09:24.088  WARN 58748 --- [           main] ationConfigEmbeddedWebApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'compiledUppercase' defined in com.example.SampleApplication: Invocation of init method failed; nested exception is org.springframework.cloud.function.compiler.java.CompilationFailedException: 

Fatal: Unable to find package java.lang in classpath or bootclasspath 

the JavaCompiler is com.sun.tools.javac.api.JavacTool
so I tried it with more simple code using default FileManager

import java.io.File;
import java.io.FileWriter;
import java.lang.reflect.Method;
import java.net.URL;
import java.net.URLClassLoader;
import javax.tools.JavaCompiler;
import javax.tools.JavaCompiler.CompilationTask;
import javax.tools.JavaFileObject;
import javax.tools.StandardJavaFileManager;
import javax.tools.ToolProvider;


public class CompilerMain {
    public static void main(String[] args) throws Exception {
        testCompiler();
    }
    public static void testCompiler() throws Exception {
        String currentDir = System.getProperty("user.dir");
        String src =
                "import java.util.*;\n" +
                "import java.util.function.*;\n" +
                "public class CompiledUppercaseFunctionFactory {\n" +
                " public Function<String,String> getResult() {\n" +
                "  return (Function<String,String> & java.io.Serializable) s -> s.toUpperCase();\n" +
                " }\n" +
                "}";
        String filename = currentDir + "/src/CompiledUppercaseFunctionFactory.java";
        File file = new File(filename);
        FileWriter fw = new FileWriter(file);
        fw.write(src);
        fw.flush();
        fw.close();
        JavaCompiler jc = ToolProvider.getSystemJavaCompiler();
        StandardJavaFileManager fileManager = jc.getStandardFileManager(null, null, null);
        Iterable<? extends JavaFileObject> fileObjects = fileManager.getJavaFileObjects(filename);
        CompilationTask cTask = jc.getTask(null, fileManager, null, null, null, fileObjects);
        cTask.call();
        fileManager.close();
        URL[] urls = new URL[] { new URL("file:/" + currentDir + "/src/") };
        URLClassLoader cLoader = new URLClassLoader(urls);
        Class<?> c = cLoader.loadClass("CompiledUppercaseFunctionFactory");
        cLoader.close();
        Object obj = c.newInstance();
        Method method = c.getMethod("getResult");
        method.invoke(obj);
    }
} 

And this works ... so anything missing here can u give me some advice

Support Servlet API and Spring MVC applications/microservices

I'd like to propose support for Servlet API and Spring MVC. Specifically, in order to do this there are incremental enhancements needed, each one builds upon previous:

1. A standard way and best practices to deploy arbitrary event processing (SCF already does this)

   • For example, since startup time is more important in cloud functions than regular Spring Boot, there could be a guide that mention best practices on how to reduce startup time
     • Tooling (e.g. spring-cloud-function-maven-plugin) to trade runtime time vs. build time. For example, classpath scanning can be moved to build time, so at runtime there's much less work to do. This may require support in other Spring components.
     • A nice side effect is these optimizations can be used outside of SCF, in regular Spring projects
   • Guide on how to optimize the "warm" container state, i.e. reusing Hibernate connections/connection pool
   • Guide for restrictions and gotchas that need to be followed (i.e. 50 MB limit, threading, shared stuff, filesystem, etc.)
   • Guide to optimize/preload classpath scanning

2. SCF to serve/wrap Servlet applications. For example for AWS, there is aws-serverless-java-container-spring made by @sapessi of AWSLabs

   • https://github.com/spring-cloud/spring-cloud-function/tree/master/spring-cloud-function-web is for exposing functions as REST services, what we'd like to do is expose Servlet/REST "services" as functions conforming to API Gateway's proxy request & response
   • Maybe it can be named spring-cloud-function-servlet, as what it does is simulating a Servlet 3.1 API "container"

3. Extend support for serving Spring MVC controllers.

   • This is already possible as proof-of-concept (thanks @joeyvmason!): Developing Serverless Applications with Spring MVC and AWS Lambda and serverless-spring-boilerplate, however very far from optimized i.e. 20+ second startup time (which is normal for some Spring Boot applications but is bad for lambda functions, the default timeout is 6 seconds although it can be increased to 60 seconds).

   • This potentially allows Spring to run many Java web frameworks not on top of servlet container anymore, but on top of serverless. See Zappa for successful example in Python world, that can run microservice frameworks like Flask, Bottle, Pyramid, but also full-stack Django apps, mostly without any code change. Please see Zappa's presentation slides to get a feel why this is important and such a game changer.

     • Of course the "other" web framework will need to ensure behave well in a serverless environment, but this will be much easier if Spring Boot components already laid the foundation to enable this support smoothly.

   • I also did a proof-of-concept that this can work: https://github.com/ceefour/wicket8serverless

     It's using Kotlin + Spring Boot + Web 1.5.6 and @martin-g's Wicket 8.0.0-M7 + Wicket Bootstrap 2.0.0-M6. Not slimmed versions of these frameworks, but the entire Spring Boot loader, Spring MVC dispatcherServlet and Wicket+Wicket Bootstrap installed as servlet Filter. It's definitely buggy and lots of gotchas, but initial work is in https://github.com/ceefour/aws-serverless-java-container/tree/spring-boot and hopefully @sapessi will accept my pull request. :) The package is 24 MB but it's possible to reduce it more by configuring shade to minimizeJar, however my initial minimizeJar efforts resulted in CNF exceptions so I disabled minimizeJar.

     Deployed result is here: https://im9ntgimlc.execute-api.us-east-1.amazonaws.com/prod/

     Of course as expected, the startup time leaves so much to be desired: (there's not even Hibernate yet)

     

     27 whole seconds just for startup. However, after it's warm, subsequent requests only take 300 ms which is still slow in serverless world, but can be optimized.

4. Tooling to help local development. For example @Serverless annotation, if given, will enable checks during local development to detect restricted features in serverless environment and give WARN or even ERROR log or throw exception.

5. Helpful extras to help Spring app developers modify Spring Boot/MVC apps to SCF's programming model. To illustrate the kind of extras possible, allow me to quote Zappa's feature list:

   • Automatic support for all AWS event sources, allowing you to build robust "hybrid" applications
   • Automatic support for hundreds of pre-compiled and pre-optimized C-extentions packages (SciPy, etc.)
   • Support for applications which use multiple cookies simultaneously (i.e., anything with a login)
   • Free, auto-renewing SSL certificates from Let's Encrypt
   • CI integration
   • Automatic support for "global" (multi-region) applications
   • HTTP logging in the Common Log Format
   • Automatic "keep-warm"
   • Intelligent application and variable caching
   • Zero-effort CORS
   • Automatic and intelligent project initializion (zappa init)
   • Multiple environment variable sources (local, S3)
   • Multiple "secure endpoint" authentication methods (API Key, IAM, custom authorizers)
   • Content-Type aliases
   • Highly customizable configurations
   • Managed IAM credientials with the option to supply custom credentials
   • Package optimization
   • VPC-awareness
   • Ability to deploy HTTP-less, event-driven applications
   • Custom error-reporting (ex, to Sentry or Raygun)
   • Remote command invocation (include raw Python)

In the past, Java EE (and still to some extent Spring Boot) has been regarded to some people as hard to develop, hard to deploy, hard to maintain (DevOps, scaling, etc.), and expensive to operate (requiring servers etc.) Spring Boot is making it easy to develop. And Kotlin and upcoming Java 9 also makes the entire Spring Boot experience pleasant.

If SCF can support existing Spring Boot apps with minimal change, that means app developers gain benefits such as easy deployment, automatic scalability, zero maintenance, stability, less vendor lock-in (i.e. Spring Boot app can be moved from AWS Lambda to OpenWhisk and vice versa; and can still be deployed on PaaS like Pivotal Web Services; and regular instances). This will make SCF much more attractive and increase adoption.

I sincerely hope that this enhancement proposal can be considered. You guys rock. Thank you! :)

This is originally from spring-projects/spring-boot#10136, I hope it's okay to repost it here (with some adjustments).

Sample Test Event for AWS Lambda

I haven't been able to figure out how to get the AWS lambda sample to work with a test event.
Could you provide a valid test event that can be sent to verify the sample?

I've deployed the latest from master and configured org.springframework.cloud.function.adapter.aws.SpringBootStreamHandler as the handler.

When I send a test event

{
  "input": "test"
}

I get a NPE error response

{
  "errorMessage": "java.lang.NullPointerException",
  "errorType": "java.lang.NullPointerException",
  "stackTrace": [
    "example.Foo.uppercase(Config.java:66)",
    "example.Config.lambda$function$0(Config.java:40)",
    "org.springframework.cloud.function.core.FluxFunction.lambda$apply$0(FluxFunction.java:42)",
    "reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:107)",
    "reactor.core.publisher.FluxJust$WeakScalarSubscription.request(FluxJust.java:90)",
    "reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:159)",
    "reactor.core.publisher.BlockingIterable$SubscriberIterator.onSubscribe(BlockingIterable.java:214)",
    "reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:90)",
    "reactor.core.publisher.FluxJust.subscribe(FluxJust.java:67)",
    "reactor.core.publisher.FluxMapFuseable.subscribe(FluxMapFuseable.java:63)",
    "reactor.core.publisher.BlockingIterable.iterator(BlockingIterable.java:80)",
    "org.springframework.cloud.function.adapter.aws.SpringBootStreamHandler.result(SpringBootStreamHandler.java:62)",
    "org.springframework.cloud.function.adapter.aws.SpringBootStreamHandler.handleRequest(SpringBootStreamHandler.java:57)"
  ]
}

Cannot run web apps with Spring 5

There's a binary incompatibility. Maybe need to look at fixing it in Spring: https://jira.spring.io/browse/SPR-15410

Caused by: java.lang.NoSuchMethodError: org.springframework.web.servlet.mvc.method.annotation.ResponseBodyEmitterReturnValueHandler.<init>(Ljava/util/List;)V
    at org.springframework.cloud.function.web.flux.FluxReturnValueHandler.<init>(FluxReturnValueHandler.java:53) ~[classes/:na]
    at org.springframework.cloud.function.web.flux.ReactorAutoConfiguration.fluxReturnValueHandler(ReactorAutoConfiguration.java:56) ~[classes/:na]
    at org.springframework.cloud.function.web.flux.ReactorAutoConfiguration$$EnhancerBySpringCGLIB$$e9d8b8ef.CGLIB$fluxReturnValueHandler$1(<generated>) ~[classes/:na]
    at org.springframework.cloud.function.web.flux.ReactorAutoConfiguration$$EnhancerBySpringCGLIB$$e9d8b8ef$$FastClassBySpringCGLIB$$64d3f2be.invoke(<generated>) ~[classes/:na]
    at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228) ~[spring-core-5.0.0.BUILD-20170403.133901-52.jar:5.0.0.BUILD-SNAPSHOT]
    at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:358) ~[spring-context-5.0.0.BUILD-20170403.133901-52.jar:5.0.0.BUILD-SNAPSHOT]
    at org.springframework.cloud.function.web.flux.ReactorAutoConfiguration$$EnhancerBySpringCGLIB$$e9d8b8ef.fluxReturnValueHandler(<generated>) ~[classes/:na]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_112]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_112]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_112]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_112]
    at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:162) ~[spring-beans-5.0.0.BUILD-20170403.133901-52.jar:5.0.0.BUILD-SNAPSHOT]
    ... 47 common frames omitted

How to configure the Kafka input/output topic names?

How can we customize the destination Kafka topic name to which stream functions write?

Given this supplier:

@SpringBootApplication
public class SupplierApplication {

	private Logger logger = LoggerFactory.getLogger(getClass());
	
	public static void main(String[] args) {
		SpringApplication.run(SupplierApplication.class, args);
	}
	
	@Bean
	public Supplier<Flux<String>> getValue(){
		return () -> {
			return Flux.interval(Duration.ofSeconds(1)).map( i -> String.valueOf(Math.random()))
				.doOnNext( val -> logger.info("*** Producing {} ***", val));
		};
	}
}

application.yml

spring:
  application:
    name: supplier

server:
  port: 8282

I see that this stream writes to supplier:8282.output:
o.s.integration.channel.DirectChannel : Channel 'supplier:8282.output' has 1 subscriber(s).

How can I customize it?

Idem for a consumer that should read from the previous topic.

Simple proxy features for easy gateway implementation

The Petflix sample shows that a gateway with thin but important processing of payloads and computing URLs is a common task that can be reduced to boilerplate. We would like to make an EZ Proxy feature (Zuul lite) to enable this kind of pattern with a "functional" flavour.

Here's the gateway controller in the Petflix core:

    @GetMapping("/owners/videos/{id}")
    public Map<String, Object> video(@PathVariable Integer id) throws Exception {
        String path = videosUrl.toString() + "/videos/" + id;
        return get(path);
    }

    @PostMapping("/owners/videos/{id}")
    public Map<String, Object> rate(@PathVariable Integer id,
            @RequestBody Map<String, Object> body) throws Exception {
        body = new LinkedHashMap<>(body);
        body.put("id", id);
        String path = videosUrl.toString() + "/ratings";
        return post(body, path);
    }

    private Map<String, Object> post(Object body, String path) throws URISyntaxException {
        return template.exchange(
                RequestEntity.post(new URI(path)).accept(MediaType.APPLICATION_JSON)
                        .contentType(MediaType.APPLICATION_JSON)
                        .body(Arrays.asList(body)),
                new ParameterizedTypeReference<List<Map<String, Object>>>() {
                }).getBody().iterator().next();
    }

    private Map<String, Object> get(String path) throws URISyntaxException {
        return template.exchange(
                RequestEntity.get(new URI(path)).accept(MediaType.APPLICATION_JSON)
                        .build(),
                new ParameterizedTypeReference<List<Map<String, Object>>>() {
                }).getBody().iterator().next();
    }

Kafka Topic stream consumer not getting previous messages

I apologize in advance if this is a newbie question I would've asked on Stackoverflow but I think there isn't dedicated Spring Cloud Function support over there yet.

Scenario in Spring Cloud Stream Apps:
App1 sends messages to a topic - App2 offline
App2 starts and processes all the messages sent to the topic by App1 while App2 was offline.

Scenario in Spring Cloud Function Apps:
App1 (supplier) sends messages to a topic - App2 offline
App2 (consumer) doesn't get the previous messages, only the ones produced while App1 and App2 are up at the same time.

What's the explanation of this different behaviour?

Support for consumer groups in HTTP

What does that even mean? Spring Cloud Stream has "consumer group" as a core domain feature (although there has been some discussion about making it optional). What would that look like for HTTP consumers? See also #36.

Stream invoker swallows exception in consumer

I can see a JSON string coming into the message channel, and if I set a break point I can see a ClassCastException where ProxyConsumer.accept() is called (String cannot be converted to Flux). There's no sign of it in the app though, in logs, and the metrics show an error, but also a send sount greater than the error count (so what does that mean?).

Polish POM files to remove redundant entries

This may serve as placeholder for future polishing efforts

Runnable as new abstraction for function

Is there a plan to add support for some other interfaces to spring cloud function, I see usefull especially interface for job execution - Runnable.
Because some service can do only simple job without giving any feedback about result f.e.: data pump/sync, triggering reports. It is possible to use for this scenarios Consumer abstraction, but without input(it is not necessary), but with runnable it will be better understandable.

Issue running spring-cloud-function-samples

Hi - I am exploring spring-cloud-function to leverage this framework for serverless deployment. I am facing an issue when running function-sample project. Below is the stacktrace. Please advise.

java -jar function-sample-1.0.0.BUILD-SNAPSHOT.jar
Exception in thread "main" java.lang.IllegalStateException: Cannot locate library: /Users/amg871/.m2/repository/org/springframework/boot/experimental/spring-boot-thin-launcher/1.0.7.RELEASE/spring-boot-thin-launcher-1.0.7.RELEASE-exec.jar
	at org.springframework.boot.loader.wrapper.ThinJarWrapper.launch(ThinJarWrapper.java:113)
	at org.springframework.boot.loader.wrapper.ThinJarWrapper.main(ThinJarWrapper.java:90)

Consider adding support for Google Cloud Platform - Cloud Functions

It would be great to have an adapter for GCP Cloud Functions.

FunctionInspector.getInputType(String) is ambiguous

There might be more than one answer:

	@Configuration
	protected static class AmbiguousConfiguration {
		private List<Foo> list = new ArrayList<>();

		@Bean
		public Function<String, Foo> foos() {
			return value -> new Foo(value.toUpperCase());
		}

		@Bean
		@Qualifier("foos")
		public Consumer<Foo> consumer() {
			return value -> list.add(value);
		}
	}

The current implementation of the inspector returns Foo for inspector.getInputType("foos") because it does a lookup by bean name (which is also wrong and might lead to Object being the answer when the qualifier is used).

Same goes for getOutputType(String).

The ContextFunctionPostProcessor doesn't resolve function from the FactoryBean

There is the code in the ContextFunctionPostProcessor.findType():

else if (source instanceof Resource) {
	try {
		Class<?> beanType = ClassUtils.forName(definition.getBeanClassName(),
					null);
		for (Type type : beanType.getGenericInterfaces()) {
			if (type.getTypeName().startsWith("java.util.function")) {
					param = extractType(type, paramType, index);
					break;
			}
		}
		if (param == null) {
			// Last chance
			param = beanType;
		}
	}
	catch (ClassNotFoundException e) {
		throw new IllegalStateException(
			"Cannot instrospect bean: " + definition, e);
	}
}

Where beanType can be resolved to some FactoryBean. For example GatewayProxyFactoryBean from Spring Integration for Function service interface.

This code must include something similar what SF has for the BeanFactory.isTypeMatch() resolution:

if (beanInstance instanceof FactoryBean) {
	if (!BeanFactoryUtils.isFactoryDereference(name)) {
		Class<?> type = getTypeForFactoryBean((FactoryBean<?>) beanInstance);
		return (type != null && typeToMatch.isAssignableFrom(type));
	}
	else {
		return typeToMatch.isInstance(beanInstance);
	}
}

Also param = beanType; looks suspicious as well. In my case we fail into the FactoryBean type.
However we definitely expect here a generic type since we have came here just because this bean is Function. So, an Object.class should be good one as a general fallback.

Update to reactor 3.1

aka Bismuth

Finish or remove composable function support

The FunctionCatalog can be used to lookup composed functions using comma separated function names, but the FunctionInspector doesn't know how to find the input and output types.

Unnecessary dependencies

Is the commons-io dependency used for anything in the core and compiler modules?

I see it in the pom but find no usages of it. If possible the dependencies should be minimised for the result jars because it affects the cold-start times of the functions.

Also are the commons-* dependencies really necessary?

Deployed jar endpoints not all working

In the deployer app the pojo sample has 3 endpoints, but only /words works. The others get the wrong name for the function (so the MVC handler isn't working properly).

Cannot run with AWS adapter: "Source must not be null"

I'm attempting to run a lambda (practically identical to the adapter sample), but having difficulty when it tries to run the SpringApplication.

What I am seeing is as follows:

21:26:56.384 [main] INFO lambdainternal.LambdaRTEntry - No active profile set, falling back to default profiles: default
21:26:56.384 [main] DEBUG org.springframework.boot.SpringApplication - Loading source null
21:26:56.406 [main] ERROR org.springframework.boot.SpringApplication - Application startup failed
java.lang.IllegalArgumentException: Source must not be null
	at org.springframework.util.Assert.notNull(Assert.java:134)
	at org.springframework.boot.BeanDefinitionLoader.load(BeanDefinitionLoader.java:133)
	at org.springframework.boot.BeanDefinitionLoader.load(BeanDefinitionLoader.java:127)
	at org.springframework.boot.SpringApplication.load(SpringApplication.java:635)
	at org.springframework.boot.SpringApplication.prepareContext(SpringApplication.java:355)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:301)
	at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:134)
	at org.springframework.cloud.function.adapter.aws.SpringFunctionInitializer.initialize(SpringFunctionInitializer.java:90)
	at org.springframework.cloud.function.adapter.aws.SpringBootStreamHandler.handleRequest(SpringBootStreamHandler.java:54)
	at lambdainternal.EventHandlerLoader$2.call(EventHandlerLoader.java:888)
	at lambdainternal.AWSLambda.startRuntime(AWSLambda.java:281)
	at lambdainternal.AWSLambda.<clinit>(AWSLambda.java:64)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:348)
	at lambdainternal.LambdaRTEntry.main(LambdaRTEntry.java:94)

I've not dug into the guts of Spring much, but below is my interpretation of what I see happening:

As I look at the SpringFunctionInitializer, it strikes me odd that at line 88 it creates a new SpringApplicationBuilder: new SpringApplicationBuilder(configurationClass);.
This sets the sources with my main class, yet when it performs builder.web(false).run() on line 90, that call appears to set sources to null again (since no sources were passed to run()).
See SpringApplicationBuilder line 134: this.context = build().run(args);. At this point, as far as I can tell by tracing, args is null, and this will overwrite the sources passed to the SpringApplicationBuilder constructor?

I've no doubt I've made some incorrect assumptions along the way...

Use thin plugin for AWS sample

To be more like the Maven build we would need to use the thin plugin to build the non-AWS jar, but it doesn't work at the moment because of some issue with the shadow plugin.

See #105

Issue running spring-cloud-function-deployer

Hi,

I built the whole project with ./mvnw clean install but function-deployer application fails to start.

I ran: mvn spring-boot:run getting the following exception:

org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'bindingService' defined in class path resource [org/springframework/cloud/stream/config/BindingServiceConfiguration.class]: Unsatisfied dependency expressed through method 'bindingService' parameter 1; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'org.springframework.cloud.stream.config.BinderFactoryConfiguration': Unsatisfied dependency expressed through field 'binderTypeRegistry'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'binderTypeRegistry' defined in class path resource [org/springframework/cloud/stream/config/BinderFactoryConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.cloud.stream.binder.BinderTypeRegistry]: Circular reference involving containing bean 'org.springframework.cloud.stream.config.BinderFactoryConfiguration' - consider declaring the factory method as static for independence from its containing instance. Factory method 'binderTypeRegistry' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Cannot create binder factory, no META-INF/spring.binders resources found on the classpath at org.springframework.beans.factory.support.ConstructorResolve

I tried also to run the application starting it from the IDE and running the jar with java -jar spring-cloud-function-deployer-1.0.0.BUILD-SNAPSHOT.jar but nothing seems to work.

Any hint?

Thanks!

Figure out what to do if RootBeanDefinition source is not StandardMethodMetadata

The FunctionCatalog relies on the concrete type of the source of a bean definition. Unfortunately there are 2 concrete types which implement MethodMetadata and sometimes you get the other one (apparently as soon as the @Bean is not in the main @SpringBootApplication?).

Aws API Gateway support (AwsProxyRequest, AwsProxyResponse)?

Everything in this issue relates to AWS Lambda with API Gateway.
According to the documentation, Functions are auto-detected and applied to input by the SpringBootStreamHandler of SpringBootRequestHandler.
I have found that, for my usage, the stream version of the lambda handler is more flexible, with Api gateway integration type set to "proxy".
However, the input stream won't be mappable to a "business" type (Foo.class in your examples) but to an AwsProxyRequest (structure described there , and in turn API gateway expects an AwsProxyResponse in the output stream.

It it expected to create Function<AwsProxyRequest, AwsProxyResponse>, which is broad and maybe too generic, or is there already a way to transparently map this api gateway interface in spring-cloud-functions ?

AWS adapter needs `Start-Class` entry in MANIFEST.MF of shaded jar

Tried a number of combinations. Looks like the spring-boot-thin-layout plugin in the example for AWS does add Start-Class to the manifest. Removing that plugin, I had to add a Start-Class entry in the manifest for the maven-jar-plugin plugin to get it to work. This wasn't obvious from reading the docs.

Feature request: Support Paging like MVC

I am testing the spring-cloud-function with AWS lambda and serverless framework , and every thing is good , while testing some scenario with spring-data-rest i wanted to specify page , size ... which i know i can do that manually but thought it will be great to automate it so the idea was

1- In case we use lambda-proxy then all variables (Path, Query , ... ) will be sent as Map
2- Use this to create Pageable , QueryDsl predict ...
3- Inject them some how in the function .

Do you think this is possible or doable ?

Support for compiled functions that are not just String->String

If you insert this code in AbstractByteCodeLoadingProxy.afterPropertiesSet() you can see (e.g. in the compiler sample) that we know the input and output types of the functions we compiled:

			AtomicReference<Method> reference = new AtomicReference<Method>();
			ReflectionUtils.doWithMethods(factoryClass, m -> {
				if (m.getName().equals("getResult")
						&& m.getReturnType().getName().startsWith("java.util.function")) {
					reference.set(m);
				}
			});
			Method method = reference.get();
			System.err.println(method.getGenericReturnType());

@aclement probably needs to step in and tell us how to do this properly. I was thinking maybe of adding some methods to the ByteCodeLoading* utilities (the ones that get actually returned by the bean definitions I think) to tell us the input and output types (and wrappers).

Kotlin lambda support?

Now spring cloud function supports Function, Consumer, and Supplier in java.util.function package. Any idea to support Kotlin lambda?

Function == (T) -> R
Consumer == (T) -> Unit
Supplier == ()-> T

Kotlin Code with java function

 @Bean
    fun uppercase(): Function<String, String> {
        return Function { it.toUpperCase() }
    }

Kotlin Code with Lambda

    @Bean
    fun uppercase2(): (String) -> String {
        return { it.toUpperCase() }
    }

Integration with spring-state-machine for simple bpm engine

it will be like a stateless bpm. With custom dsl we could register some flow like:

functionFlow {
  consumer(name: 'somename', transport: queue|http, port:9000){
    if (response == 'foo'){
      payload = payload + 'foo')
      send('logger')
    } else send('nop')
  }

  consumer(name: 'logger', transport: queue|http, port:9000){
    println "LOG: $payload"
  }
}

inner {} is a groovy closure, what can be dehydrated(serialized)

Strategy for Stream Listener Function Mapping

Adapter for Azure functions

Azure supports Java functions now, so it should be possible to make progress on an adapter.

How to modularize a single resource under single project?

Saw this project while learning AWS Lamda and it's perfect timing! Correct me if I'm wrong but Spring Cloud Function is designed to do single function. If this is the case then it may get really annoying really fast. For example, I want to create a project called UserFunction that has CRUD operation powered by Spring Data. So, do I really need to create 4 different project to support User CRUD operation? I can easily imagine to have 20~30 projects to maintain 10 REST API point. Trying to deploy that to dev, staging, prod can be a bit nightmarish..in my opinion. Please correct me if I'm completely wrong! Like to know how others would use this library.

Special handling of functions that process finite streams

A function that explicitly maps Flux<Foo> to Mono<Bar> (for instance) is very clearly sending a signal that it should not be applied to an infinite stream.

Remove (in principle) unnecessary gradle configuration for thinProperties in samples

We shouldn't need the thinProperties task (we don't need the equivalent in the Maven builds). Needs to wait for spring-gradle-plugins/dependency-management-plugin#184.

Make FunctionRegistration public again

592eef9 reduced visibility of FunctionRegistration, which renders the injection of Map<String, FunctionRegistration> into ContextFunctionCatalogAutoConfiguration effectively impossible.

This is a shame, as providing FunctionRegistration directly is useful in certain scenarios where we don't want the framework to try to guess the functional type from e.g. @Bean method signatures.

how to extract headers from when function is invoked by aws api gateway ?

SpringBootRequestHandler<E,Q> seems to only expose request body, how to access headers ?

Add documentation module

spring-cloud-function-openwhisk-adapter - error when running as web action

When action is invoked from CLI OpenWhisk sends following parameters:

{"activation_id":"9bd55a68dd774597aaf604c09d41402f","action_name":"/guest/insertAction","deadline":"1501710494358","api_key":"******","value":{"payload":"PARAMVALUE"},"namespace":"guest"}
When action is invoked as a web action it sends some additional parameters:
{"activation_id":"1a586c3872064ab6a03d6ebf306c853f","action_name":"/guest/insertAction","deadline":"1501710696980","api_key":"******","value":{"__ow_method":"post","__ow_headers":{"x-forwarded-port":"8080","accept":"*/*","x-request-id":"8YTk5zhGtzebPvz2iSpRKoCeecpHoKOt","user-agent":"curl/7.35.0","x-forwarded-proto":"http","host":"controllers","content-length":"30","content-type":"application/json","x-real-ip":"172.17.0.1","connection":"close","x-forwarded-host":"172.17.0.1","x-forwarded-for":"172.17.0.1"},"__ow_path":"","payload":"PARAMVALUE"},"namespace":"guest"}
This params can't be mapped to ActionRequest object because it contains "value" field of type Map<String,String> and __ow_headers is an object not string.

Is it possible to make a sequence of remote functions?

Is it possible to make a sequence of remote functions (not in one jvm)?
If it possible, it will be greate, to provide some snippet.
Thanks!

Add `@EnableFunctionRegistry`

Currently the spring-cloud-function-compiler can act as ether an app (java -jar) or a library. As an app, it exposes the /function/ REST endpoints, but as a library it simply provides the proxy app listener that itself uses FunctionCompiler et al.

We should add support for @EnableFunctionRegistry so that the /function/ REST endpoints can be enabled when spring-cloud-function-compiler is a dependency (and perhaps add @EnableLambdaCompilation or something similar as well, rather than always-on auto-config for the app listener).

Extend semantics of HTTP endpoints

When trying to write actual application code, I'm finding the simple mapping from Function, Supplier and Consumer to HTTP/REST actions a bit limiting. Some ideas...

1. Tighten up the content negotiation, so that HTTP clients know what to expect. E.g. actually test what happens when you POST to a Consumer.

2. Make it easy to express the common idiom "POST an entity" get back a 202 (Accepted), or a 201 (Created) or 204 (No Content)?

3. Make it possible for the developer to control these patterns.

4. Maybe allow users to "replay" content that was streamed into a Consumer (automatically generate a GET endpoint that mirrors a POST for instance).

5. More nuanced handling of infinite streams. Pagination as an alternative to SSEs, for instance.

6. Allow resource paths to be configured more flexibly (e.g. by bean name and by alias as well). This could open up the possibility of having "/" resource separators in URI paths. Makes configuring a messaging stream more of a headache, but I guess that can be done with aliases.

7. Make single-valued functions return single values (not arrays) if the request is single-valued

• = done