Comments (4)
Lucas Ward commented
There are three separate concerns that are being controlled by these policies:
- RestartPolicy: should the same job instance always be used for given RuntimeInformation? - Job Level
- StartPolicy: should a step be allowed to start? - Step Level. This policy is absolutely meaningless if restart = false.
- RecoveryPolicy: should restart data be persisted for a given step? This is equally as meaningless if restart = false.
Since both start and recovery policy rely on RestartPolicy being true, there could be just one RestartPolicy. However, things get a bit hairy when you want to control the start counts for different steps independently of the job as a whole. For example, if the job has a restart limit of 10 (a start limit on a non-restartable job is pointless) and has two steps, the first has an infinite start limit (functionally whatever the restart limit on the job is) and the second step has a start limit of one. This means that for some reason, step number two can only be run once, any failures will require some stop of manual intervention before trying to run again, or a new job to be created, so that all data is 'blown out'. Using only one limit in a RestartPolicy means that the whole job would have to have a start limit of 1, which, if step1 failed, wouldn't be correct.
One solution would be to separate the 'start limit' from the restart policy. This would mean that the JobConfiguration and StepConfiguration objects would all have integer startLimit's similar to StepConfiguration's commitInterval. The only real downside to this is that if a job is restartable, all start limit's would be effectively useless. (although I'm not sure I mind this)
Another Solution would be to continue and have policies and name them differently, although, I'm not sure what the best name would be.
A final wrinkle to this issue is, if limits are moved out of the policies and directly into configuration, is there really a need for policies? Is there any reason why RestartPolicy would ever contain more than a boolean true/false, which is acted upon by the execution environment? The same could be said for RecoveryPolicy, which is essentially :shouldRestartDataBePersisted=true/false. Again, acted upon by the execution environment.
from spring-batch.
Dave Syer commented
I'm in favour of keeping it simple at this stage. I.e. remove the policies and replace with booleans in the Job/StepConfiguration. That would solve the problem of naming the policies at least. And the shouldRestart flag could have the same name for Steps and Jobs (actually I'm not sure the policy couldn't but that's irrelevant if we ditch it).
from spring-batch.
Lucas Ward commented
Modified Step and JobConfiguration to no longer use policies, but instead have all values wired directly in:
JobConfiguration->setRestartable(boolean)
JobConfiguration->setStartLimit(int)
StepConfiguration->setStartLimit(int)
StepConfiguration->setSaveRestartData(boolean)
StepConfiguration->setAllowStartIfComplete(boolean)
The logic for determining if a step should start is now contained within the JobExecutor, which I think I'm okay with. In the unlikely event that a project team really needs to have some other type of logic to determine if a step should be started, they can create a custom JobExecutor. This meshes well with separating the configuration domain from the execution domain, since now the individual developer will no longer need to change their wiring.
from spring-batch.
Dave Syer commented
Assume closed as resolved and released
from spring-batch.
Related Issues (20)
- DELETE CASCADE on Foreign Keys HOT 6
- Spring Batch step write_count less than read_count and filter and skip counts are all zero HOT 1
- Incorrect Chunk property value in implementation of ItemWriter write method HOT 3
- 5.1.2 Backported issues HOT 1
- 5.0.6 Backported issues HOT 1
- remote partitioning doesn't work if you're using graalvm
- Access Job Description
- Kotlin data class support for `FlatFileItemReaderBuilder` HOT 4
- Deserialization of JobParameters throws exception
- Default value for ignoreWarnings in JdbcCursorItemReaderBuilder does not align with documentation HOT 1
- JdbcPagingItemReader - When using sortKeys with alias, I think it should paging by column name rather than alias in the select clause.
- MockStatic and MockConstruction not working in spring batch test cases HOT 1
- ListItemWriter<T>::getWrittenItems has bad wildcard return type HOT 1
- Data class support in JdbcCursorItemReaderBuilder & JdbcPagingItemReaderBuilder HOT 2
- Discrepancy in Thread Safety Documentation for AbstractPagingItemReader Subclasses
- Improve Error Messages in JobParametersBuilder methods
- The table field type in the SQL Server database is ntext and synchronized to the Oracle database, resulting in invalid column types in the corresponding NCLOB
- Add static variables to prevent typos in the metadata tables string path HOT 1
- Deprecate the usage of LobHandler HOT 1
- Spring Batch Job Restart: What should be the correct values for chunk and readerPageSize with respect to grid size.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from spring-batch.