General Swift properties
Swift behavior can be configured through general Swift properties. Below is a list of properties:
| Name | Valid Values | Default Value | Description |
|---|---|---|---|
| config.rundirs | true, false | true | By default, Swift will generate a run directory that contains logs, scheduler submit scripts, debug directories, and other files associated with a particular Swift run. Setting this value to false disables the creation of run directories and causes all logs and directories to be created in the current working directory. |
| execution.retries | Positive integer | 2 | The number of time a job will be retried if it fails (giving a maximum of 1 + execution.retries attempts at execution) |
| file.gc.enabled | true, false | true | Files mapped by the concurrent mapper (i.e. when you don’t explicitly specify a mapper) are deleted when they are not in use any more. This property can be used to prevent files mapped by the concurrent mapper from being deleted. |
| foreach.max.threads | Positive integer | 1024 | Limits the number of concurrent iterations that each foreach statement can have at one time. This conserves memory for swift programs that have large numbers of iterations (which would otherwise all be executed in parallel) |
| lazy.errors | true, false | false | Swift can report application errors in two modes, depending on the value of this property. If set to false, Swift will report the first error encountered and immediately stop execution. If set to true, Swift will attempt to run as much as possible from a Swift script before stopping execution and reporting all errors encountered. When developing Swift scripts, using the default value of false can make the program easier to debug. However in production runs, using true will allow more of a Swift script to be run bef |
| swift.home | String | Points to the Swift installation directory ($SWIFT_HOME). In general, this should not be set as Swift can find its own installation directory, and incorrectly setting it may impair the correct functionality of Swift. | |
| pgraph | true, false | false | Swift can generate a Graphviz http://www.graphviz.org/ file representing the structure of the Swift script it has run. If this property is set to true, Swift will save the provenance graph in a file named by concatenating the program name and the instance ID (e.g. helloworldht0adgi315l61.dot). If set to false, no provenance graph will be generated. If a file name is used, then the provenance graph will be saved in the specified file. The generated dot file can be rendered into a graphical form using Graphviz http://www.graphviz.org/, for example with a command-line such as: $ swift -pgraph graph1.dot q1.swift $ dot -ograph.png -Tpng graph1.dot |
| pgraph.graph.options | String | splines="compound", rankdir="TB" | This property specifies a Graphviz http://www.graphviz.org specific set of parameters for the graph. |
| pgraph.node.options | String | color="seagreen", style="filled" | Used to specify a set of Graphviz http://www.graphviz.org specific properties for the nodes in the graph. |
| provenance.log | true, false | false | This property controls whether the log file will contain provenance information enabling this will increase the size of log files, sometimes significantly. |
| provider.staging.pin.swiftfiles | true, false | false | When provider staging is enabled and provider.staging.pin.swiftfiles is set, cache some small files needed by Swift to avoid the cost of staging more than once. |
| sitedir.keep | true, false | false | Indicates whether the working directory on the remote site should be left intact even when a run completes successfully. This can be used to inspect the site working directory for debugging purposes. |
| status.mode | files, provider | files | Controls how Swift will communicate the result code of running user programs from workers to the submit side. In files mode, a file indicating success or failure will be created on the site shared filesystem. In provider mode, the execution provider job status will be used. provider mode requires the underlying job execution system to correctly return exit codes. |
| tcp.port.range | none |
|
A TCP port range can be specified to restrict the ports on which GRAM callback services are started. This is likely needed if your submit host is behind a firewall, in which case the firewall should be configured to allow incoming connections on ports in the range |
| throttle.file.operations | < int>, off | 8 | Limits the total number of concurrent file operations that can happen at any given time. File operations (like transfers) require an exclusive connection to a site. These connections can be expensive to establish. A large number of concurrent file operations may cause Swift to attempt to establish many such expensive connections to various sites. Limiting the number of concurrent file operations causes Swift to use a small number of cached connections and achieve better overall performance. |
| throttle.host.submit | < int>, off | 2 | Limits the number of concurrent submissions for any of the sites Swift will try to send jobs to. In other words it guarantees that no more than the value of this throttle jobs sent to any site will be concurrently in a state of being submitted |
| throttle.score.job.factor | < int>, off | 4 | The Swift scheduler has the ability to limit the number of concurrent jobs allowed on a site based on the performance history of that site. Each site is assigned a score (initially 1), which can increase or decrease based on whether the site yields successful or faulty job runs. The score for a site can take values in the (0.1, 100) interval. The number of allowed jobs is calculated using the following formula: 2 + score*throttle.score.job.factor This means a site will always be allowed at least two concurrent jobs and at most 2 + 100*throttle.score.job.factor. With a default of 4 this means at least 2 jobs and at most 402. This parameter can also be set per site using the jobThrottle profile key in a site catalog entry |
| throttle.submit | < int>, off | 4 | Limits the number of concurrent submissions for a run. This throttle only limits the number of concurrent tasks (jobs) that are being sent to sites, not the total number of concurrent jobs that can be run. The submission stage in GRAM is one of the most CPU expensive stages (due mostly to the mutual authentication and delegation). Having too many concurrent submissions can overload either or both the submit host CPU and the remote host/head node causing degraded performance |
| throttle.transfers | < int>, off | 4 | Limits the total number of concurrent file transfers that can happen at any given time. File transfers consume bandwidth. Too many concurrent transfers can cause the network to be overloaded preventing various other signaling traffic from flowing properly |
| ticker.date.format | String | Describes how to format the ticker date output. The format of this string is documented in the Java SimpleDateFormat class, at http://docs.oracle.com/- javase/6/docs/api/java/text/- SimpleDateFormat.html | |
| ticker.disable | true, false | false | When set to true, suppresses the output progress ticker that Swift sends to the console every few seconds during a run |
| ticker.prefix | String | Progress: | String to prepend to ticker output |
| tracing.enabled | true, false | true | Enables tracing of procedure invocations, assignments, iteration constructs, as well as certain dataflow events such as data intialization and waiting. This is done at a slight decrease in performance. Traces will be available in the log file. |
| use.wrapper.staging | true, false | false | Determines if the Swift wrapper should do file staging |
| use.provider.staging | true, false | false | If true, files will be staged by Swift over the network. |
| wrapper.invocation.mode | absolute, relative | absolute | Determines if Swift remote wrappers will be executed by specifying an absolute path, or a path relative to the job initial working directory. In most cases, execution will be successful with either option. However, some execution sites ignore the specified initial working directory, and so absolute must be used. Conversely on some sites, job directories appear in a different place on the worker node file system than on the filesystem access node, with the execution system handling translation of the job initial working directory. In such cases, relative mode must be used. |
| wrapper.parameter.mode | args,files | args | Controls how Swift will supply parameters to the remote wrapper script. args mode will pass parameters on the command line. Some execution systems do not pass commandline parameters sufficiently cleanly for Swift to operate correctly. files mode will pass parameters through an additional input file. This provides a cleaner communication channel for parameters, at the expense of transferring an additional file for each job invocation |
| wrapperlog.always.transfer | true, false | false | This property controls when output from the Swift remote wrapper is transfered back to the submit site. When set to false, wrapper logs are only transfered for jobs that fail. If set to true, wrapper logs are transfered after every job is completed or failed |