Test.Tasty.Golden

tasty-golden-2.3.5: Golden tests support for tasty

Safe Haskell | None
Language | Haskell2010

Test.Tasty.Golden

Contents

- Functions to create a golden test
- Options
- Various utilities

Description

## Getting Started

To get started with golden testing and this library, see Introduction to golden testing.

This module provides a simplified interface. If you want more, see Test.Tasty.Golden.Advanced.

## Filenames

Filenames are looked up in the usual way, Thus relative names are relative to the processes current working directory. It is common to run tests from the package's root directory (via `cabal test` or `cabal install --enable-tests`), so if your test files are under the `tests/` subdirectory, your relative file names should start with `tests/` (even if your `test.hs` is itself under `tests/`, too).

## Line endings

The best way to avoid headaches with line endings (when running tests both on UNIX and Windows) is to treat your golden files as binary, even when they are actually textual.

This means:

- When writing output files from Haskell code, open them in binary mode (see `openBinaryFile`, `withBinaryFile` and `hSetBinaryMode`). This will disable automatic `\n -> \r\n` conversion on Windows. For convenience, this module exports `writeBinaryFile` which is just like `writeFile` but opens the file in binary mode. When using `ByteString`s note that Data.ByteString and Data.ByteString.Lazy use binary mode for `writeFile`, while Data.ByteString.Char8 and Data.ByteString.Lazy.Char8 use text mode.
- Tell your VCS not to do any newline conversion for golden files. For git check in a `.gitattributes` file with the following contents (assuming your golden files have `.golden` extension):

    *.golden	-text

On its side, tasty-golden reads and writes files in binary mode, too.

Why not let Haskell/git do automatic conversion on Windows? Well, for instance, `tar` will not do the conversion for you when unpacking a release tarball, so when you run `cabal install your-package --enable-tests`, the tests will be broken.

As a last resort, you can strip all `\r`s from both arguments in your comparison function when necessary. But most of the time treating the files as binary does the job.

## Linking

The test suite should be compiled with `-threaded` if you want to avoid blocking any other threads while `goldenVsFileDiff` and similar functions wait for the result of the diff command.

## Windows limitations

When using `goldenVsFileDiff` or `goldenVsStringDiff` under Windows the exit code from the diff program that you specify will not be captured correctly if that program uses `exec`.

More specifically, you will get the exit code of the original child (which always exits with code 0, since it called `exec`), not the exit code of the process which carried on with execution after `exec`. This is different from the behavior prescribed by POSIX but is the best approximation that can be realised under the restrictions of the Windows process model. See `Process` for further details or https://github.com/haskell/process/pull/168 for even more.

Synopsis

- goldenVsFile :: TestName -> FilePath -> FilePath -> IO () -> TestTree
- goldenVsString :: TestName -> FilePath -> IO ByteString -> TestTree
- goldenVsFileDiff :: TestName -> (FilePath -> FilePath -> [String]) -> FilePath -> FilePath -> IO () -> TestTree
- goldenVsStringDiff :: TestName -> (FilePath -> FilePath -> [String]) -> FilePath -> IO ByteString -> TestTree
- newtype SizeCutoff = SizeCutoff {
  - getSizeCutoff :: Int64
  }
- data DeleteOutputFile
  - = Never
  - | OnPass
  - | Always
- writeBinaryFile :: FilePath -> String -> IO ()
- findByExtension :: [FilePath] -> FilePath -> IO [FilePath]
- createDirectoriesAndWriteFile :: FilePath -> ByteString -> IO ()

# Functions to create a golden test

goldenVsFile #

Arguments

:: TestName | test name
-> FilePath | path to the «golden» file (the file that contains correct output)
-> FilePath | path to the output file
-> IO () | action that creates the output file
-> TestTree | the test verifies that the output file contents is the same as the golden file contents

Compare the output file's contents against the golden file's contents after the given action has created the output file.

goldenVsString #

Arguments

:: TestName | test name
-> FilePath | path to the «golden» file (the file that contains correct output)
-> IO ByteString | action that returns a string
-> TestTree | the test verifies that the returned string is the same as the golden file contents

Compare a given string against the golden file's contents.

goldenVsFileDiff #

Arguments

:: TestName | test name
-> (FilePath -> FilePath -> [String]) | function that constructs the command line to invoke the diff command. E.g.
    \ref new -> ["diff", "-u", ref, new]
-> FilePath | path to the golden file
-> FilePath | path to the output file
-> IO () | action that produces the output file
-> TestTree |

Same as `goldenVsFile`, but invokes an external diff command.

See the notes at the top of this module regarding linking with `-threaded` and Windows-specific issues.

goldenVsStringDiff #

Arguments

:: TestName | test name
-> (FilePath -> FilePath -> [String]) | function that constructs the command line to invoke the diff command. E.g.
    \ref new -> ["diff", "-u", ref, new]
-> FilePath | path to the golden file
-> IO ByteString | action that returns a string
-> TestTree |

Same as `goldenVsString`, but invokes an external diff command.

See the notes at the top of this module regarding linking with `-threaded` and Windows-specific issues.

# Options

newtype SizeCutoff #

The size, in bytes, such that the (incorrect) test output is not displayed when it exceeds this size. Numeric underscores are accepted for readability.

The default value is 1000 (i.e. 1Kb).

Since: 2.3.3

Constructors

SizeCutoff |
Fields getSizeCutoff :: Int64

#### Instances

Instances details

Enum SizeCutoff # |
Instance details Defined in Test.Tasty.Golden.Internal Methods succ :: SizeCutoff -> SizeCutoff # pred :: SizeCutoff -> SizeCutoff # toEnum :: Int -> SizeCutoff # fromEnum :: SizeCutoff -> Int # enumFrom :: SizeCutoff -> [SizeCutoff] # enumFromThen :: SizeCutoff -> SizeCutoff -> [SizeCutoff] # enumFromTo :: SizeCutoff -> SizeCutoff -> [SizeCutoff] # enumFromThenTo :: SizeCutoff -> SizeCutoff -> SizeCutoff -> [SizeCutoff] #
Num SizeCutoff # |
Instance details Defined in Test.Tasty.Golden.Internal Methods (+) :: SizeCutoff -> SizeCutoff -> SizeCutoff # (-) :: SizeCutoff -> SizeCutoff -> SizeCutoff # (*) :: SizeCutoff -> SizeCutoff -> SizeCutoff # negate :: SizeCutoff -> SizeCutoff # abs :: SizeCutoff -> SizeCutoff # signum :: SizeCutoff -> SizeCutoff # fromInteger :: Integer -> SizeCutoff #
Integral SizeCutoff # |
Instance details Defined in Test.Tasty.Golden.Internal Methods quot :: SizeCutoff -> SizeCutoff -> SizeCutoff # rem :: SizeCutoff -> SizeCutoff -> SizeCutoff # div :: SizeCutoff -> SizeCutoff -> SizeCutoff # mod :: SizeCutoff -> SizeCutoff -> SizeCutoff # quotRem :: SizeCutoff -> SizeCutoff -> (SizeCutoff, SizeCutoff) # divMod :: SizeCutoff -> SizeCutoff -> (SizeCutoff, SizeCutoff) # toInteger :: SizeCutoff -> Integer #
Real SizeCutoff # |
Instance details Defined in Test.Tasty.Golden.Internal Methods toRational :: SizeCutoff -> Rational #
Eq SizeCutoff # |
Instance details Defined in Test.Tasty.Golden.Internal Methods (==) :: SizeCutoff -> SizeCutoff -> Bool # (/=) :: SizeCutoff -> SizeCutoff -> Bool #
Ord SizeCutoff # |
Instance details Defined in Test.Tasty.Golden.Internal Methods compare :: SizeCutoff -> SizeCutoff -> Ordering # (<) :: SizeCutoff -> SizeCutoff -> Bool # (<=) :: SizeCutoff -> SizeCutoff -> Bool # (>) :: SizeCutoff -> SizeCutoff -> Bool # (>=) :: SizeCutoff -> SizeCutoff -> Bool # max :: SizeCutoff -> SizeCutoff -> SizeCutoff # min :: SizeCutoff -> SizeCutoff -> SizeCutoff #
IsOption SizeCutoff # |
Instance details Defined in Test.Tasty.Golden.Internal Methods defaultValue :: SizeCutoff # parseValue :: String -> Maybe SizeCutoff # optionName :: Tagged SizeCutoff String # optionHelp :: Tagged SizeCutoff String # showDefaultValue :: SizeCutoff -> Maybe String # optionCLParser :: Parser SizeCutoff #

data DeleteOutputFile #

When / whether to delete the test output file, when there is a golden file

Since: 2.3.4

Constructors

Never | Never delete the output file (default)
OnPass | Delete the output file if the test passes
Always | Always delete the output file. (May not be commonly used, but provided for completeness.)

#### Instances

Instances details

Show DeleteOutputFile # |
Instance details Defined in Test.Tasty.Golden.Internal Methods showsPrec :: Int -> DeleteOutputFile -> ShowS # show :: DeleteOutputFile -> String # showList :: [DeleteOutputFile] -> ShowS #
Eq DeleteOutputFile # |
Instance details Defined in Test.Tasty.Golden.Internal Methods (==) :: DeleteOutputFile -> DeleteOutputFile -> Bool # (/=) :: DeleteOutputFile -> DeleteOutputFile -> Bool #
Ord DeleteOutputFile # |
Instance details Defined in Test.Tasty.Golden.Internal Methods compare :: DeleteOutputFile -> DeleteOutputFile -> Ordering # (<) :: DeleteOutputFile -> DeleteOutputFile -> Bool # (<=) :: DeleteOutputFile -> DeleteOutputFile -> Bool # (>) :: DeleteOutputFile -> DeleteOutputFile -> Bool # (>=) :: DeleteOutputFile -> DeleteOutputFile -> Bool # max :: DeleteOutputFile -> DeleteOutputFile -> DeleteOutputFile # min :: DeleteOutputFile -> DeleteOutputFile -> DeleteOutputFile #
IsOption DeleteOutputFile # | This option controls when / whether the test output file is deleted For example, it may be convenient to delete the output file when a test passes, since it will be the same as the golden file. It does nothing if running the test or accessing an existing golden value threw an exception. there is no golden file for the test
Instance details Defined in Test.Tasty.Golden.Internal Methods defaultValue :: DeleteOutputFile # parseValue :: String -> Maybe DeleteOutputFile # optionName :: Tagged DeleteOutputFile String # optionHelp :: Tagged DeleteOutputFile String # showDefaultValue :: DeleteOutputFile -> Maybe String # optionCLParser :: Parser DeleteOutputFile #

# Various utilities

writeBinaryFile :: FilePath -> String -> IO () #

Like `writeFile`, but uses binary mode. (Needed only when you work with `String`.)

findByExtension #

Arguments

:: [FilePath] | extensions
-> FilePath | directory
-> IO [FilePath] | paths

Find all files in the given directory and its subdirectories that have the given extensions.

It is typically used to find all test files and produce a golden test per test file.

The returned paths use forward slashes to separate path components, even on Windows. Thus if the file name ends up in a golden file, it will not differ when run on another platform.

The semantics of extensions is the same as in `takeExtension`. In particular, non-empty extensions should have the form `".ext"`.

This function may throw any exception that `getDirectoryContents` may throw.

It doesn't do anything special to handle symlinks (in particular, it probably won't work on symlink loops).

Nor is it optimized to work with huge directory trees (you'd probably want to use some form of coroutines for that).

createDirectoriesAndWriteFile :: FilePath -> ByteString -> IO () #

Like `writeFile`, but also create parent directories if they are missing.