Giter Site home page Giter Site logo

swift-image's Introduction

SwiftImage

Build Status

SwiftImage is an image library written in Swift, which provides Swifty APIs and image types with value semantics.

var image = Image<RGBA<UInt8>>(named: "ImageName")!

let pixel: RGBA<UInt8> = image[x, y]
image[x, y] = RGBA(red: 255, green: 0, blue: 0, alpha: 127)
image[x, y] = RGBA(0xFF00007F) // red: 255, green: 0, blue: 0, alpha: 127

// Iterates over all pixels
for pixel in image {
    // ...
}

// Image processing (e.g. binarizations)
let binarized: Image<Bool> = image.map { $0.gray >= 127 }

// From/to `UIImage`
image = Image<RGBA<UInt8>>(uiImage: imageView.image!)
imageView.image = image.uiImage

Introduction

SwiftImage makes it easy to access pixels of images. The Image type in SwiftImage can be used intuitively like 2D Array.

var image: Image<UInt8> = Image(width: 640, height: 480, pixels: [255, 248, /* ... */])

let pixel: UInt8 = image[x, y]
image[x, y] = 255

let width: Int = image.width // 640
let height: Int = image.height // 480

We can also access pixels of images using CoreGraphics. However, CoreGraphics requires us to struggle with complicated formats, old C APIs and painful memory management. SwiftImage provides clear and Swifty APIs for images.

Typically Image is used with the RGBA type. RGBA is a simple struct declared as follows.

struct RGBA<Channel> {
    var red: Channel
    var green: Channel
    var blue: Channel
    var alpha: Channel
}

Because RGBA is a generic type, it can represent various formats of pixels. For example, RGBA<UInt8> represents a pixel of 8-bit RGBA image (each channel has a value in 0...255). Similarly, RGBA<UInt16> represents a pixel of 16-bit RGBA image (0...65535). RGBA<Float> can represent a pixel whose channels are Floats, which is often used for machine learning. A pixel of binary images, which have only black or white pixels and are used for fax, can be represented using RGBA<Bool>.

When RGBA is used with Image, type parameters are nested like Image<RGBA<UInt8>> because both Image and RGBA are generic types. On the other hand, grayscale images can be represented without nested parameters: Image<UInt8> for 8-bit grayscale images and Image<UInt16> for 16-bit grayscale images.

Image and RGBA provide powerful APIs to handle images. For example, it is possible to convert a RGBA image to grayscale combining Image.map with RGBA.gray in one line.

let image: Image<RGBA<UInt8>> = // ...
let grayscale: Image<UInt8> = image.map { $0.gray }

Another notable feature of SwiftImage is that Image is a struct with value semantics, which is achieved using copy-on-write. Therefore,

  • Image instances never be shared
  • defensive copying is unnecessary
  • there are no wasteful copying of Image instances
  • copying is executed lazily only when it is inevitable
var another: Image<UInt8> = image // Not copied here because of copy-on-write
another[x, y] = 255               // Copied here lazily
another[x, y] == image[x, y]      // false: Instances are never shared

Usage

Import

import SwiftImage

Initialization

let image = Image<RGBA<UInt8>>(named: "ImageName")!
let image = Image<RGBA<UInt8>>(contentsOfFile: "path/to/file")!
let image = Image<RGBA<UInt8>>(data: Data(/* ... */))!
let image = Image<RGBA<UInt8>>(uiImage: imageView.image!) // from a UIImage
let image = Image<RGBA<UInt8>>(nsImage: imageView.image!) // from a NSImage
let image = Image<RGBA<UInt8>>(cgImage: cgImage) // from a CGImage
let image = Image<RGBA<UInt8>>(width: 640, height: 480, pixels: pixels) // from a pixel array
let image = Image<RGBA<UInt8>>(width: 640, height: 480, pixel: .black) // a black RGBA image
let image = Image<UInt8>(width: 640, height: 480, pixel: 0) // a black grayscale image
let image = Image<Bool>(width: 640, height: 480, pixel: false) // a black binary image

Access to a pixel

// Gets a pixel by subscripts
let pixel = image[x, y]
// Sets a pixel by subscripts
image[x, y] = RGBA(0xFF0000FF)
image[x, y].alpha = 127
// Safe get for a pixel
if let pixel = image.pixelAt(x: x, y: y) {
    print(pixel.red)
    print(pixel.green)
    print(pixel.blue)
    print(pixel.alpha)
    
    print(pixel.gray) // (red + green + blue) / 3
    print(pixel) // formatted like "#FF0000FF"
} else {
    // `pixel` is safe: `nil` is returned when out of bounds
    print("Out of bounds")
}

Iteration

for pixel in image {
    ...
}

Rotation

let result = image.rotated(by: .pi) // Rotated clockwise by π
let result = image.rotated(byDegrees: 180) // Rotated clockwise by 180 degrees
// Rotated clockwise by π / 4 and fill the background with red
let result = image.rotated(by: .pi / 4, extrapolatedBy: .filling(.red))

Flip

let result = image.xReversed() // Flip Horizontally
let result = image.yReversed() // Flip Vertically

Resizing

let result = image.resizedTo(width: 320, height: 240)
let result = image.resizedTo(width: 320, height: 240,
    interpolatedBy: .nearestNeighbor) // Nearest neighbor

Crop

Slicing is executed with no copying costs.

let slice: ImageSlice<RGBA<UInt8>> = image[32..<64, 32..<64] // No copying costs
let cropped = Image<RGBA<UInt8>>(slice) // Copying is executed here

Conversion

Image can be converted by map in the same way as Array. Followings are the examples.

Grayscale

let result: Image<UInt8> = image.map { (pixel: RGBA<UInt8>) -> UInt8 in
    pixel.gray
}
// Shortened form
let result = image.map { $0.gray }

Binarization

let result: Image<Bool> = image.map { (pixel: RGBA<UInt8>) -> Bool in
    pixel.gray >= 128
}
// Shortened form
let result = image.map { $0.gray >= 128 }

Binarization (auto threshold)

let threshold = UInt8(image.reduce(0) { $0 + $1.grayInt } / image.count)
let result = image.map { $0.gray >= threshold }

Mean filter

let kernel = Image<Float>(width: 3, height: 3, pixel: 1.0 / 9.0)
let result = image.convoluted(kernel)

Gaussian filter

let kernel = Image<Int>(width: 5, height: 5, pixels: [
    1,  4,  6,  4, 1,
    4, 16, 24, 16, 4,
    6, 24, 36, 24, 6,
    4, 16, 24, 16, 4,
    1,  4,  6,  4, 1,
]).map { Float($0) / 256.0 }
let result = image.convoluted(kernel)

With UIImage

// From `UIImage`
let image = Image<RGBA<UInt8>>(uiImage: imageView.image!)

// To `UIImage`
imageView.image = image.uiImage

With NSImage

// From `NSImage`
let image = Image<RGBA<UInt8>>(nsImage: imageView.image!)

// To `NSImage`
imageView.image = image.nsImage

With CoreGraphics

// Drawing on images with CoreGraphics
var image = Image<PremultipliedRGBA<UInt8>>(uiImage: imageView.image!)
image.withCGContext { context in
    context.setLineWidth(1)
    context.setStrokeColor(UIColor.red.cgColor)
    context.move(to: CGPoint(x: -1, y: -1))
    context.addLine(to: CGPoint(x: 640, y: 480))
    context.strokePath()
}
imageView.image = image.uiImage

Requirements

  • Swift 5.0 or later
  • Xcode 10.2 or later

Installation

Swift Package Manager

.package(url: "https://github.com/koher/swift-image.git", from: "0.7.0"),
github "koher/swift-image" ~> 0.7.0

Manually

  1. Put SwiftImage.xcodeproj into your project/workspace in Xcode.
  2. Click your project icon and select the application target and the "General" tab.
  3. Add SwiftImage.framework to "Embedded Binaries".

License

The MIT License

swift-image's People

Contributors

dastrobu avatar jeehut avatar koher avatar rayfix avatar t-ae avatar tellowkrinkle avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

swift-image's Issues

Carthage install not working

when I try to install via carthage I get this error:
Parse error: expected submodule commit SHA in output of task (ls-tree -z HEAD EasyImagy) but encountered:

my Cartfile:
github "koher/EasyImagy" ~> 0.5.0

Any ideas how to fix it?

Using the library on Linux

I am trying to compile the package on Linux which fails. I am aware that you are not claiming compatibility with Linux but was hoping that you may know a quick fix that would address the below compilation errors.

First of all I am using it as follows:

func imageDimensions(fromData data: Data) -> (width: Int, height: Int) {
   if let image = Image<RGBA<UInt8>>(data: data) {
        let width = image.width
        let height = image.height
        return (width: width, height: height)
    }
    return (width: -1, height: -1)
}

This method works fine on macOS.

Then when I compile this on Linux I get the following compilation errors:

/project/.build-linux/checkouts/Mercury/Sources/Mercury/ImageDimensions.swift:26:38: 
error: extraneous argument label 'data:' in call
    if let image = Image<RGBA<UInt8>>(data: data) {
                                     ^~~~~~~

/project/.build-linux/checkouts/Mercury/Sources/Mercury/ImageDimensions.swift:26:20: 
error: initializer 'init(_:)' requires the types 'RGBA<UInt8>' 
and 'UInt8' be equivalent
    if let image = Image<RGBA<UInt8>>(data: data) {
                   ^
EasyImagy.ImageProtocol:6:12: note: where 'Self.Element' = 'RGBA<UInt8>', 
'I.Element' = 'UInt8'
    public init<I>(_ image: I) where I : ImageProtocol, Self.Element == I.Element
           ^
/project/.build-linux/checkouts/Mercury/Sources/Mercury/ImageDimensions.swift:26:20: 
error: initializer 'init(_:)' requires that 'Data' conform to 'ImageProtocol'
    if let image = Image<RGBA<UInt8>>(data: data) {
                   ^
EasyImagy.ImageProtocol:6:12: note: where 'I' = 'Data'
    public init<I>(_ image: I) where I : ImageProtocol, Self.Element == I.Element
           ^

Based on the above compilation errors, could you perhaps give a hint on how to fix this?

Does not compile for Mac Catalyst

When compiling targeting My Mac (Mac Catalyst), I get the following errors:

swift-image/Sources/SwiftImage/UIKit.swift:56:12: error: invalid redeclaration of 'init(named:)'
    public init?(named name: String) {
           ^
swift-image/Sources/SwiftImage/AppKit.swift:22:12: note: 'init(named:)' previously declared here
    public init?(named name: NSImage.Name) {
           ^
swift-image/Sources/SwiftImage/UIKit.swift:68:12: error: invalid redeclaration of 'init(contentsOfFile:)'
    public init?(contentsOfFile path: String) {
           ^
swift-image/Sources/SwiftImage/AppKit.swift:27:12: note: 'init(contentsOfFile:)' previously declared here
    public init?(contentsOfFile path: String) {
           ^
swift-image/Sources/SwiftImage/UIKit.swift:73:12: error: invalid redeclaration of 'init(data:)'
    public init?(data: Data) {
           ^
swift-image/Sources/SwiftImage/AppKit.swift:32:12: note: 'init(data:)' previously declared here
    public init?(data: Data) {
           ^
swift-image/Sources/SwiftImage/UIKit.swift:83:17: error: invalid redeclaration of 'pngData()'
    public func pngData() -> Data? {
                ^
swift-image/Sources/SwiftImage/AppKit.swift:42:17: note: 'pngData()' previously declared here
    public func pngData() -> Data? {
                ^
swift-image/Sources/SwiftImage/UIKit.swift:91:17: error: invalid redeclaration of 'jpegData(compressionQuality:)'
    public func jpegData(compressionQuality: Double) -> Data? {
                ^
swift-image/Sources/SwiftImage/AppKit.swift:53:17: note: 'jpegData(compressionQuality:)' previously declared here
    public func jpegData(compressionQuality: Double) -> Data? {
                ^
swift-image/Sources/SwiftImage/UIKit.swift:99:17: error: invalid redeclaration of 'data(using:)'
    public func data(using format: ImageFormat) -> Data? {
                ^
swift-image/Sources/SwiftImage/AppKit.swift:64:17: note: 'data(using:)' previously declared here
    public func data(using format: ImageFormat) -> Data? {
                ^
swift-image/Sources/SwiftImage/UIKit.swift:109:17: error: invalid redeclaration of 'write(to:atomically:format:)'
    public func write(to url: URL, atomically: Bool, format: ImageFormat) throws {
                ^
swift-image/Sources/SwiftImage/AppKit.swift:74:17: note: 'write(to:atomically:format:)' previously declared here
    public func write(to url: URL, atomically: Bool, format: ImageFormat) throws {
                ^
swift-image/Sources/SwiftImage/UIKit.swift:117:17: error: invalid redeclaration of 'write(toFile:atomically:format:)'
    public func write<S : StringProtocol>(toFile path: S, atomically: Bool, format: ImageFormat) throws {
                ^
swift-image/Sources/SwiftImage/AppKit.swift:82:17: note: 'write(toFile:atomically:format:)' previously declared here
    public func write<S : StringProtocol>(toFile path: S, atomically: Bool, format: ImageFormat) throws {
                ^

Is it posso lê to compare 2 images given a % diff threshold with this lib?

I saw a pretty cool video of wwdc regarding filtering our two images to generate a third one with the difference between the two compared ones... But it's not what I really need for the project I'm working on.

I need to compare two images given a % threshold of difference accepted between them... I would have an uiimage referenced with the expected result and compared with a runtime generated one... I found how to compare pixel by pixel out there, but I'd like to be more soft comparing in % so it does necessarily needs to match 100% the expected and actual results.

Is it possible to compare pixels/images with diff threshold with this lib? Would you have an example case on how to do that with two uiimage objects?

Thanks

Dennis

Context being lost

I have a program that reads data and plots it using different backends. Each backend implements a protocol and one of the methods in the the protocol is to start clipping. I have used SwiftImage in my png backend, source here. I use image.withCGContext closures to get the context each time I want to do something. What I discovered is that the clipping path I set in one closure was gone in the next.

My initial feeling was that the context should be the same but on the other hand it doesn't seem unreasonable to behave that way. On the off chance that it is a bug I thought I'd let you know.

Slower on iOS 13

I have noticed since updating to iOS 13 from 12.4.1, the library has been running a lot slower than it was before. I used to consistently clock in around 250ms to flip an image over the x-axis, but now it is constantly at 3.1 seconds for the same exact image. Many other built in functions are slower as well. I tried updating Xcode to 11, Swift to 5, and the library itself to 0.7.0 but nothing has helped, and the code has not changed. Has anyone else had this issue or found a solution?

Does this work with Xcode 8 yet?

carthage update returns:
The following build commands failed:
Check dependencies
(1 failure)
A shell task (/usr/bin/xcrun xcodebuild -project /Users//Carthage/Checkouts/EasyImagy/EasyImagy.xcodeproj -scheme EasyImagy-iOS -configuration Release -sdk iphoneos ONLY_ACTIVE_ARCH=NO BITCODE_GENERATION_MODE=bitcode CODE_SIGNING_REQUIRED=NO CODE_SIGN_IDENTITY= CARTHAGE=YES clean build) failed with exit code 65:
** CLEAN FAILED **

The following build commands failed:
Check dependencies
(1 failure)
** BUILD FAILED **

Generics for updating array by matched expression eg. I[I == 144] = 0

Is it possible to implement this kind of expression in swift?

eg.
I[I == 144] = 0 // would match all elements within the array at value 144.

https://github.com/johndpope/gym/blob/master/lingfei/cartpole/policyGrad.py#L61

def prepro(I):
    """ prepro 210x160x3 uint8 frame into 6400 (80x80) 1D float vector """
    I = I[35:195]  # crop
    I = I[::2, ::2, 0]  # downsample by factor of 2
    I[I == 144] = 0  # erase background (background type 1)
    I[I == 109] = 0  # erase background (background type 2)
    I[I != 0] = 1  # everything else (paddles, ball) just set to 1
    return I.astype(np.float).ravel()

image url

Is there an example to use this with image from URL?

helpers for Image<RGBA<UInt8>> -> byte array of [Uint8]

how to flatten out image to an array of uint 8 pixels?
Is there a helper - I'm not worried about expense of copying.

typealias Byte = UInt8

extension Image where Pixel == UInt8 {
    func byteArray()->[Byte]{
        var pixelIterator = self.makeIterator()
        var pixelArray:[Byte] = []
        while let pixel = pixelIterator.next() {
            pixelArray.append(pixel)
        }
        return pixelArray
    }
}

Save images on Linux

Is there any way to save images to png format on linux? That would be great :)

Preparation for Swift 6

Currently, warnings are shown when compiling swift-image. It means that swift-image cannot be compiled in Swift 6. Moreover, it seems that Swift 6 will be released in the not-too-distant future.

$ swift build
Building for debugging...
...
/path/to/swift-image/Sources/SwiftImage/AnyImage.swift:63:31: warning: generic parameter 'Pixel' shadows generic parameter from outer scope with the same name; this is an error in Swift 6
    private class AnyImageBox<Pixel> {
                              ^
/path/to/swift-image/Sources/SwiftImage/AnyImage.swift:3:24: note: 'Pixel' previously declared here
public struct AnyImage<Pixel> : ImageProtocol {
                       ^
...

(Xcode 15.2, Swift 5.9.2, macOS 14.3.1 arm64)

Changing pixels in png's with transparent background fails

Hi I am using your library currently to create a steganography tool for macOS using Swift.

Basically I write the message into the last bit of each RGB value (not altering Alpha).
When I try this on an Image that has a transparent background It won't keep the changed RGB values but just sets them all to 0 because the Alpha is set to 0. I can only change the RGB values if the Alpha is all the way up at 255 (which means no transparency).

Either it's a problem with my code or with your library/framework because I created the same program already in Python and in C# where I have had no problem writing values to the pixels if they have transparency.
C# code for reference: https://paste.purpl3.net/16
Python3 code for reference: https://hider.purpl3.net/static/app.py
These two projects encode and decode images and the resulting images / messages are compatible with the two programs. The Swift version can also generate Images that can be read by the C# and Python version but only if they are PNG's without transparency.

Can you maybe help me out? I tried so much already for 3 days straight...

What is weird is that when I write to the debug console the before and after pixels they are exactly what I want but they don't end up in the final image
Maybe I am not saving the images correctly?

The Swfit project is here
The class I am doing all the work in is this

Does this work with macOS applications?

Hey, just wanna know if this library works with macOS apps? And if not is there a similar library for image (specifically pixel)-manipulation?

EDIT:

I tried importing the macOS Framework to my app but I get a bunch of compiler errors...

/Users/user/Documents/hider/Carthage/Checkouts/EasyImagy/EasyImagy/ImageTyped.swift:171:29: error: same-type requirement makes generic parameter 'Pixel' non-generic
extension Image where Pixel == Int { // Convolution
                            ^
/Users/user/Documents/hider/Carthage/Checkouts/EasyImagy/EasyImagy/ImageTyped.swift:157:29: error: same-type requirement makes generic parameter 'Pixel' non-generic
extension Image where Pixel == UInt8 { // Convolution
                            ^
/Users/user/Documents/hider/Carthage/Checkouts/EasyImagy/EasyImagy/ImageTyped.swift:143:29: error: same-type requirement makes generic parameter 'Pixel' non-generic
extension Image where Pixel == RGBA { // Convolution
                            ^
/Users/user/Documents/hider/Carthage/Checkouts/EasyImagy/EasyImagy/ImageTyped.swift:137:29: error: same-type requirement makes generic parameter 'Pixel' non-generic
extension Image where Pixel == Double { // map with indices

Swift is version 3.0.1

Build error in Xcode10 beta

On Xcode 10.0 beta4 I've got two errors Invalid redeclaration of 'countableRange(from:relativeTo:)' for internal func countableRange(from range: Range<Int>, relativeTo collection: CountableRange<Int>) -> CountableRange<Int> and internal func countableRange(from range: ClosedRange<Int>, relativeTo collection: CountableRange<Int>) -> CountableRange<Int> in Util.swift

With the implementation of SE-0143 Conditional conformances in Swift 4.2, CountableRange and CountableClosedRange are not separate types relative to Range, ClosedRange, but their type aliases.

Release on CocoaPods?

This library looks to be just what I need. Are you planning to release this on CocoaPods?

enhancement/helper - LBP - local binary patterns

https://youtu.be/-Ja-vLbHWLc?t=49
https://en.wikipedia.org/wiki/Local_binary_patterns
was thinking
screen shot 2018-08-22 at 11 26 14 am

extension Pixel{
func determinePattern()->LBP{
    

     get the surrounding pixels
     do the calculations
     return 01010111



}

digging deeper into this - > there's a suite of these

BG-LBP (BackGround Local Binary Pattern) by Davarpanah et al. (2015)
CS-LBP (First-order Center-Symmetric Local Binary Patterns) by Heikkilä et al. (2006)
CS-LDP (Second-order Center-Symmetric Local Derivative Pattern) by Xue et al. (2011)
CS-SILTP (Center-Symmetric Scale Invariant Local Ternary Patterns) by Wu et al. (2013)
E-LBP (Extended LBP or Circular LBP) by Mdakane and Bergh (2012)
OC-LBP (Opponent Color Local Binary Pattern) by Maenpaa and Pietikainen (2004)
O-LBP (Original LBP) by Ojala et al. (2001)
SCS-LBP (Spatial extended Center-Symmetric Local Binary Pattern) by Xue et al. (2010)
SI-LTP (Scale Invariant Local Ternary Pattern) by Liao et al. (2010)
VAR-LBP (Variance-based LBP) by Ojala et al. (2002)
XCS-LBP (eXtended Center-Symmetric Local Binary Pattern) by Silva et al. (2015)

https://github.com/carolinepacheco/lbplibrary

  void OLBP::OLBP_(const cv::Mat& src, cv::Mat& dst)
  {
    dst = cv::Mat::zeros(src.rows - 2, src.cols - 2, CV_8UC1);
    for (int i = 1; i < src.rows - 1; i++) {
      for (int j = 1; j < src.cols - 1; j++) {
        _Tp center = src.at<_Tp>(i, j);
        unsigned char code = 0;
        code |= (src.at<_Tp>(i - 1, j - 1) > center) << 7;
        code |= (src.at<_Tp>(i - 1, j) > center) << 6;
        code |= (src.at<_Tp>(i - 1, j + 1) > center) << 5;
        code |= (src.at<_Tp>(i, j + 1) > center) << 4;
        code |= (src.at<_Tp>(i + 1, j + 1) > center) << 3;
        code |= (src.at<_Tp>(i + 1, j) > center) << 2;
        code |= (src.at<_Tp>(i + 1, j - 1) > center) << 1;
        code |= (src.at<_Tp>(i, j - 1) > center) << 0;
        dst.at<unsigned char>(i - 1, j - 1) = code;
      }
    }
  }

All pixels are returned as transparent

Am I using this incorrectly? All pixels appear empty.

let image = UIImage(named: "01", in: .main, with: nil)!
let rgbaImage = SwiftImage.Image<RGBA<UInt8>>(uiImage: image)

(0..<rgbaImage.width).forEach { x in
    (0..<rgbaImage.height).forEach { y in

        guard let pixel = rgbaImage.pixelAt(x: x, y: y) else { return }
        // NOTE: pixel.red, green, blue and alpha are all 0 for every pixel
    }
}

thoughts on supporting terminal line app

I'm reviewing a couple of repos that are using tensorflow for swift as a command line app.

I'd like to plug in EasyImagy but unfortuantely - including frameworks with terminal line apps - isn't so supported.

A couple of options came to mind

  1. attempt to leverage marathon for using easyimagy
    https://github.com/JohnSundell/Marathon/

  2. cherry picking code into a single include file. (may sound bizarre / but consider - no pod - just spit out code and concatenate into a versioned file. )

https://github.com/JulienViguier/CNN_Swift4TensorFlow/blob/master/model.swift
https://github.com/saschaschramm/SwiftReinforce/blob/master/SwiftReinforce/Common/Utilities.swift#L55

//CNN_Swift4TensorFlow

struct Pixel {
    var r: Float
    var g: Float
    var b: Float
    var a: Float
    var row: Int
    var col: Int
    
    init(r: UInt8, g: UInt8, b: UInt8, a: UInt8, row: Int, col: Int) {
        self.r = Float(r)
        self.g = Float(g)
        self.b = Float(b)
        self.a = Float(a)
        self.row = row
        self.col = col
    }
}

func pixelToFloat(_pixelTab: [[Pixel]]) -> [Float] {
    var tab: [Float] = []
    for (_, row) in _pixelTab.enumerated() {
        for (_, cell) in row.enumerated() {
            tab.append(Float(cell.r/255.0))
            tab.append(Float(cell.g/255.0))
            tab.append(Float(cell.b/255.0))
        }
    }
    return tab
}

current solution for swift interfacing to python.

func renderPixels(_ pixels: [UInt8], rows: Int, cols: Int) {
    let sys = Python.import("sys")
    let np = Python.import("numpy")
    let path = "\(NSHomeDirectory())/gym/lib/python2.7/site-packages/"
    sys.path.append(path)
    let image = Python.import("PIL.Image")
    
    let foo = np.array(pixels).reshape([rows,cols])
    let img = image.fromarray(np.uint8(foo))
    img.show()
}

Memory leak on creating data from image

First of all, great library!
There seems so be a memory leak when creating data from images. This causes some trouble in one of my apps. To reproduce, do the following

        for _ in 0..<100 {
            let image = Image<UInt8>(width: 1, height: 1, pixels: [0])
            let _ = image.data(using: .png)!
        }

When setting a breakpoint after the loop and investigate the memory of core graphics one will find all the images, see screenshot.
screenshot 2018-12-12 at 01 51 10

So what happens in the image.data is basically the following:

            let data: [UInt8] = [0, 0, 0, 0]
            let provider: CGDataProvider = CGDataProvider(data: Data(
                bytes: UnsafeMutableRawPointer(mutating: data),
                count: 4
            ) as CFData)!

            let cgImage = CGImage(
                width: 1,
                height: 1,
                bitsPerComponent: 1 * 8,
                bitsPerPixel: 4 * 8,
                bytesPerRow: 4 * 1,
                space: CGColorSpaceCreateDeviceRGB(),
                bitmapInfo: CGBitmapInfo(),
                provider: provider,
                decode: nil,
                shouldInterpolate: false,
                intent: CGColorRenderingIntent.defaultIntent
            )!

            let imageRep: NSBitmapImageRep = NSBitmapImageRep(cgImage: cgImage)
            imageRep.size = CGSize(width: CGFloat(1), height: CGFloat(1))
            
            // memory leak only with the following line
            let _ = imageRep.representation(using: .png, properties: [:])

This self contained block of code does basically the same as image.data and is reproducing the memory leak (replace the body of the for loop, and you will find the same leak). So the line of code, causing the trouble is the last one. When removing the last line, the memory leak is gone.

Unfortunately, I have no idea what is wrong with this code.

clarification / helpers - image.pixels

I wanting to pass all this pixel data to another method but it's asking for an array of pixels.
specifically ->
https://github.com/johndpope/fast-swift/blob/master/FAST.swift

this doesn't quite work.

extension Image{
    func pixels()->[Pixel]{
        return self.enumerated()
    }
}

screen shot 2018-08-08 at 2 23 24 pm

screen shot 2018-08-08 at 2 13 14 pm

P.S.

 let image = Image<RGBA<UInt8>>(nsImage: NSImage(named: "fast_1.png")!) -> consider
->
 let image = Image<RGBA<UInt8>>(nsImageNamed: "fast_1.png")  // then coerce this behind scenes.

this
let result = image.map { $0.gray }

should be exposed as just
image.grayScale()

Very slow performance on initialization and resizing

Hi,
I've been trying to load an image (attached) (1.9MB, 2162 × 3843), crop it and then resize the results. It's working, but taking about 30 seconds for the process!
I've seen a closed issue regarding performance in iOS 13, but I've compiled with speed optimizations and I see no change. Here's the code I'm running, along with the time it takes:
Creating image data for image of size: (2162.0, 3843.0):
Code: let imageData = Image<RGBA>(uiImage: image!) ; Time: 6.934875965118408 s
Cropping photo ( 0 < x < 2161, 1316 < y < 2525)
Code: let croppedImageData = Image<RGBA>(data[minX..<maxX, minY..<maxY]) ; Time: 1.0871039628982544
resizing photo to: 2736.0 by 1530.0
Code: let resizedCroppedImageData = croppedImageData.resizedTo(width: 2736, height: 1530) ; Time: 22.0144259929657

My Compiler settings: Optimization level [-O] (Swift compiler - code generation), [-Ofast] (Apple CLang - Code Generation)

The machine I'm running on: iPhone XS, iOS 13.5.1

Is there anything I'm doing wrong? Any help would be appreciated!

18052DE9-661F-4E53-8EFE-24DF60D45282

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.