Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved handling of corrupted JPEG #91

Merged
merged 28 commits into from
Apr 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
4f4bf13
Added some more problematic files
StefanOltmann Apr 15, 2024
6749c8a
More robust reading of broken files.
StefanOltmann Apr 15, 2024
9ef26d4
Ignore zero length JPEG segments
StefanOltmann Apr 15, 2024
4f824e9
Reduced complexity
StefanOltmann Apr 15, 2024
7f55d56
Updated XmpExtractionTest.kt
StefanOltmann Apr 15, 2024
c6abb84
metadata.csv updated
StefanOltmann Apr 15, 2024
3d4a329
Fixed ExifThumbnailExtractionTest
StefanOltmann Apr 15, 2024
5fbf74e
Updated JpegImageParser.getImageSize()
StefanOltmann Apr 15, 2024
60c879a
Updated JpegOrientationOffsetFinder
StefanOltmann Apr 15, 2024
071cc1e
Updated JpegSegmentAnalyzerTest.kt
StefanOltmann Apr 15, 2024
fc0ade6
Updated JpegMetadataExtractor.kt
StefanOltmann Apr 15, 2024
f714362
JpegRewriterTest.kt: Skip image 41 in tests for now.
StefanOltmann Apr 15, 2024
e0b9765
Updated modification of corrupted files
StefanOltmann Apr 15, 2024
45d103d
assertEquals() is more helpful to see the difference
StefanOltmann Apr 15, 2024
9ff86ce
Added notes on photo_42.jpg
StefanOltmann Apr 15, 2024
2b51d50
Added notes on photo_43.jpg
StefanOltmann Apr 15, 2024
0994800
photo_46.jpg: Remove offsets
StefanOltmann Apr 15, 2024
1726a42
Improved ExifDateUtil.kt
StefanOltmann Apr 15, 2024
1166da6
photo_46.txt: Illegal offsets field dropped
StefanOltmann Apr 15, 2024
d75b3f3
Refactored PhotoMetadataConverter
StefanOltmann Apr 15, 2024
4a9d7a7
Renamed
StefanOltmann Apr 15, 2024
f4168ac
Bump version number
StefanOltmann Apr 15, 2024
9288ca6
Updated Photo Organizer project link
StefanOltmann Apr 15, 2024
9f047c9
Another exception
StefanOltmann Apr 15, 2024
e6cd07b
Fixed ExifDateUtil.kt
StefanOltmann Apr 15, 2024
2fc17d2
Added a warning note to the sample
StefanOltmann Apr 15, 2024
ad6a78b
"Ashampoo Photos" is now "Ashampoo Photo Organizer"
StefanOltmann Apr 15, 2024
e597f33
Removed debug code
StefanOltmann Apr 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

Kim is a Kotlin Multiplatform library for reading and writing image metadata.

It's part of [Ashampoo Photos](https://ashampoo.com/photos).
It's part of [Ashampoo Photo Organizer](https://ashampoo.com/photo-organizer).

## Features

Expand All @@ -34,12 +34,12 @@ It's part of [Ashampoo Photos](https://ashampoo.com/photos).
+ JPG: Lossless rotation by modifying only one byte (where present)

The future development of features on our part is driven entirely by the needs
of Ashampoo Photos, which, in turn, is driven by user community feedback.
of Ashampoo Photo Organizer, which, in turn, is driven by user community feedback.

## Installation

```
implementation("com.ashampoo:kim:0.17.3")
implementation("com.ashampoo:kim:0.17.4")
```

For the targets `wasmJs` & `js` you also need to specify this:
Expand Down
2 changes: 2 additions & 0 deletions examples/kim-kotlin-jvm-sample/src/main/kotlin/Main.kt
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,8 @@ fun setGeoTiffToJpeg() {

/**
* Shows how to update set GeoTiff to a TIF file using JVM API.
*
* CAUTION: Writing TIFF is experimental and may corrupt the file!
*/
fun setGeoTiffToTiff() {

Expand Down
10 changes: 3 additions & 7 deletions src/commonMain/kotlin/com/ashampoo/kim/common/ExifDateUtil.kt
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,9 @@
*/
package com.ashampoo.kim.common

/**
* This class contains extensions needed to
* interpret bytes in photo files.
*/

private val emptyExifDates = setOf(
private val emptyExifDateStrings = setOf(
"0000:00:00 00:00:00",
" : : : : ",
" "
)

Expand All @@ -43,7 +39,7 @@ private const val FIRST_SECOND_INDEX = 17
private const val SECOND_SECOND_INDEX = 18

fun isExifDateEmpty(exifDate: String?): Boolean =
exifDate.isNullOrEmpty() || emptyExifDates.contains(exifDate)
exifDate.isNullOrBlank() || emptyExifDateStrings.contains(exifDate)

/**
* EXIF dates are in the format of "yyyy:MM:dd HH:mm:ss" (19 chars),
Expand Down
268 changes: 144 additions & 124 deletions src/commonMain/kotlin/com/ashampoo/kim/common/PhotoMetadataConverter.kt
Original file line number Diff line number Diff line change
Expand Up @@ -31,167 +31,187 @@ import com.ashampoo.kim.model.TiffOrientation
import kotlinx.datetime.LocalDateTime
import kotlinx.datetime.TimeZone
import kotlinx.datetime.toInstant
import kotlin.jvm.JvmOverloads
import kotlin.jvm.JvmStatic

fun ImageMetadata.convertToPhotoMetadata(
ignoreOrientation: Boolean = false
): PhotoMetadata {
/*
* This is a dedicated object with @JvmStatic methods
* to provide a better API to pure Java projects.
*/
object PhotoMetadataConverter {

@JvmStatic
@JvmOverloads
@Suppress("LongMethod")
fun convertToPhotoMetadata(
imageMetadata: ImageMetadata,
ignoreOrientation: Boolean = false
): PhotoMetadata {

val orientation = if (ignoreOrientation)
TiffOrientation.STANDARD
else
TiffOrientation.of(imageMetadata.findShortValue(TiffTag.TIFF_TAG_ORIENTATION)?.toInt())

val orientation = if (ignoreOrientation)
TiffOrientation.STANDARD
else
TiffOrientation.of(findShortValue(TiffTag.TIFF_TAG_ORIENTATION)?.toInt())
val takenDateMillis = extractTakenDateMillis(imageMetadata)

val takenDateMillis = extractTakenDateMillis(this)
val gpsDirectory = imageMetadata.findTiffDirectory(TiffConstants.TIFF_DIRECTORY_GPS)

val gpsDirectory = findTiffDirectory(TiffConstants.TIFF_DIRECTORY_GPS)
val gps = gpsDirectory?.let { GPSInfo.createFrom(it) }

val gps = gpsDirectory?.let { GPSInfo.createFrom(it) }
val latitude = gps?.getLatitudeAsDegreesNorth()
val longitude = gps?.getLongitudeAsDegreesEast()

val latitude = gps?.getLatitudeAsDegreesNorth()
val longitude = gps?.getLongitudeAsDegreesEast()
val cameraMake = imageMetadata.findStringValue(TiffTag.TIFF_TAG_MAKE)
val cameraModel = imageMetadata.findStringValue(TiffTag.TIFF_TAG_MODEL)

val cameraMake = findStringValue(TiffTag.TIFF_TAG_MAKE)
val cameraModel = findStringValue(TiffTag.TIFF_TAG_MODEL)
val lensMake = imageMetadata.findStringValue(ExifTag.EXIF_TAG_LENS_MAKE)
val lensModel = imageMetadata.findStringValue(ExifTag.EXIF_TAG_LENS_MODEL)

val lensMake = findStringValue(ExifTag.EXIF_TAG_LENS_MAKE)
val lensModel = findStringValue(ExifTag.EXIF_TAG_LENS_MODEL)
/* Look for ISO at the standard place and fall back to test RW2 logic. */
val iso = imageMetadata.findShortValue(ExifTag.EXIF_TAG_ISO)
?: imageMetadata.findShortValue(ExifTag.EXIF_TAG_ISO_PANASONIC)

/* Look for ISO at the standard place and fall back to test RW2 logic. */
val iso = findShortValue(ExifTag.EXIF_TAG_ISO)
?: findShortValue(ExifTag.EXIF_TAG_ISO_PANASONIC)
val exposureTime = imageMetadata.findDoubleValue(ExifTag.EXIF_TAG_EXPOSURE_TIME)
val fNumber = imageMetadata.findDoubleValue(ExifTag.EXIF_TAG_FNUMBER)
val focalLength = imageMetadata.findDoubleValue(ExifTag.EXIF_TAG_FOCAL_LENGTH)

val exposureTime = findDoubleValue(ExifTag.EXIF_TAG_EXPOSURE_TIME)
val fNumber = findDoubleValue(ExifTag.EXIF_TAG_FNUMBER)
val focalLength = findDoubleValue(ExifTag.EXIF_TAG_FOCAL_LENGTH)
val keywords = mutableSetOf<String>()

val keywords = mutableSetOf<String>()
val iptcRecords = imageMetadata.iptc?.records

val iptcRecords = iptc?.records
iptcRecords?.forEach {

iptcRecords?.forEach {
if (it.iptcType == IptcTypes.KEYWORDS)
keywords.add(it.value)
}

if (it.iptcType == IptcTypes.KEYWORDS)
keywords.add(it.value)
}
val gpsCoordinates =
if (latitude != null && longitude != null)
GpsCoordinates(
latitude = latitude,
longitude = longitude
)
else
null

val gpsCoordinates =
if (latitude != null && longitude != null)
GpsCoordinates(
latitude = latitude,
longitude = longitude
)
else
null
val xmpMetadata: PhotoMetadata? = imageMetadata.xmp?.let {
XmpReader.readMetadata(it)
}

val xmpMetadata: PhotoMetadata? = xmp?.let {
XmpReader.readMetadata(it)
}
val thumbnailBytes = imageMetadata.getExifThumbnailBytes()

val thumbnailBytes = getExifThumbnailBytes()
val thumbnailImageSize = thumbnailBytes?.let {
JpegImageParser.getImageSize(
ByteArrayByteReader(thumbnailBytes)
)
}

val thumbnailImageSize = thumbnailBytes?.let {
JpegImageParser.getImageSize(
ByteArrayByteReader(thumbnailBytes)
/*
* Embedded XMP metadata has higher priority than EXIF or IPTC
* for certain fields because it's the newer format. Some fields
* like rating, faces and persons in image are exclusive to XMP.
*
* Resolution, orientation and capture parameters (camera make,
* iso, exposure time, etc.) are always taken from EXIF.
*/
return PhotoMetadata(
widthPx = imageMetadata.imageSize?.width,
heightPx = imageMetadata.imageSize?.height,
orientation = orientation,
takenDate = xmpMetadata?.takenDate ?: takenDateMillis,
gpsCoordinates = xmpMetadata?.gpsCoordinates ?: gpsCoordinates,
location = xmpMetadata?.location,
cameraMake = cameraMake,
cameraModel = cameraModel,
lensMake = lensMake,
lensModel = lensModel,
iso = iso?.toInt(),
exposureTime = exposureTime,
fNumber = fNumber,
focalLength = focalLength,
flagged = xmpMetadata?.flagged ?: false,
rating = xmpMetadata?.rating,
keywords = keywords.ifEmpty { xmpMetadata?.keywords ?: emptySet() },
faces = xmpMetadata?.faces ?: emptyMap(),
personsInImage = xmpMetadata?.personsInImage ?: emptySet(),
albums = xmpMetadata?.albums ?: emptySet(),
thumbnailImageSize = thumbnailImageSize,
thumbnailBytes = thumbnailBytes
)
}

/*
* Embedded XMP metadata has higher priority than EXIF or IPTC
* for certain fields because it's the newer format. Some fields
* like rating, faces and persons in image are exclusive to XMP.
*
* Resolution, orientation and capture parameters (camera make,
* iso, exposure time, etc.) are always taken from EXIF.
*/
return PhotoMetadata(
widthPx = imageSize?.width,
heightPx = imageSize?.height,
orientation = orientation,
takenDate = xmpMetadata?.takenDate ?: takenDateMillis,
gpsCoordinates = xmpMetadata?.gpsCoordinates ?: gpsCoordinates,
location = xmpMetadata?.location,
cameraMake = cameraMake,
cameraModel = cameraModel,
lensMake = lensMake,
lensModel = lensModel,
iso = iso?.toInt(),
exposureTime = exposureTime,
fNumber = fNumber,
focalLength = focalLength,
flagged = xmpMetadata?.flagged ?: false,
rating = xmpMetadata?.rating,
keywords = keywords.ifEmpty { xmpMetadata?.keywords ?: emptySet() },
faces = xmpMetadata?.faces ?: emptyMap(),
personsInImage = xmpMetadata?.personsInImage ?: emptySet(),
albums = xmpMetadata?.albums ?: emptySet(),
thumbnailImageSize = thumbnailImageSize,
thumbnailBytes = thumbnailBytes
)
}
@JvmStatic
fun extractTakenDateAsIsoString(metadata: ImageMetadata): String? {

private fun extractTakenDateAsIso(metadata: ImageMetadata): String? {
val takenDateField = metadata.findTiffField(ExifTag.EXIF_TAG_DATE_TIME_ORIGINAL)
?: return null

val takenDateField = metadata.findTiffField(ExifTag.EXIF_TAG_DATE_TIME_ORIGINAL)
var takenDate = takenDateField.value as? String

var takenDate = takenDateField?.value as? String
/*
* Workaround in case that it's a String array.
*/
if (takenDate == null)
takenDate = takenDateField.toStringValue()

/*
* photo_53.jpg of our test data triggers a bug here.
* This is workaround code.
*/
if (takenDate == null && takenDateField != null)
takenDate = takenDateField.toStringValue()
if (isExifDateEmpty(takenDate))
return null

if (takenDate == null || isExifDateEmpty(takenDate))
return null
return convertExifDateToIso8601Date(takenDate)
}

return convertExifDateToIso8601Date(takenDate)
}
@JvmStatic
fun extractTakenDateMillis(metadata: ImageMetadata): Long? {

private fun extractTakenDateMillis(metadata: ImageMetadata): Long? {
var takenDate: String? = null

val exif = metadata.exif
try {

if (exif == null)
return exif
takenDate = extractTakenDateAsIsoString(metadata) ?: return null

var takenDate: String? = null
val takenDateSubSecond = metadata
.findStringValue(ExifTag.EXIF_TAG_SUB_SEC_TIME_ORIGINAL)
?.toIntOrNull()
?: 0

try {
/*
* If the date string itself contains a sub second like "2020-08-30T18:43:00.500"
* this should be used. We append it, if the string does not have a dot yet.
*/
val takenDatePlusSubSecond = if (!takenDate.contains('.'))
"$takenDate.$takenDateSubSecond"
else
takenDate

takenDate = extractTakenDateAsIso(metadata) ?: return null
val timeZone = if (underUnitTesting)
TimeZone.of("GMT+02:00")
else
TimeZone.currentSystemDefault()

val takenDateSubSecond = metadata
.findStringValue(ExifTag.EXIF_TAG_SUB_SEC_TIME_ORIGINAL)
?.toIntOrNull()
?: 0
return LocalDateTime.parse(takenDatePlusSubSecond)
.toInstant(timeZone)
.toEpochMilliseconds()

/*
* If the date string itself contains a sub second like "2020-08-30T18:43:00.500"
* this should be used. We append it, if the string does not have a dot yet.
*/
val takenDatePlusSubSecond = if (!takenDate.contains('.'))
"$takenDate.$takenDateSubSecond"
else
takenDate
} catch (ignore: Exception) {

val timeZone = if (underUnitTesting)
TimeZone.of("GMT+02:00")
else
TimeZone.currentSystemDefault()
/*
* Many photos contain wrong values here. We ignore this problem and hope
* that another taken date source like embedded XMP has a valid date instead.
*/
println("Ignore invalid EXIF DateTimeOriginal: '$takenDate'")

return LocalDateTime.parse(takenDatePlusSubSecond)
.toInstant(timeZone)
.toEpochMilliseconds()
return null
}
}

} catch (ignore: Exception) {
}

/*
* Many photos contain wrong values here. We ignore this problem and hope
* that another taken date source like embedded XMP has a valid date instead.
*/
println("Ignore invalid EXIF DateTimeOriginal: '$takenDate'")
fun ImageMetadata.convertToPhotoMetadata(
ignoreOrientation: Boolean = false
): PhotoMetadata =
PhotoMetadataConverter.convertToPhotoMetadata(
imageMetadata = this,
ignoreOrientation = ignoreOrientation
)

return null
}
}
Loading
Loading