diff --git a/manual/includes/_database-dump.md b/manual/includes/_database-dump.md
index c4e13e8..d747d92 100644
--- a/manual/includes/_database-dump.md
+++ b/manual/includes/_database-dump.md
@@ -477,6 +477,13 @@ $dumpForSong = new PicoDatabaseDump();
echo $dumpForSong->dumpStructure($song, PicoDatabaseType::DATABASE_TYPE_MYSQL, true, true);
```
+**Parameters:**
+
+- **$song:** An instance of the Song class, representing the table structure you want to dump.
+- **PicoDatabaseType::DATABASE_TYPE_MYSQL:** The type of database you are targeting (e.g., MySQL, PostgreSQL).
+- **true (createIfNotExists):** Whether to include the "IF NOT EXISTS" clause in the CREATE statement.
+- **true (dropIfExists):** Whether to include the "DROP TABLE IF EXISTS" statement before creating the table.
+
### Dump Data
We can dump data by connecting to real database. Don't forget to define the target database type. If we will dump multiple table, we must use dedicated instance of `PicoDatabaseDump`.
@@ -488,6 +495,39 @@ $dumpForSong = new PicoDatabaseDump();
echo $dumpForSong->dumpData($pageData, PicoDatabaseType::DATABASE_TYPE_MYSQL);
```
+**Important Note:**
+
+When exporting a table with a large amount of data, the above method may not be suitable, as it can consume a significant amount of memory while trying to accommodate all the data before writing it to a file.
+
+To efficiently handle large datasets, you can use the following approach:
+
+```php
+$song = new Song(null, $database);
+/*
+$speficication = null
+$pageable = null
+$sortable = null
+$passive = true
+$subqueryMap = null
+$findOption = MagicObject::FIND_OPTION_NO_COUNT_DATA | MagicObject::FIND_OPTION_NO_FETCH_DATA
+*/
+$pageData = $song->findAll(null, null, null, true, null, MagicObject::FIND_OPTION_NO_COUNT_DATA | MagicObject::FIND_OPTION_NO_FETCH_DATA);
+$dumpForSong = new PicoDatabaseDump();
+
+$dumpForSong->dumpData($pageData, PicoDatabaseType::DATABASE_TYPE_MYSQL, new Song(), $maxRecord, function($sql){
+ $fp = fopen("dump.sql", "a");
+ fputs($fp, $sql);
+ fclose($fp);
+});
+```
+
+**Explanation:**
+
+- **findAll Parameters:** This allows you to customize your query. The options provided (e.g., FIND_OPTION_NO_COUNT_DATA, FIND_OPTION_NO_FETCH_DATA) ensure that only the necessary data is fetched, thus reducing memory consumption.
+- **Callback Function:** The anonymous function passed as a parameter to dumpData handles the SQL output, appending it to dump.sql in an efficient manner, thereby avoiding excessive memory usage.
+
+By following these guidelines, you can effectively manage both the structure and data dumping processes while optimizing for performance and resource utilization.
+
### Summary
- **Structure Dumping**: Use `dumpStructure` to get the schema of the database without a real connection.
diff --git a/manual/includes/_magic-dto.md b/manual/includes/_magic-dto.md
new file mode 100644
index 0000000..e48d276
--- /dev/null
+++ b/manual/includes/_magic-dto.md
@@ -0,0 +1,414 @@
+
+## MagicDto
+
+### Introduction to DTOs
+
+A Data Transfer Object (DTO) is a design pattern used to transfer data between software application subsystems or layers. DTOs encapsulate data, reducing the number of method calls needed to retrieve or send information. JSON (JavaScript Object Notation) has become the standard for data serialization due to its simplicity and ease of integration with various programming languages.
+
+The properties defined in MagicDto adhere strictly to the specifications set forth by the developer, ensuring a well-defined structure. This means that users are prohibited from adding any input or output that falls outside the established DTO framework. As a result, the integrity of the data is maintained, and the application remains predictable and reliable, as any deviation from the predefined structure is not permitted. This strict adherence to the DTO structure promotes clarity and consistency, facilitating better communication between different layers of the application while reducing the risk of errors or unintended behavior.
+
+### The Need for MagicDto
+
+In modern applications, especially those that interact with third-party services, maintaining consistent data formats can be challenging. Different systems may use varying naming conventions, such as camel case (`myProperty`) and snake case (`my_property`). Additionally, inconsistencies can occur with uppercase and lowercase letters, leading to potential mismatches when exchanging data.
+
+**MagicDto** addresses these issues by allowing developers to create DTOs that seamlessly translate property names between different naming conventions. This ensures that data is properly formatted for both internal and external use, enhancing interoperability and reducing errors.
+
+### Features of MagicDto
+
+1. **Flexible Naming Strategies**:
+
+ - MagicDto supports both camel case and snake case naming strategies. This flexibility is particularly useful when integrating with diverse APIs or legacy systems that may employ different conventions.
+
+2. **Automatic Property Mapping**:
+
+ - Users can define DTOs that automatically map properties from their internal representation to the expected format of third-party services. This reduces boilerplate code and simplifies maintenance.
+
+3. **Annotations for Clarity**:
+
+ - The MagicDto class utilizes PHP annotations to clarify the purpose of each property. These annotations enhance code readability and provide useful metadata for serialization.
+
+### Class Structure
+
+The `MagicDto` class is designed with properties that have protected access levels, ensuring encapsulation while still allowing derived classes to access these properties. Each property is annotated with `@var`, which specifies its data type. This structured approach enhances type safety and improves code quality.
+
+#### Key Annotations
+
+1. **@Source**
+
+ The `@Source` annotation indicates the source property that maps to a specific field in the incoming data. If this annotation is omitted, MagicDto will default to using the property name that matches the class property name. This allows for flexibility in cases where the external API may use different naming conventions.
+
+```php
+/**
+ * @Source("album_name")
+ * @var string
+ */
+protected $title;
+```
+
+2. **@JsonProperty**
+
+ The `@JsonProperty` annotation specifies the output property name when data is serialized to JSON. If this annotation is not provided, MagicDto will serialize the property using its class property name. This ensures that data sent to third-party applications adheres to their expected format.
+
+```php
+/**
+ * @JsonProperty("album_title")
+ * @var string
+ */
+protected $title;
+```
+
+We can put it together
+
+```php
+/**
+ * @Source("album_name")
+ * @JsonProperty("album_title")
+ * @var string
+ */
+protected $title;
+```
+
+In this example, `@Source("album_name")` indicates that the incoming data will use `album_name`, while `@JsonProperty("album_title")` specifies that when the data is serialized, it will be output as `album_title`.
+
+To facilitate bidirectional communication, we need two different DTOs. The `@Source` annotation in the first DTO corresponds to the `@JsonProperty` annotation in the second DTO, while the `@JsonProperty` in the first DTO maps to the `@Source` in the second DTO.
+
+**Example:**
+
+DTO on the Input Side
+
+```php
+class AlbumDtoInput extends MagicDto
+{
+ /**
+ * @Source("album_id")
+ * @JsonProperty("albumId")
+ * @var string
+ */
+ protected $id;
+
+ /**
+ * @Source("album_name")
+ * @JsonProperty("albumTitle")
+ * @var string
+ */
+ protected $title;
+
+ /**
+ * @Source("date_release")
+ * @JsonProperty("releaseDate")
+ * @var string
+ */
+ protected $release;
+
+ /**
+ * @Source("song")
+ * @JsonProperty("numberOfSong")
+ * @var string
+ */
+ protected $songs;
+}
+```
+
+DTO on the Output Side
+
+```php
+class AlbumDtoOutput extends MagicDto
+{
+ /**
+ * @Source("albumId")
+ * @JsonProperty("album_id")
+ * @var string
+ */
+ protected $id;
+
+ /**
+ * @Source("albumTitle")
+ * @JsonProperty("album_name")
+ * @var string
+ */
+ protected $title;
+
+ /**
+ * @Source("releaseDate")
+ * @JsonProperty("date_release")
+ * @var string
+ */
+ protected $release;
+
+ /**
+ * @Source("numberOfSong")
+ * @JsonProperty("song")
+ * @var string
+ */
+ protected $songs;
+}
+```
+
+**Description**
+
+In this example, we have two DTO classes: AlbumDtoInput and AlbumDtoOutput. The AlbumDtoInput class is designed to receive data from external sources, using the @Source annotation to specify the incoming property names and the @JsonProperty annotation to define the corresponding properties in the internal representation.
+
+Conversely, the AlbumDtoOutput class is structured for sending data outwards. Here, the @Source annotation reflects the internal property names, while the @JsonProperty annotation defines the expected property names when the data is serialized for external use. This bidirectional mapping ensures that data flows seamlessly between internal and external systems.
+
+The `@Source` annotation allows a Data Transfer Object (DTO) to inherit properties from an underlying object, enabling seamless data integration across related entities.
+
+### Cross Object Mapping
+
+
+#### Cross Object Mapping Explanation
+
+1. **Concept Clarification**:
+
+ - Cross Object Mapping refers to the ability to access and utilize properties from related objects in a hierarchical structure. In your case, the `SongDto` pulls in the agency name associated with the artist of a song.
+2. **DTO Definition**:
+
+ - A DTO is a simple object that carries data between processes. In this context, `SongDto` aggregates data from the `Song`, `Artist`, and `Agency` models without duplicating properties unnecessarily.
+
+For example, we want to directly include properties from the agency within the SongDto.
+
+- **Song**
+ - **Artist**
+ - **Agency**
+
+When creating a DTO for a `Song`, the user can incorporate properties from the associated `Agency` into the `SongDto`. This is particularly useful for aggregating data from related models without needing to replicate information.
+
+#### Code Implementation
+
+**Song**
+
+```php
+class Song extends MagicObject {
+ /**
+ * Song ID
+ *
+ * @Column(name="song_id")
+ * @var string
+ */
+ protected $songId;
+
+ /**
+ * Name
+ *
+ * @Column(name="name")
+ * @var string
+ */
+ protected $name;
+
+ /**
+ * Artist
+ *
+ * @JoinColumn(name="artist_id")
+ * @var Artist
+ */
+ protected $artist;
+
+ // Additional properties and methods for the Song can be defined here.
+}
+```
+
+**Artist**
+
+```php
+class Artist extends MagicObject {
+ /**
+ * Artist ID
+ *
+ * @Column(name="artist_id")
+ * @var string
+ */
+ protected $artistId;
+
+ /**
+ * Name
+ *
+ * @Column(name="name")
+ * @var string
+ */
+ protected $name;
+
+ /**
+ * Agency
+ *
+ * @JoinColumn(name="agency_id")
+ * @var Agency
+ */
+ protected $agency;
+
+ // Additional properties and methods for the Artist can be defined here.
+}
+```
+
+**Agency**
+
+```php
+class Agency extends MagicObject {
+ /**
+ * Agency ID
+ *
+ * @Column(name="agency_id")
+ * @var string
+ */
+ protected $agencyId;
+
+ /**
+ * Name
+ *
+ * @Column(name="name")
+ * @var string
+ */
+ protected $name;
+
+ // Additional properties and methods for the Agency can be defined here.
+}
+```
+
+**SongDto**
+
+```php
+class SongDto extends MagicDto
+{
+ /**
+ * Song ID
+ *
+ * @Source("songId")
+ * @JsonProperty(name="song_id")
+ * @var string
+ */
+ protected $songId;
+
+ /**
+ * Title
+ *
+ * @Source("title")
+ * @JsonProperty("title")
+ * @var string
+ */
+ protected $title;
+
+ /**
+ * Artist
+ *
+ * @Source("artist")
+ * @JsonProperty("artist")
+ * @var ArtistDto
+ */
+ protected $artist;
+
+ /**
+ * The name of the agency associated with the artist.
+ *
+ * This property is sourced from the agency related to the artist of the song.
+ *
+ * @Source("artist->agency->name")
+ * @JsonProperty("agency_name")
+ * @var string
+ */
+ protected $agencyName;
+
+ // Additional properties and methods for the SongDto can be defined here.
+}
+```
+
+**ArtistDto**
+
+```php
+class ArtistDto extends MagicDto
+{
+ /**
+ * Artist ID
+ *
+ * @Source("artistId")
+ * @JsonProperty(name="artist_id")
+ * @var string
+ */
+ protected $artistId;
+
+ /**
+ * Name
+ *
+ * @Source("name")
+ * @JsonProperty("name")
+ * @var string
+ */
+ protected $name;
+
+ /**
+ * Agency
+ *
+ * @Source("agency")
+ * @JsonProperty("agency")
+ * @var AgencyDto
+ */
+ protected $agency;
+
+ /**
+ * The name of the agency associated with the artist.
+ *
+ * This property is sourced from the agency related to the artist of the song.
+ *
+ * @Source("artist->agency->name")
+ * @JsonProperty("agency_name")
+ * @var string
+ */
+ protected $agencyName;
+
+ // Additional properties and methods for the SongDto can be defined here.
+}
+```
+
+**AgencyDto**
+
+```php
+class AgencyDto extends MagicDto
+{
+ /**
+ * Agency ID
+ *
+ * @Source("agencyId")
+ * @JsonProperty(name="agency_id")
+ * @var string
+ */
+ protected $agencyId;
+
+ /**
+ * Name
+ *
+ * @Source("name")
+ * @JsonProperty("name")
+ * @var string
+ */
+ protected $name;
+}
+
+```
+
+**Usage**
+
+```php
+$song = new Song(null, $database);
+$song->find("1234");
+$songDto = new SongDto($song);
+
+header("Content-type: application/json");
+echo $songDto;
+```
+
+#### Explanation
+
+- **@Source**: This annotation specifies the path to the property within the nested object structure. In this case, `artist->agency->name` indicates that the `agencyName` will pull data from the `name` property of the `Agency` object linked to the `Artist`.
+
+- **@JsonProperty**: This annotation maps the `agencyName` property to a different key in the JSON representation of the DTO. Here, it will be serialized as `agency_name`.
+
+- **protected $agencyName**: This declares the `agencyName` property with protected visibility, ensuring that it can only be accessed within the class itself and by subclasses.
+
+This approach enhances data encapsulation and promotes cleaner code by allowing DTOs to automatically gather necessary data from related entities.
+
+### Benefits of Using MagicDto
+
+- **Reduced Complexity**: By automating property translation, MagicDto minimizes the need for manual mapping code, reducing complexity and potential errors.
+- **Improved Maintainability**: With clearly defined annotations and a structured approach, developers can easily understand and maintain the DTOs, even as systems evolve.
+- **Enhanced Interoperability**: MagicDto ensures that data exchanged between different systems is consistent and correctly formatted, leading to smoother integrations and fewer runtime issues.
+
+### Conclusion
+
+MagicDto is a powerful solution for managing data transfer in applications that need to communicate with external systems. By leveraging flexible naming strategies and clear annotations, it simplifies the process of creating and maintaining DTOs, ensuring seamless data exchange. Whether you’re building a new application or integrating with legacy systems, MagicDto can help you navigate the complexities of data serialization and improve overall application reliability.
diff --git a/manual/index.html b/manual/index.html
index 8c92f94..a49d713 100644
--- a/manual/index.html
+++ b/manual/index.html
@@ -1326,6 +1326,378 @@
Secure Config from DynamicObject
+MagicDto
+Introduction to DTOs
+A Data Transfer Object (DTO) is a design pattern used to transfer data between software application subsystems or layers. DTOs encapsulate data, reducing the number of method calls needed to retrieve or send information. JSON (JavaScript Object Notation) has become the standard for data serialization due to its simplicity and ease of integration with various programming languages.
+The properties defined in MagicDto adhere strictly to the specifications set forth by the developer, ensuring a well-defined structure. This means that users are prohibited from adding any input or output that falls outside the established DTO framework. As a result, the integrity of the data is maintained, and the application remains predictable and reliable, as any deviation from the predefined structure is not permitted. This strict adherence to the DTO structure promotes clarity and consistency, facilitating better communication between different layers of the application while reducing the risk of errors or unintended behavior.
+The Need for MagicDto
+In modern applications, especially those that interact with third-party services, maintaining consistent data formats can be challenging. Different systems may use varying naming conventions, such as camel case (myProperty
) and snake case (my_property
). Additionally, inconsistencies can occur with uppercase and lowercase letters, leading to potential mismatches when exchanging data.
+MagicDto addresses these issues by allowing developers to create DTOs that seamlessly translate property names between different naming conventions. This ensures that data is properly formatted for both internal and external use, enhancing interoperability and reducing errors.
+Features of MagicDto
+
+-
+
Flexible Naming Strategies:
+
+- MagicDto supports both camel case and snake case naming strategies. This flexibility is particularly useful when integrating with diverse APIs or legacy systems that may employ different conventions.
+
+
+-
+
Automatic Property Mapping:
+
+- Users can define DTOs that automatically map properties from their internal representation to the expected format of third-party services. This reduces boilerplate code and simplifies maintenance.
+
+
+-
+
Annotations for Clarity:
+
+- The MagicDto class utilizes PHP annotations to clarify the purpose of each property. These annotations enhance code readability and provide useful metadata for serialization.
+
+
+
+Class Structure
+The MagicDto
class is designed with properties that have protected access levels, ensuring encapsulation while still allowing derived classes to access these properties. Each property is annotated with @var
, which specifies its data type. This structured approach enhances type safety and improves code quality.
+Key Annotations
+
+-
+
@Source
+The @Source
annotation indicates the source property that maps to a specific field in the incoming data. If this annotation is omitted, MagicDto will default to using the property name that matches the class property name. This allows for flexibility in cases where the external API may use different naming conventions.
+
+
+/**
+ * @Source("album_name")
+ * @var string
+ */
+protected $title;
+
+-
+
@JsonProperty
+The @JsonProperty
annotation specifies the output property name when data is serialized to JSON. If this annotation is not provided, MagicDto will serialize the property using its class property name. This ensures that data sent to third-party applications adheres to their expected format.
+
+
+/**
+ * @JsonProperty("album_title")
+ * @var string
+ */
+protected $title;
+We can put it together
+/**
+ * @Source("album_name")
+ * @JsonProperty("album_title")
+ * @var string
+ */
+protected $title;
+In this example, @Source("album_name")
indicates that the incoming data will use album_name
, while @JsonProperty("album_title")
specifies that when the data is serialized, it will be output as album_title
.
+To facilitate bidirectional communication, we need two different DTOs. The @Source
annotation in the first DTO corresponds to the @JsonProperty
annotation in the second DTO, while the @JsonProperty
in the first DTO maps to the @Source
in the second DTO.
+Example:
+DTO on the Input Side
+class AlbumDtoInput extends MagicDto
+{
+ /**
+ * @Source("album_id")
+ * @JsonProperty("albumId")
+ * @var string
+ */
+ protected $id;
+
+ /**
+ * @Source("album_name")
+ * @JsonProperty("albumTitle")
+ * @var string
+ */
+ protected $title;
+
+ /**
+ * @Source("date_release")
+ * @JsonProperty("releaseDate")
+ * @var string
+ */
+ protected $release;
+
+ /**
+ * @Source("song")
+ * @JsonProperty("numberOfSong")
+ * @var string
+ */
+ protected $songs;
+}
+DTO on the Output Side
+class AlbumDtoOutput extends MagicDto
+{
+ /**
+ * @Source("albumId")
+ * @JsonProperty("album_id")
+ * @var string
+ */
+ protected $id;
+
+ /**
+ * @Source("albumTitle")
+ * @JsonProperty("album_name")
+ * @var string
+ */
+ protected $title;
+
+ /**
+ * @Source("releaseDate")
+ * @JsonProperty("date_release")
+ * @var string
+ */
+ protected $release;
+
+ /**
+ * @Source("numberOfSong")
+ * @JsonProperty("song")
+ * @var string
+ */
+ protected $songs;
+}
+Description
+In this example, we have two DTO classes: AlbumDtoInput and AlbumDtoOutput. The AlbumDtoInput class is designed to receive data from external sources, using the @Source annotation to specify the incoming property names and the @JsonProperty annotation to define the corresponding properties in the internal representation.
+Conversely, the AlbumDtoOutput class is structured for sending data outwards. Here, the @Source annotation reflects the internal property names, while the @JsonProperty annotation defines the expected property names when the data is serialized for external use. This bidirectional mapping ensures that data flows seamlessly between internal and external systems.
+The @Source
annotation allows a Data Transfer Object (DTO) to inherit properties from an underlying object, enabling seamless data integration across related entities.
+Cross Object Mapping
+Cross Object Mapping Explanation
+
+-
+
Concept Clarification:
+
+- Cross Object Mapping refers to the ability to access and utilize properties from related objects in a hierarchical structure. In your case, the
SongDto
pulls in the agency name associated with the artist of a song.
+
+
+-
+
DTO Definition:
+
+- A DTO is a simple object that carries data between processes. In this context,
SongDto
aggregates data from the Song
, Artist
, and Agency
models without duplicating properties unnecessarily.
+
+
+
+For example, we want to directly include properties from the agency within the SongDto.
+
+When creating a DTO for a Song
, the user can incorporate properties from the associated Agency
into the SongDto
. This is particularly useful for aggregating data from related models without needing to replicate information.
+Code Implementation
+Song
+class Song extends MagicObject {
+ /**
+ * Song ID
+ *
+ * @Column(name="song_id")
+ * @var string
+ */
+ protected $songId;
+
+ /**
+ * Name
+ *
+ * @Column(name="name")
+ * @var string
+ */
+ protected $name;
+
+ /**
+ * Artist
+ *
+ * @JoinColumn(name="artist_id")
+ * @var Artist
+ */
+ protected $artist;
+
+ // Additional properties and methods for the Song can be defined here.
+}
+Artist
+class Artist extends MagicObject {
+ /**
+ * Artist ID
+ *
+ * @Column(name="artist_id")
+ * @var string
+ */
+ protected $artistId;
+
+ /**
+ * Name
+ *
+ * @Column(name="name")
+ * @var string
+ */
+ protected $name;
+
+ /**
+ * Agency
+ *
+ * @JoinColumn(name="agency_id")
+ * @var Agency
+ */
+ protected $agency;
+
+ // Additional properties and methods for the Artist can be defined here.
+}
+Agency
+class Agency extends MagicObject {
+ /**
+ * Agency ID
+ *
+ * @Column(name="agency_id")
+ * @var string
+ */
+ protected $agencyId;
+
+ /**
+ * Name
+ *
+ * @Column(name="name")
+ * @var string
+ */
+ protected $name;
+
+ // Additional properties and methods for the Agency can be defined here.
+}
+SongDto
+class SongDto extends MagicDto
+{
+ /**
+ * Song ID
+ *
+ * @Source("songId")
+ * @JsonProperty(name="song_id")
+ * @var string
+ */
+ protected $songId;
+
+ /**
+ * Title
+ *
+ * @Source("title")
+ * @JsonProperty("title")
+ * @var string
+ */
+ protected $title;
+
+ /**
+ * Artist
+ *
+ * @Source("artist")
+ * @JsonProperty("artist")
+ * @var ArtistDto
+ */
+ protected $artist;
+
+ /**
+ * The name of the agency associated with the artist.
+ *
+ * This property is sourced from the agency related to the artist of the song.
+ *
+ * @Source("artist->agency->name")
+ * @JsonProperty("agency_name")
+ * @var string
+ */
+ protected $agencyName;
+
+ // Additional properties and methods for the SongDto can be defined here.
+}
+ArtistDto
+class ArtistDto extends MagicDto
+{
+ /**
+ * Artist ID
+ *
+ * @Source("artistId")
+ * @JsonProperty(name="artist_id")
+ * @var string
+ */
+ protected $artistId;
+
+ /**
+ * Name
+ *
+ * @Source("name")
+ * @JsonProperty("name")
+ * @var string
+ */
+ protected $name;
+
+ /**
+ * Agency
+ *
+ * @Source("agency")
+ * @JsonProperty("agency")
+ * @var AgencyDto
+ */
+ protected $agency;
+
+ /**
+ * The name of the agency associated with the artist.
+ *
+ * This property is sourced from the agency related to the artist of the song.
+ *
+ * @Source("artist->agency->name")
+ * @JsonProperty("agency_name")
+ * @var string
+ */
+ protected $agencyName;
+
+ // Additional properties and methods for the SongDto can be defined here.
+}
+AgencyDto
+class AgencyDto extends MagicDto
+{
+ /**
+ * Agency ID
+ *
+ * @Source("agencyId")
+ * @JsonProperty(name="agency_id")
+ * @var string
+ */
+ protected $agencyId;
+
+ /**
+ * Name
+ *
+ * @Source("name")
+ * @JsonProperty("name")
+ * @var string
+ */
+ protected $name;
+}
+
+Usage
+$song = new Song(null, $database);
+$song->find("1234");
+$songDto = new SongDto($song);
+
+header("Content-type: application/json");
+echo $songDto;
+Explanation
+
+-
+
@Source: This annotation specifies the path to the property within the nested object structure. In this case, artist->agency->name
indicates that the agencyName
will pull data from the name
property of the Agency
object linked to the Artist
.
+
+-
+
@JsonProperty: This annotation maps the agencyName
property to a different key in the JSON representation of the DTO. Here, it will be serialized as agency_name
.
+
+-
+
protected $agencyName: This declares the agencyName
property with protected visibility, ensuring that it can only be accessed within the class itself and by subclasses.
+
+
+This approach enhances data encapsulation and promotes cleaner code by allowing DTOs to automatically gather necessary data from related entities.
+Benefits of Using MagicDto
+
+- Reduced Complexity: By automating property translation, MagicDto minimizes the need for manual mapping code, reducing complexity and potential errors.
+- Improved Maintainability: With clearly defined annotations and a structured approach, developers can easily understand and maintain the DTOs, even as systems evolve.
+- Enhanced Interoperability: MagicDto ensures that data exchanged between different systems is consistent and correctly formatted, leading to smoother integrations and fewer runtime issues.
+
+Conclusion
+MagicDto is a powerful solution for managing data transfer in applications that need to communicate with external systems. By leveraging flexible naming strategies and clear annotations, it simplifies the process of creating and maintaining DTOs, ensuring seamless data exchange. Whether you’re building a new application or integrating with legacy systems, MagicDto can help you navigate the complexities of data serialization and improve overall application reliability.
+
+
+
Input POST/GET/COOKIE/REQUEST/SERVER
In PHP, handling user input can be done through various superglobals, such as $_POST, $_GET, $_COOKIE, $_REQUEST, and $_SERVER. Each of these superglobals serves a specific purpose for gathering data from different types of requests.
Input POST
@@ -1452,7 +1824,7 @@ Conclusion
In summary, handling input in PHP through superglobals is straightforward but requires careful filtering to ensure security. Using classes like InputPost
, InputGet
, InputCookie
, InputRequest
, and InputServer
can abstract the underlying superglobal accesses, making the code cleaner and potentially more secure by enforcing consistent input handling and sanitization practices.
-
+
Session
Session variables keep information about one single user, and are available to all pages in one application.
Session with File
@@ -1553,7 +1925,7 @@ Conclusion
This implementation provides a robust framework for session management in a PHP application, allowing flexibility in storage options (files or Redis) while emphasizing security through encryption. The use of YAML for configuration keeps the setup clean and easily adjustable. By encapsulating session configuration in dedicated classes, you enhance maintainability and security.
-
+
Entity
Entity is class to access database. Entity is derived from MagicObject. Some annotations required to activated all entity features.
Constructor
@@ -6041,7 +6413,7 @@ Method
}
-
+
Specification
Specifications are implemented in the PicoSpecification and PicoPredicate classes. PicoSpecification is a framework that can contain one or more PicoPredicate.
For example, we have the following query:
@@ -6534,7 +6906,7 @@ Specification
}
-
+
Pageable and Sortable
In MagicObject, pageable is used to divide data rows into several pages. This is required by the application to display a lot of data per page. While sortable is used to sort data before the data is divided per page.
Pageable can stand alone without sortable. However, this method is not recommended because the data sequence is not as expected. If new data is entered, users will have difficulty finding where it is located in the list and on which page the data will appear. The solution is to add a sortable that will sort the data based on certain columns. For example, the time of data creation is descending, then the new data will be on the first page. Conversely, if sorted based on the time of data creation is ascending, then the new data will be on the last page.
@@ -6826,7 +7198,7 @@ Pageable and Sortable
ORDER BY user_name ASC, email DESC, phone ASC LIMIT 200 OFFSET 400
-
+
Filtering, Ordering and Pagination
MagicObject will filter data according to the given criteria. On the other hand, MagicObject will only retrieve data on the specified page by specifying limit
and offset
data in the select
query.
Example parameters:
@@ -7709,7 +8081,7 @@ Filtering, Ordering and Pagination
?>
-
+
Native Query
In MagicObject version 2, native queries have been introduced as an efficient way to interact with the database.
Native queries offer significant performance improvements when handling large volumes of data, allowing users to craft highly efficient queries that meet diverse requirements.
@@ -8339,7 +8711,7 @@ Best Practices
By leveraging the native query feature in MagicObject, you can create efficient and maintainable database interactions, enhancing your application's performance and security.
-
+
Multiple Database Connections
MagicObject version 2 introduces support for multiple database connections, enabling users to manage entities stored across different databases seamlessly. When performing operations such as JOINs with entities from multiple databases, it is essential to define a database connection for each entity involved.
Example Scenario
@@ -8504,7 +8876,7 @@ Conclusion
With MagicObject version 2, managing entities across multiple database connections is straightforward. By defining the correct associations and utilizing the provided methods, users can effectively work with complex data structures that span multiple databases. Make sure to handle exceptions properly to ensure robustness in your application.
-
+
Dump Database
We can dump database to another database type. We do not need any database converter. Just define the target database type when we dump the database.
Database Dump Overview
@@ -8971,12 +9343,45 @@ Dump Structure
$song = new Song();
$dumpForSong = new PicoDatabaseDump();
echo $dumpForSong->dumpStructure($song, PicoDatabaseType::DATABASE_TYPE_MYSQL, true, true);
+Parameters:
+
+- $song: An instance of the Song class, representing the table structure you want to dump.
+- PicoDatabaseType::DATABASE_TYPE_MYSQL: The type of database you are targeting (e.g., MySQL, PostgreSQL).
+- true (createIfNotExists): Whether to include the "IF NOT EXISTS" clause in the CREATE statement.
+- true (dropIfExists): Whether to include the "DROP TABLE IF EXISTS" statement before creating the table.
+
Dump Data
We can dump data by connecting to real database. Don't forget to define the target database type. If we will dump multiple table, we must use dedicated instance of PicoDatabaseDump
.
$song = new Song(null, $database);
$pageData = $song->findAll();
$dumpForSong = new PicoDatabaseDump();
echo $dumpForSong->dumpData($pageData, PicoDatabaseType::DATABASE_TYPE_MYSQL);
+Important Note:
+When exporting a table with a large amount of data, the above method may not be suitable, as it can consume a significant amount of memory while trying to accommodate all the data before writing it to a file.
+To efficiently handle large datasets, you can use the following approach:
+$song = new Song(null, $database);
+/*
+$speficication = null
+$pageable = null
+$sortable = null
+$passive = true
+$subqueryMap = null
+$findOption = MagicObject::FIND_OPTION_NO_COUNT_DATA | MagicObject::FIND_OPTION_NO_FETCH_DATA
+*/
+$pageData = $song->findAll(null, null, null, true, null, MagicObject::FIND_OPTION_NO_COUNT_DATA | MagicObject::FIND_OPTION_NO_FETCH_DATA);
+$dumpForSong = new PicoDatabaseDump();
+
+$dumpForSong->dumpData($pageData, PicoDatabaseType::DATABASE_TYPE_MYSQL, new Song(), $maxRecord, function($sql){
+ $fp = fopen("dump.sql", "a");
+ fputs($fp, $sql);
+ fclose($fp);
+});
+Explanation:
+
+- findAll Parameters: This allows you to customize your query. The options provided (e.g., FIND_OPTION_NO_COUNT_DATA, FIND_OPTION_NO_FETCH_DATA) ensure that only the necessary data is fetched, thus reducing memory consumption.
+- Callback Function: The anonymous function passed as a parameter to dumpData handles the SQL output, appending it to dump.sql in an efficient manner, thereby avoiding excessive memory usage.
+
+By following these guidelines, you can effectively manage both the structure and data dumping processes while optimizing for performance and resource utilization.
Summary
- Structure Dumping: Use
dumpStructure
to get the schema of the database without a real connection.
@@ -8985,7 +9390,7 @@ Summary
This approach allows developers to quickly switch between database types and manage their database schemas and data efficiently. The use of dedicated instances of PicoDatabaseDump for multiple tables ensures clarity and organization in your database operations.
-
+
Object Label
<?php
@@ -9110,7 +9515,7 @@ Object Label
// it will print "Admin Create"
-
+
Database Query Builder
Database Query Builder is a feature for creating object-based database queries. The output of the Database Query Builder is a query that can be directly executed by the database used.
Database Query Builder is actually designed for all relational databases but is currently only available in two languages, namely MySQL and PostgreSQL. MagicObject internally uses the Database Query Builder to create queries based on given methods and parameters.
@@ -9377,7 +9782,7 @@ Methods
This way, $active
will be escaped before being executed by the database. You don't need to escape it first.
-
+
Upload File
Uploading lots of files with arrays is difficult for some developers, especially novice developers. There is a significant difference between uploading a single file and multiple files.
When the developer decides to change the form from single file to multiple files or vice versa, the backend developer must change the code to handle the uploaded files.
@@ -9437,7 +9842,7 @@ Summary
This implementation offers a straightforward way to manage file uploads in PHP, abstracting complexities for developers. By using methods like getAll() and isMultiple(), developers can seamlessly handle both types of uploads without needing to write separate logic for each scenario. This approach not only improves code maintainability but also enhances the developer experience.
-
+
Language
MagicObject supports multilingual applications. MagicObject allows developers to create entities that support a wide variety of languages that users can choose from. At the same time, different users can use different languages.
To create table with multiple language, create new class from DataTable
object. We can copy data from aother object to DataTable
easly.
@@ -9657,7 +10062,7 @@ Language
echo $apa;
-
+
Database Migration
MagicObject allows users to import data from a database with different table names and column names between the source database and the destination database. This feature is used by developers who develop applications that are already used in production environments.
On the one hand, the application requires a new database structure according to what is defined by the developer. On the other hand, users want to use existing data.
diff --git a/manual/index.html.md b/manual/index.html.md
index c8babc2..58ead7c 100644
--- a/manual/index.html.md
+++ b/manual/index.html.md
@@ -22,6 +22,7 @@ includes:
- object-from-ini
- environment-variable
- secret-object
+ - magic-dto
- input
- session
- entity
diff --git a/src/Database/PicoDatabaseQueryBuilder.php b/src/Database/PicoDatabaseQueryBuilder.php
index 4d26fb9..b1bdc3b 100644
--- a/src/Database/PicoDatabaseQueryBuilder.php
+++ b/src/Database/PicoDatabaseQueryBuilder.php
@@ -2,8 +2,12 @@
namespace MagicObject\Database;
/**
- * Query builder
+ * Class PicoDatabaseQueryBuilder
*
+ * A query builder for constructing SQL statements programmatically. This class
+ * facilitates the creation of various SQL commands including SELECT, INSERT,
+ * UPDATE, and DELETE, while managing database-specific nuances.
+ *
* @author Kamshory
* @package MagicObject\Database
* @link https://github.com/Planetbiru/MagicObject
diff --git a/src/MagicDto.php b/src/MagicDto.php
new file mode 100644
index 0000000..dab24cd
--- /dev/null
+++ b/src/MagicDto.php
@@ -0,0 +1,510 @@
+loadData($data);
+ $jsonAnnot = new PicoAnnotationParser(get_class($this));
+ $params = $jsonAnnot->getParameters();
+ foreach($params as $paramName=>$paramValue)
+ {
+ try
+ {
+ $vals = $jsonAnnot->parseKeyValue($paramValue);
+ $this->_classParams[$paramName] = $vals;
+ }
+ catch(InvalidQueryInputException $e)
+ {
+ throw new InvalidAnnotationException("Invalid annotation @".$paramName);
+ }
+ }
+ }
+
+ /**
+ * Loads data into the object.
+ *
+ * This method accepts various data types, including:
+ * - An instance of the class itself
+ * - An array
+ * - A standard object (stdClass)
+ * - Other specific object types such as MagicObject, SetterGetter,
+ * SecretObject, and PicoGenericObject.
+ *
+ * The method processes the input data and stores it in the internal
+ * data source of the object, ensuring that only non-scalar values are
+ * handled.
+ *
+ * @param self|array|stdClass|MagicObject|SetterGetter|SecretObject|PicoGenericObject|null $data
+ * The data to load, which can be one of the specified types
+ * or null.
+ * @return self Returns the current instance for method chaining.
+ */
+ public function loadData($data)
+ {
+ if (isset($data)) {
+ // Check if data is not a scalar value
+ if (is_object($data) || is_array($data)) {
+ // Check if the data is one of the allowed object types
+ if ($data instanceof self || $data instanceof MagicObject ||
+ $data instanceof SetterGetter || $data instanceof SecretObject ||
+ $data instanceof PicoGenericObject) {
+ // Directly assign the data source if it is an allowed object type
+ $this->_dataSource = $data;
+ } else {
+ // Parse the object or array recursively
+ $this->_dataSource = PicoObjectParser::parseRecursiveObject($data);
+ }
+ }
+ }
+ return $this;
+ }
+
+ /**
+ * Get the object values
+ *
+ * @return stdClass An object containing the values of the properties
+ */
+ public function value()
+ {
+ $parentProps = $this->propertyList(true, true);
+ $returnValue = new stdClass;
+
+ foreach ($this as $key => $val) {
+ if (!in_array($key, $parentProps)) {
+ $doc = $this->getPropertyDocComment($key);
+ $source = $this->extractSource($doc);
+ $jsonProperty = $this->extractJsonProperty($doc);
+ $var = $this->extractVar($doc);
+ $propertyName = $jsonProperty ? $jsonProperty : $key;
+
+ $objectTest = class_exists($var) ? new $var() : null;
+
+ if ($this->isSelfInstance($var, $objectTest)) {
+ $returnValue->$propertyName = $this->handleSelfInstance($source, $var, $propertyName);
+ } elseif ($this->isMagicObjectInstance($objectTest)) {
+ $returnValue->$propertyName = $this->handleMagicObject($source, $propertyName);
+ } else {
+ $returnValue->$propertyName = $this->handleDefaultCase($source, $key, $propertyName);
+ }
+ }
+ }
+ return $returnValue;
+ }
+
+ /**
+ * Retrieves the documentation comment for a specified property.
+ *
+ * @param string $key The name of the property.
+ * @return string|null The documentation comment for the property, or null if not found.
+ */
+ private function getPropertyDocComment($key)
+ {
+ $propReflect = new ReflectionProperty($this, $key);
+ return $propReflect->getDocComment();
+ }
+
+ /**
+ * Extracts the source from the documentation comment.
+ *
+ * @param string $doc The documentation comment containing the source.
+ * @return string|null The extracted source or null if not found.
+ */
+ private function extractSource($doc)
+ {
+ preg_match('/@Source\("([^"]+)"\)/', $doc, $matches);
+ return !empty($matches[1]) ? $matches[1] : null;
+ }
+
+ /**
+ * Extracts the JSON property name from the documentation comment.
+ *
+ * @param string $doc The documentation comment containing the JSON property.
+ * @return string|null The extracted JSON property name or null if not found.
+ */
+ private function extractJsonProperty($doc)
+ {
+ preg_match('/@JsonProperty\("([^"]+)"\)/', $doc, $matches);
+ return !empty($matches[1]) ? $matches[1] : null;
+ }
+
+ /**
+ * Extracts the variable type from the documentation comment.
+ *
+ * @param string $doc The documentation comment containing the variable type.
+ * @return string|null The extracted variable type or null if not found.
+ */
+ private function extractVar($doc)
+ {
+ preg_match('/@var\s+(\S+)/', $doc, $matches);
+ return !empty($matches[1]) ? $matches[1] : null;
+ }
+
+ /**
+ * Extracts the label from the documentation comment.
+ *
+ * @param string $doc The documentation comment containing the label.
+ * @return string|null The extracted label or null if not found.
+ */
+ private function extractLabel($doc)
+ {
+ preg_match('/@Label\("([^"]+)"\)/', $doc, $matches);
+ return !empty($matches[1]) ? $matches[1] : null;
+ }
+
+ /**
+ * Checks if the given variable is a self-instance.
+ *
+ * @param string $var The variable name.
+ * @param mixed $objectTest The object to test against.
+ * @return bool True if it's a self-instance, otherwise false.
+ */
+ private function isSelfInstance($var, $objectTest)
+ {
+ return strtolower($var) != 'stdclass' && $objectTest instanceof self;
+ }
+
+ /**
+ * Handles the case where the property is a self-instance.
+ *
+ * @param string|null $source The source to extract the value from.
+ * @param string $var The variable type.
+ * @param string $propertyName The name of the property.
+ * @return mixed The handled value for the self-instance.
+ */
+ private function handleSelfInstance($source, $var, $propertyName)
+ {
+ if (strpos($source, "->") === false) {
+ $value = isset($source) ? $this->_dataSource->get($source) : $this->_dataSource->get($propertyName);
+ $objectValid = new $var($value);
+ return $objectValid->value();
+ } else {
+ return $this->getNestedValue($source);
+ }
+ }
+
+ /**
+ * Checks if the given object is an instance of MagicObject or its derivatives.
+ *
+ * @param mixed $objectTest The object to test.
+ * @return bool True if it is a MagicObject instance, otherwise false.
+ */
+ private function isMagicObjectInstance($objectTest)
+ {
+ return $objectTest instanceof MagicObject ||
+ $objectTest instanceof SetterGetter ||
+ $objectTest instanceof SecretObject ||
+ $objectTest instanceof PicoGenericObject;
+ }
+
+ /**
+ * Handles the case where the property is an instance of MagicObject.
+ *
+ * @param string|null $source The source to extract the value from.
+ * @param string $propertyName The name of the property.
+ * @return mixed The handled value for the MagicObject instance.
+ */
+ private function handleMagicObject($source, $propertyName)
+ {
+ if (strpos($source, "->") === false) {
+ $value = isset($source) ? $this->_dataSource->get($source) : $this->_dataSource->get($propertyName);
+ return ($value instanceof MagicObject || $value instanceof SetterGetter ||
+ $value instanceof SecretObject || $value instanceof PicoGenericObject)
+ ? $value->value()
+ : json_decode(json_encode($value));
+ } else {
+ return $this->getNestedValue($source);
+ }
+ }
+
+ /**
+ * Handles the default case when retrieving property values.
+ *
+ * @param string|null $source The source to extract the value from.
+ * @param string $key The key of the property.
+ * @param string $propertyName The name of the property.
+ * @return mixed The handled default value.
+ */
+ private function handleDefaultCase($source, $key, $propertyName)
+ {
+ if (strpos($source, "->") === false) {
+ return isset($source) ? $this->_dataSource->get($source) : $this->_dataSource->get($key);
+ } else {
+ return $this->getNestedValue($source);
+ }
+ }
+
+ /**
+ * Retrieves nested values from the data source based on a specified source string.
+ *
+ * @param string $source The source string indicating the path to the value.
+ * @return mixed The nested value retrieved from the data source.
+ */
+ private function getNestedValue($source)
+ {
+ $currentVal = null;
+ $arr = explode("->", $source);
+ $fullKey = $arr[0];
+ $currentVal = $this->_dataSource->get($fullKey);
+ for ($i = 1; $i < count($arr); $i++) {
+ if (isset($currentVal) && $currentVal->get($arr[$i]) != null) {
+ $currentVal = $currentVal->get($arr[$i]);
+ } else {
+ break;
+ }
+ }
+ return $currentVal;
+ }
+
+ /**
+ * Get the object value as a specified format
+ *
+ * @return stdClass An object representing the value of the instance
+ */
+ public function valueObject()
+ {
+ $obj = clone $this;
+ foreach($obj as $key=>$value)
+ {
+ if($value instanceof self)
+ {
+ $value = $this->stringifyObject($value);
+ $obj->{$key} = $value;
+ }
+ }
+ return $obj->value();
+ }
+
+ /**
+ * Get the object value as an associative array
+ *
+ * @return array An associative array representing the object values
+ */
+ public function valueArray()
+ {
+ $value = $this->value();
+ return json_decode(json_encode($value), true);
+ }
+
+ /**
+ * Get the object value as an associative array with the first letter of each key in upper camel case
+ *
+ * @return array An associative array with keys in upper camel case
+ */
+ public function valueArrayUpperCamel()
+ {
+ $obj = clone $this;
+ $array = (array) $obj->value();
+ $renameMap = array();
+ $keys = array_keys($array);
+ foreach($keys as $key)
+ {
+ $renameMap[$key] = ucfirst($key);
+ }
+ $array = array_combine(array_map(function($el) use ($renameMap) {
+ return $renameMap[$el];
+ }, array_keys($array)), array_values($array));
+ return $array;
+ }
+
+ /**
+ * Check if the JSON output should be prettified
+ *
+ * @return bool True if JSON output is set to be prettified; otherwise, false
+ */
+ protected function _pretty()
+ {
+ return isset($this->_classParams[self::JSON])
+ && isset($this->_classParams[self::JSON][self::PRETTIFY])
+ && strcasecmp($this->_classParams[self::JSON][self::PRETTIFY], 'true') == 0
+ ;
+ }
+
+ /**
+ * Get a list of properties
+ *
+ * @param bool $reflectSelf Flag indicating whether to reflect properties of the current class
+ * @param bool $asArrayProps Flag indicating whether to return properties as an array
+ * @return array An array of property names or ReflectionProperty objects
+ */
+ protected function propertyList($reflectSelf = false, $asArrayProps = false)
+ {
+ $reflectionClass = $reflectSelf ? self::class : get_called_class();
+ $class = new ReflectionClass($reflectionClass);
+
+ // filter only the calling class properties
+ // skip parent properties
+ $properties = array_filter(
+ $class->getProperties(),
+ function($property) use($class) {
+ return $property->getDeclaringClass()->getName() == $class->getName();
+ }
+ );
+ if($asArrayProps)
+ {
+ $result = array();
+ $index = 0;
+ foreach ($properties as $key) {
+ $prop = $key->name;
+ $result[$index] = $prop;
+
+ $index++;
+ }
+ return $result;
+ }
+ else
+ {
+ return $properties;
+ }
+ }
+
+ /**
+ * Recursively stringify an object or array of objects.
+ *
+ * @param self $value The object to stringify.
+ * @return mixed The stringified object or array.
+ */
+ private function stringifyObject($value)
+ {
+ if(is_array($value))
+ {
+ foreach($value as $key2=>$val2)
+ {
+ if($val2 instanceof self)
+ {
+ $value[$key2] = $val2->stringifyObject($val2);
+ }
+ }
+ }
+ else if(is_object($value))
+ {
+ foreach($value as $key2=>$val2)
+ {
+ if($val2 instanceof self)
+ {
+
+ $value->{$key2} = $val2->stringifyObject($val2);
+ }
+ }
+ }
+ return $value->value();
+ }
+
+ /**
+ * Magic method to convert the object to a JSON string representation.
+ *
+ * This method recursively converts the object's properties into a JSON format.
+ * If any property is an instance of the same class, it will be stringified
+ * as well. The output can be formatted for readability based on the
+ * `_pretty()` method's return value.
+ *
+ * @return string A JSON representation of the object, possibly pretty-printed.
+ */
+ public function __toString()
+ {
+ $pretty = $this->_pretty();
+ $flag = $pretty ? JSON_PRETTY_PRINT : 0;
+ $obj = clone $this;
+ foreach($obj as $key=>$value)
+ {
+ if($value instanceof self)
+ {
+ $value = $this->stringifyObject($value);
+ $obj->{$key} = $value;
+ }
+ }
+ return json_encode($obj->value(), $flag);
+ }
+
+ /**
+ * Convert the object to a string.
+ *
+ * This method returns the string representation of the object by calling
+ * the magic `__toString()` method. It's useful for obtaining the
+ * JSON representation directly as a string.
+ *
+ * @return string The string representation of the object.
+ */
+ public function toString()
+ {
+ return (string) $this;
+ }
+
+ /**
+ * Convert the object to a JSON object.
+ *
+ * This method decodes the JSON string representation of the object
+ * (produced by the `__toString()` method) and returns it as a PHP
+ * object. This is useful for working with the data in a more
+ * structured format rather than as a JSON string.
+ *
+ * @return object|null A PHP object representation of the JSON data, or null if decoding fails.
+ */
+ public function toJson()
+ {
+ return json_decode((string) $this);
+ }
+
+ /**
+ * Convert the object to an associative array.
+ *
+ * This method decodes the JSON string representation of the object
+ * (produced by the `__toString()` method) and returns it as an
+ * associative array. This is useful for accessing the object's
+ * properties in a more straightforward array format.
+ *
+ * @return array|null An associative array representation of the JSON data, or null if decoding fails.
+ */
+ public function toArray()
+ {
+ return json_decode((string) $this, true);
+ }
+}
diff --git a/src/Util/Database/PicoDatabaseUtilBase.php b/src/Util/Database/PicoDatabaseUtilBase.php
new file mode 100644
index 0000000..2f47086
--- /dev/null
+++ b/src/Util/Database/PicoDatabaseUtilBase.php
@@ -0,0 +1,736 @@
+getAutoIncrementKeys();
+ $autoIncrementKeys = array();
+ if(is_array($autoIncrement) && !empty($autoIncrement))
+ {
+ foreach($autoIncrement as $col)
+ {
+ if($col["strategy"] == 'GenerationType.IDENTITY')
+ {
+ $autoIncrementKeys[] = $col["name"];
+ }
+ }
+ }
+ return $autoIncrementKeys;
+ }
+
+ /**
+ * Dumps data from various sources into SQL INSERT statements.
+ *
+ * This method processes data from PicoPageData, MagicObject, or an array of MagicObject instances
+ * and generates SQL INSERT statements. It supports batching of records and allows for a callback
+ * function to handle the generated SQL statements.
+ *
+ * @param array $columns Array of columns for the target table.
+ * @param string $picoTableName Name of the target table.
+ * @param MagicObject|PicoPageData|array $data Data to be dumped. Can be a PicoPageData instance,
+ * a MagicObject instance, or an array of MagicObject instances.
+ * @param int $maxRecord Maximum number of records to process in a single query (default is 100).
+ * @param callable|null $callbackFunction Optional callback function to process the generated SQL
+ * statements. The function should accept a single string parameter
+ * representing the SQL statement.
+ * @return string|null SQL INSERT statements or null if no data was processed.
+ */
+ public function dumpData($columns, $picoTableName, $data, $maxRecord = 100, $callbackFunction = null) //NOSONAR
+ {
+ // Check if $data is an instance of PicoPageData
+ if($data instanceof PicoPageData)
+ {
+ // Handle case where fetching data is not required
+ if($data->getFindOption() & MagicObject::FIND_OPTION_NO_FETCH_DATA && $maxRecord > 0 && isset($callbackFunction) && is_callable($callbackFunction))
+ {
+ $records = array();
+ $stmt = $data->getPDOStatement();
+ // Fetch records in batches
+ while($data = $stmt->fetch(PDO::FETCH_ASSOC, PDO::FETCH_ORI_NEXT))
+ {
+ // Ensure data has all required columns
+ $data = $this->processDataMapping($data, $columns);
+ if(count($records) < $maxRecord)
+ {
+ $records[] = $data;
+ }
+ else
+ {
+ if(isset($callbackFunction) && is_callable($callbackFunction))
+ {
+ // Call the callback function with the generated SQL
+ $sql = $this->insert($picoTableName, $records);
+ call_user_func($callbackFunction, $sql);
+ }
+ // Reset the records buffer
+ $records = array();
+ }
+ }
+ // Handle any remaining records
+ if(!empty($records) && isset($callbackFunction) && is_callable($callbackFunction))
+ {
+ $sql = $this->insert($picoTableName, $records);
+ call_user_func($callbackFunction, $sql);
+ }
+ }
+ else if(isset($data->getResult()[0]))
+ {
+ // If data is available, dump records directly
+ return $this->dumpRecords($columns, $picoTableName, $data->getResult());
+ }
+ }
+ else if($data instanceof MagicObject)
+ {
+ // Handle a single MagicObject instance
+ return $this->dumpRecords($columns, $picoTableName, array($data));
+ }
+ else if(is_array($data) && isset($data[0]) && $data[0] instanceof MagicObject)
+ {
+ // Handle an array of MagicObject instances
+ return $this->dumpRecords($columns, $picoTableName, $data);
+ }
+ return null; // Return null if no valid data was processed
+ }
+
+ /**
+ * Constructs an SQL INSERT statement for a single record.
+ *
+ * This method takes a data record and maps it to the corresponding columns of the target table,
+ * generating an SQL INSERT statement. It uses the PicoDatabaseQueryBuilder to build the query.
+ *
+ * @param array $columns Associative array mapping column names to their definitions in the target table.
+ * @param string $picoTableName Name of the target table where the record will be inserted.
+ * @param MagicObject $record The data record to be dumped into the SQL statement.
+ * @return string The generated SQL INSERT statement.
+ */
+ public function dumpRecords($columns, $picoTableName, $data)
+ {
+ $result = "";
+ foreach($data as $record)
+ {
+ $result .= $this->dumpRecord($columns, $picoTableName, $record).";\r\n";
+ }
+ return $result;
+ }
+
+ /**
+ * Retrieves the maximum record limit for a query.
+ *
+ * This method checks the specified table information for a maximum record value
+ * and ensures that the returned value is at least 1. If the table's maximum
+ * record limit is defined, it overrides the provided maximum record.
+ *
+ * @param SecretObject $tableInfo The table information containing maximum record settings.
+ * @param int $maxRecord The maximum record limit per query specified by the user.
+ * @return int The effective maximum record limit to be used in queries.
+ */
+ public function getMaxRecord($tableInfo, $maxRecord)
+ {
+ // Check if the table information specifies a maximum record limit
+ if ($tableInfo->getMaximumRecord() !== null) {
+ $maxRecord = $tableInfo->getMaximumRecord(); // Override with table's maximum record
+ }
+
+ // Ensure the maximum record is at least 1
+ if ($maxRecord < 1) {
+ $maxRecord = 1;
+ }
+
+ return $maxRecord; // Return the final maximum record value
+ }
+
+ /**
+ * Processes data mapping according to specified column types and mappings.
+ *
+ * This method updates the input data by mapping source fields to target fields
+ * based on the provided mappings, then filters and fixes the data types
+ * according to the column definitions.
+ *
+ * @param mixed[] $data The input data to be processed.
+ * @param string[] $columns An associative array mapping column names to their types.
+ * @param string[]|null $maps Optional array of mapping definitions in the format 'target:source'.
+ * @return mixed[] The updated data array with fixed types and mappings applied.
+ */
+ public function processDataMapping($data, $columns, $maps = null)
+ {
+ // Check if mappings are provided and are in array format
+ if(isset($maps) && is_array($maps))
+ {
+ foreach($maps as $map)
+ {
+ // Split the mapping into target and source
+ $arr = explode(':', $map, 2);
+ $target = trim($arr[0]);
+ $source = trim($arr[1]);
+ // Map the source value to the target key
+ if (isset($data[$source])) {
+ $data[$target] = $data[$source];
+ unset($data[$source]); // Remove the source key
+ }
+ }
+ }
+ // Filter the data to include only keys present in columns
+ $data = array_intersect_key($data, array_flip(array_keys($columns)));
+
+ // Fix data types based on column definitions
+ $data = $this->fixImportData($data, $columns);
+ return $data; // Return the processed data
+ }
+
+ /**
+ * Creates a mapping template between source and target database tables.
+ *
+ * This method generates a mapping array that indicates which columns
+ * in the target table do not exist in the source table, providing a template
+ * for further processing or data transformation.
+ *
+ * @param PicoDatabase $databaseSource The source database connection.
+ * @param PicoDatabase $databaseTarget The target database connection.
+ * @param string $target The name of the target table.
+ * @return string[] An array of mapping strings indicating missing columns in the source.
+ */
+ public function createMapTemplate($databaseSource, $databaseTarget, $target)
+ {
+ $targetColumns = array_keys($this->showColumns($databaseTarget, $target));
+ $sourceColumns = array_keys($this->showColumns($databaseSource, $target));
+ $map = array();
+ foreach($targetColumns as $column)
+ {
+ if(!in_array($column, $sourceColumns))
+ {
+ $map[] = "$column : ???";
+ }
+ }
+ return $map;
+ }
+
+ /**
+ * Automatically configures import data settings from one database to another.
+ *
+ * This method checks if the target table exists in the existing tables. If it does not, it creates
+ * a new `SecretObject` for the table, determining whether the table is present in the source
+ * database and configuring the mapping accordingly.
+ *
+ * @param PicoDatabase $databaseSource The source database connection.
+ * @param PicoDatabase $databaseTarget The target database connection.
+ * @param array $tables The current array of table configurations.
+ * @param array $sourceTables List of source table names.
+ * @param string $target The name of the target table to be configured.
+ * @param array $existingTables List of existing tables in the target database.
+ * @return array Updated array of table configurations with the new table info added if applicable.
+ */
+ public function updateConfigTable($databaseSource, $databaseTarget, $tables, $sourceTables, $target, $existingTables)
+ {
+ if(!in_array($target, $existingTables))
+ {
+ $tableInfo = new SecretObject();
+ if(in_array($target, $sourceTables))
+ {
+ // ada di database sumber
+ $tableInfo->setTarget($target);
+ $tableInfo->setSource($target);
+ $map = $this->createMapTemplate($databaseSource, $databaseTarget, $target);
+ if(isset($map) && !empty($map))
+ {
+ $tableInfo->setMap($map);
+ }
+ }
+ else
+ {
+ // tidak ada di database sumber
+ $tableInfo->setTarget($target);
+ $tableInfo->setSource("???");
+ }
+ $tables[] = $tableInfo;
+ }
+ return $tables;
+ }
+
+ /**
+ * Checks if the provided array is not empty.
+ *
+ * This method verifies that the input is an array and contains at least one element.
+ *
+ * @param array $array The array to be checked.
+ * @return bool True if the array is not empty; otherwise, false.
+ */
+ public function isNotEmpty($array)
+ {
+ return $array != null && is_array($array) && !empty($array);
+ }
+
+ /**
+ * Imports data from a source database table to a target database table.
+ *
+ * This method fetches records from the specified source table and processes them
+ * according to the provided mapping and column information. It uses a callback
+ * function to handle the generated SQL insert statements in batches, up to a
+ * specified maximum record count.
+ *
+ * @param PicoDatabase $databaseSource The source database from which to import data.
+ * @param PicoDatabase $databaseTarget The target database where data will be inserted.
+ * @param string $tableNameSource The name of the source table.
+ * @param string $tableNameTarget The name of the target table.
+ * @param SecretObject $tableInfo Information about the table, including mapping and constraints.
+ * @param int $maxRecord The maximum number of records to process in a single batch.
+ * @param callable $callbackFunction A callback function to handle the generated SQL statements.
+ * @return bool True on success, false on failure.
+ */
+ public function importDataTable($databaseSource, $databaseTarget, $tableNameSource, $tableNameTarget, $tableInfo, $maxRecord, $callbackFunction)
+ {
+ $maxRecord = $this->getMaxRecord($tableInfo, $maxRecord);
+ try
+ {
+ $columns = $this->showColumns($databaseTarget, $tableNameTarget);
+ $queryBuilderSource = new PicoDatabaseQueryBuilder($databaseSource);
+ $sourceTable = $tableInfo->getSource();
+ $queryBuilderSource->newQuery()
+ ->select("*")
+ ->from($sourceTable);
+ $stmt = $databaseSource->query($queryBuilderSource);
+ $records = array();
+ while($data = $stmt->fetch(PDO::FETCH_ASSOC, PDO::FETCH_ORI_NEXT))
+ {
+ $data = $this->processDataMapping($data, $columns, $tableInfo->getMap());
+ if(count($records) < $maxRecord)
+ {
+ $records[] = $data;
+ }
+ else
+ {
+ if(isset($callbackFunction) && is_callable($callbackFunction))
+ {
+ $sql = $this->insert($tableNameTarget, $records);
+ call_user_func($callbackFunction, $sql, $tableNameSource, $tableNameTarget);
+ }
+ // reset buffer
+ $records = array();
+ }
+ }
+ if(!empty($records) && isset($callbackFunction) && is_callable($callbackFunction))
+ {
+ $sql = $this->insert($tableNameTarget, $records);
+ call_user_func($callbackFunction, $sql, $tableNameSource, $tableNameTarget);
+ }
+ }
+ catch(Exception $e)
+ {
+ error_log($e->getMessage());
+ return false;
+ }
+ return true;
+ }
+
+ /**
+ * Imports data from the source database to the target database.
+ *
+ * This method connects to the source and target databases, executes any pre-import scripts,
+ * transfers data from the source tables to the target tables, and executes any post-import scripts.
+ *
+ * @param SecretObject $config Configuration object containing database and table details.
+ * @param callable $callbackFunction Callback function to execute SQL scripts.
+ * @return bool Returns true on successful import, false on failure.
+ */
+ public function importData($config, $callbackFunction)
+ {
+ $databaseConfigSource = $config->getDatabaseSource();
+ $databaseConfigTarget = $config->getDatabaseTarget();
+
+ $databaseSource = new PicoDatabase($databaseConfigSource);
+ $databaseTarget = new PicoDatabase($databaseConfigTarget);
+ try
+ {
+ $databaseSource->connect();
+ $databaseTarget->connect();
+ $tables = $config->getTable();
+ $maxRecord = $config->getMaximumRecord();
+
+ // query pre import data
+ foreach($tables as $tableInfo)
+ {
+ $tableNameTarget = $tableInfo->getTarget();
+ $tableNameSource = $tableInfo->getSource();
+ $preImportScript = $tableInfo->getPreImportScript();
+ if($this->isNotEmpty($preImportScript))
+ {
+ foreach($preImportScript as $sql)
+ {
+ call_user_func($callbackFunction, $sql, $tableNameSource, $tableNameTarget);
+ }
+ }
+ }
+
+ // import data
+ foreach($tables as $tableInfo)
+ {
+ $tableNameTarget = $tableInfo->getTarget();
+ $tableNameSource = $tableInfo->getSource();
+ $this->importDataTable($databaseSource, $databaseTarget, $tableNameSource, $tableNameTarget, $tableInfo, $maxRecord, $callbackFunction);
+ }
+
+ // query post import data
+ foreach($tables as $tableInfo)
+ {
+ $tableNameTarget = $tableInfo->getTarget();
+ $tableNameSource = $tableInfo->getSource();
+ $postImportScript = $tableInfo->getPostImportScript();
+ if($this->isNotEmpty($postImportScript))
+ {
+ foreach($postImportScript as $sql)
+ {
+ call_user_func($callbackFunction, $sql, $tableNameSource, $tableNameTarget);
+ }
+ }
+ }
+ }
+ catch(Exception $e)
+ {
+ error_log($e->getMessage());
+ return false;
+ }
+ return true;
+ }
+
+ /**
+ * Fixes data for safe use in SQL queries.
+ *
+ * This method processes a given value and formats it as a string suitable for SQL.
+ * It handles strings, booleans, nulls, and other data types appropriately.
+ *
+ * @param mixed $value The value to be processed.
+ * @return string The formatted string representation of the value.
+ */
+ public function fixData($value)
+ {
+ // Initialize the return variable
+ $ret = null;
+
+ // Process string values
+ if (is_string($value))
+ {
+ $ret = "'" . addslashes($value) . "'"; // Escape single quotes for SQL
+ }
+ else if(is_bool($value))
+ {
+ $ret = $value === true ? 'true' : 'false'; // Convert boolean to string
+ }
+ else if ($value === null)
+ {
+ // Handle null values
+ $ret = "null"; // Return SQL null representation
+ }
+ else
+ {
+ // Handle other types (e.g., integers, floats)
+ $ret = $value; // Use the value as-is
+ }
+ return $ret; // Return the processed value
+ }
+
+ /**
+ * Fixes boolean data in the provided array.
+ *
+ * This method updates the specified key in the input array to ensure
+ * that its value is a boolean. If the value is null or an empty string,
+ * it sets the key to null. Otherwise, it converts the value to a boolean
+ * based on the condition that if it equals 1, it is set to true; otherwise, false.
+ *
+ * @param mixed[] $data The input array containing data.
+ * @param string $name The key in the array to update.
+ * @param mixed $value The value to be processed.
+ * @return mixed[] The updated array with the fixed boolean data.
+ */
+ public function fixBooleanData($data, $name, $value)
+ {
+ // Check if the value is null or an empty string
+ if($value === null || $value === '')
+ {
+ $data[$name] = null; // Set to null if the value is not valid
+ }
+ else
+ {
+ // Convert the value to a boolean (true if 1, false otherwise)
+ $data[$name] = $data[$name] == 1 ? true : false;
+ }
+ return $data; // Return the updated array
+ }
+
+ /**
+ * Fixes integer data in the provided array.
+ *
+ * This method updates the specified key in the input array to ensure
+ * that its value is an integer. If the value is null or an empty string,
+ * it sets the key to null. Otherwise, it converts the value to an integer.
+ *
+ * @param mixed[] $data The input array containing data.
+ * @param string $name The key in the array to update.
+ * @param mixed $value The value to be processed.
+ * @return mixed[] The updated array with the fixed integer data.
+ */
+ public function fixIntegerData($data, $name, $value)
+ {
+ // Check if the value is null or an empty string
+ if($value === null || $value === '')
+ {
+ $data[$name] = null; // Set to null if value is not valid
+ }
+ else
+ {
+ // Convert the value to an integer
+ $data[$name] = intval($data[$name]);
+ }
+ return $data; // Return the updated array
+ }
+
+ /**
+ * Fixes float data in the provided array.
+ *
+ * This method updates the specified key in the input array to ensure
+ * that its value is a float. If the value is null or an empty string,
+ * it sets the key to null. Otherwise, it converts the value to a float.
+ *
+ * @param mixed[] $data The input array containing data.
+ * @param string $name The key in the array to update.
+ * @param mixed $value The value to be processed.
+ * @return mixed[] The updated array with the fixed float data.
+ */
+ public function fixFloatData($data, $name, $value)
+ {
+ // Check if the value is null or an empty string
+ if($value === null || $value === '')
+ {
+ $data[$name] = null; // Set to null if value is not valid
+ }
+ else
+ {
+ // Convert the value to a float
+ $data[$name] = floatval($data[$name]);
+ }
+ return $data; // Return the updated array
+ }
+
+ /**
+ * Creates an SQL INSERT query for multiple records.
+ *
+ * This method generates an INSERT statement for a specified table and prepares the values
+ * for binding in a batch operation. It supports multiple records and ensures proper
+ * formatting of values.
+ *
+ * @param string $tableName Name of the table where data will be inserted.
+ * @param array $data An array of associative arrays, where each associative array
+ * represents a record to be inserted.
+ * @return string The generated SQL INSERT statement with placeholders for values.
+ */
+ public function insert($tableName, $data)
+ {
+ // Collect all unique columns from the data records
+ $columns = array();
+ foreach ($data as $record) {
+ $columns = array_merge($columns, array_keys($record));
+ }
+ $columns = array_unique($columns);
+
+ // Create placeholders for the prepared statement
+ $placeholdersArr = array_fill(0, count($columns), '?');
+ $placeholders = '(' . implode(', ', $placeholdersArr) . ')';
+
+ // Build the INSERT query
+ $query = "INSERT INTO $tableName (" . implode(', ', $columns) . ") \r\nVALUES \r\n".
+ implode(",\r\n", array_fill(0, count($data), $placeholders));
+
+ // Prepare values for binding
+ $values = array();
+ foreach ($data as $record) {
+ foreach ($columns as $column) {
+ // Use null if the value is not set
+ $values[] = isset($record[$column]) && $record[$column] !== null ? $record[$column] : null;
+ }
+ }
+
+ // Format each value for safe SQL insertion
+ $formattedElements = array_map(function($element){
+ return $this->fixData($element);
+ }, $values);
+
+ // Replace placeholders with formatted values
+ return vsprintf(str_replace('?', '%s', $query), $formattedElements);
+ }
+
+ /**
+ * Dumps a single record into an SQL insert statement.
+ *
+ * @param array $columns Columns of the target table.
+ * @param string $picoTableName Table name.
+ * @param MagicObject $record Data record.
+ * @return string SQL insert statement.
+ */
+ public function dumpRecord($columns, $picoTableName, $record)
+ {
+ return null;
+ }
+
+ /**
+ * Fixes imported data based on specified column types.
+ *
+ * This method processes the input data array and adjusts the values
+ * according to the expected types defined in the columns array. It
+ * supports boolean, integer, and float types.
+ *
+ * @param mixed[] $data The input data to be processed.
+ * @param string[] $columns An associative array mapping column names to their types.
+ * @return mixed[] The updated data array with fixed types.
+ */
+ public function fixImportData($data, $columns)
+ {
+ return null;
+ }
+
+ /**
+ * Retrieves the columns of a specified table from the database.
+ *
+ * This method executes a SQL query to show the columns of the given table and returns
+ * an associative array where the keys are column names and the values are their respective types.
+ *
+ * @param PicoDatabase $database Database connection object.
+ * @param string $tableName Name of the table whose columns are to be retrieved.
+ * @return array An associative array mapping column names to their types.
+ * @throws Exception If the query fails or the table does not exist.
+ */
+ public function showColumns($database, $tableName)
+ {
+ return null;
+ }
+
+ /**
+ * Converts a MariaDB CREATE TABLE query to a PostgreSQL compatible query.
+ *
+ * This function takes a SQL CREATE TABLE statement written for MariaDB
+ * and transforms it into a format compatible with PostgreSQL. It handles
+ * common data types and syntax differences between the two databases.
+ *
+ * @param string $mariadbQuery The MariaDB CREATE TABLE query to be converted.
+ * @return string The converted PostgreSQL CREATE TABLE query.
+ */
+ public function convertMariaDbToPostgreSql($mariadbQuery) {
+ // Remove comments
+ $query = preg_replace('/--.*?\n|\/\*.*?\*\//s', '', $mariadbQuery);
+
+ // Replace MariaDB data types with PostgreSQL data types
+ $replacements = [
+ 'int' => 'INTEGER',
+ 'tinyint(1)' => 'BOOLEAN', // MariaDB TINYINT(1) as BOOLEAN
+ 'tinyint' => 'SMALLINT',
+ 'smallint' => 'SMALLINT',
+ 'mediumint' => 'INTEGER', // No direct equivalent, use INTEGER
+ 'bigint' => 'BIGINT',
+ 'float' => 'REAL',
+ 'double' => 'DOUBLE PRECISION',
+ 'decimal' => 'NUMERIC', // Decimal types
+ 'date' => 'DATE',
+ 'time' => 'TIME',
+ 'datetime' => 'TIMESTAMP', // Use TIMESTAMP for datetime
+ 'timestamp' => 'TIMESTAMP',
+ 'varchar' => 'VARCHAR', // Variable-length string
+ 'text' => 'TEXT',
+ 'blob' => 'BYTEA', // Binary data
+ 'mediumtext' => 'TEXT', // No direct equivalent
+ 'longtext' => 'TEXT', // No direct equivalent
+ 'json' => 'JSONB', // Use JSONB for better performance in PostgreSQL
+ // Add more type conversions as needed
+ ];
+
+ $query = str_ireplace(array_keys($replacements), array_values($replacements), $query);
+
+ // Handle AUTO_INCREMENT
+ $query = preg_replace('/AUTO_INCREMENT=\d+/', '', $query);
+ $query = preg_replace('/AUTO_INCREMENT/', '', $query);
+
+ // Handle default values for strings and booleans
+ $query = preg_replace('/DEFAULT \'(.*?)\'/', 'DEFAULT \'\1\'', $query);
+
+ // Handle "ENGINE=InnoDB" or other ENGINE specifications
+ $query = preg_replace('/ENGINE=\w+/', '', $query);
+
+ // Remove unnecessary commas
+ $query = preg_replace('/,\s*$/', '', $query);
+
+ // Trim whitespace
+ $query = trim($query);
+
+ return $query;
+ }
+
+ /**
+ * Converts a PostgreSQL CREATE TABLE query to a MySQL compatible query.
+ *
+ * This function takes a SQL CREATE TABLE statement written for PostgreSQL
+ * and transforms it into a format compatible with MySQL. It handles common
+ * data types and syntax differences between the two databases.
+ *
+ * @param string $postgresqlQuery The PostgreSQL CREATE TABLE query to be converted.
+ * @return string The converted MySQL CREATE TABLE query.
+ */
+ public function convertPostgreSqlToMySql($postgresqlQuery) {
+ // Remove comments
+ $query = preg_replace('/--.*?\n|\/\*.*?\*\//s', '', $postgresqlQuery);
+
+ // Replace PostgreSQL data types with MySQL data types
+ $replacements = [
+ 'bigserial' => 'BIGINT AUTO_INCREMENT',
+ 'serial' => 'INT AUTO_INCREMENT',
+ 'character varying' => 'VARCHAR', // Added handling for character varying
+ 'text' => 'TEXT',
+ 'varchar' => 'VARCHAR',
+ 'bigint' => 'BIGINT',
+ 'int' => 'INT',
+ 'integer' => 'INT',
+ 'smallint' => 'SMALLINT',
+ 'real' => 'FLOAT', // Added handling for real
+ 'double precision' => 'DOUBLE', // Added handling for double precision
+ 'boolean' => 'TINYINT(1)',
+ 'timestamp' => 'DATETIME',
+ 'date' => 'DATE',
+ 'time' => 'TIME',
+ 'json' => 'JSON',
+ 'bytea' => 'BLOB', // Added handling for bytea
+ // Add more type conversions as needed
+ ];
+
+ $query = str_ireplace(array_keys($replacements), array_values($replacements), $query);
+
+ // Replace DEFAULT on columns with strings to NULL in MySQL
+ $query = preg_replace('/DEFAULT (\'[^\']*\')/', 'DEFAULT $1', $query);
+
+ // Replace SERIAL with INT AUTO_INCREMENT
+ $query = preg_replace('/\bSERIAL\b/', 'INT AUTO_INCREMENT', $query);
+
+ // Modify "IF NOT EXISTS" for MySQL
+ $query = preg_replace('/CREATE TABLE IF NOT EXISTS/', 'CREATE TABLE IF NOT EXISTS', $query);
+
+ // Remove UNIQUE constraints if necessary (optional)
+ $query = preg_replace('/UNIQUE\s*\(.*?\),?\s*/i', '', $query);
+
+ // Remove 'USING BTREE' if present
+ $query = preg_replace('/USING BTREE/', '', $query);
+
+ return $query;
+ }
+}
\ No newline at end of file
diff --git a/src/Util/Database/PicoDatabaseUtilInterface.php b/src/Util/Database/PicoDatabaseUtilInterface.php
new file mode 100644
index 0000000..ea22955
--- /dev/null
+++ b/src/Util/Database/PicoDatabaseUtilInterface.php
@@ -0,0 +1,48 @@
+fetchAll($sql);
}
- /**
- * Gets the auto-increment keys from the provided table information.
- *
- * @param PicoTableInfo $tableInfo Table information.
- * @return array An array of auto-increment key names.
- */
- public function getAutoIncrementKey($tableInfo)
- {
- $autoIncrement = $tableInfo->getAutoIncrementKeys();
- $autoIncrementKeys = array();
- if(is_array($autoIncrement) && !empty($autoIncrement))
- {
- foreach($autoIncrement as $col)
- {
- if($col["strategy"] == 'GenerationType.IDENTITY')
- {
- $autoIncrementKeys[] = $col["name"];
- }
- }
- }
- return $autoIncrementKeys;
- }
-
/**
* Dumps the structure of a table as a SQL statement.
*
- * @param PicoTableInfo $tableInfo Table information.
- * @param string $picoTableName Table name.
- * @param bool $createIfNotExists Whether to add "IF NOT EXISTS" in the create statement.
- * @param bool $dropIfExists Whether to add "DROP TABLE IF EXISTS" before the create statement.
- * @param string $engine Storage engine (default is 'InnoDB').
- * @param string $charset Character set (default is 'utf8mb4').
- * @return string SQL statement to create the table.
+ * This method generates a SQL CREATE TABLE statement based on the provided table information,
+ * including the option to include or exclude specific clauses such as "IF NOT EXISTS" and
+ * "DROP TABLE IF EXISTS". It also handles the definition of primary keys if present.
+ *
+ * @param PicoTableInfo $tableInfo The information about the table, including column details and primary keys.
+ * @param string $picoTableName The name of the table for which the structure is being generated.
+ * @param bool $createIfNotExists Whether to add "IF NOT EXISTS" in the CREATE statement (default is false).
+ * @param bool $dropIfExists Whether to add "DROP TABLE IF EXISTS" before the CREATE statement (default is false).
+ * @param string|null $engine The storage engine to use for the table (optional, default is null).
+ * @param string|null $charset The character set to use for the table (optional, default is null).
+ * @return string The SQL statement to create the table, including column definitions and primary keys.
*/
public function dumpStructure($tableInfo, $picoTableName, $createIfNotExists = false, $dropIfExists = false, $engine = 'InnoDB', $charset = 'utf8mb4')
{
@@ -125,7 +116,7 @@ public function dumpStructure($tableInfo, $picoTableName, $createIfNotExists = f
foreach($tableInfo->getColumns() as $column)
{
- if(isset($autoIncrementKeys) && is_array($autoIncrementKeys) && in_array($column[self::KEY_NAME], $autoIncrementKeys))
+ if(isset($autoIncrementKeys) && is_array($autoIncrementKeys) && in_array($column[parent::KEY_NAME], $autoIncrementKeys))
{
$query[] = "";
$query[] = "ALTER TABLE `$picoTableName` \r\n\tMODIFY ".trim($this->createColumn($column), " \r\n\t ")." AUTO_INCREMENT";
@@ -139,14 +130,23 @@ public function dumpStructure($tableInfo, $picoTableName, $createIfNotExists = f
/**
* Creates a column definition for a SQL statement.
*
- * @param array $column Column details.
- * @return string SQL column definition.
+ * This method constructs a SQL column definition based on the provided column details,
+ * including the column name, data type, nullability, and default value. The resulting
+ * definition is formatted for use in a CREATE TABLE statement.
+ *
+ * @param array $column An associative array containing details about the column:
+ * - string name: The name of the column.
+ * - string type: The data type of the column (e.g., VARCHAR, INT).
+ * - bool|string nullable: Indicates if the column allows NULL values (true or 'true' for NULL; otherwise, NOT NULL).
+ * - mixed default_value: The default value for the column (optional).
+ *
+ * @return string The SQL column definition formatted as a string, suitable for inclusion in a CREATE TABLE statement.
*/
public function createColumn($column)
{
$col = array();
$col[] = "\t";
- $col[] = "`".$column[self::KEY_NAME]."`";
+ $col[] = "`".$column[parent::KEY_NAME]."`";
$col[] = $column['type'];
if(isset($column['nullable']) && strtolower(trim($column['nullable'])) == 'true')
{
@@ -168,9 +168,15 @@ public function createColumn($column)
/**
* Fixes the default value for SQL insertion based on its type.
*
- * @param string $defaultValue Default value to fix.
- * @param string $type Data type of the column.
- * @return string Fixed default value.
+ * This method processes the given default value according to the specified data type,
+ * ensuring that it is correctly formatted for SQL insertion. For string-like types,
+ * the value is enclosed in single quotes, while boolean and null values are returned
+ * as is.
+ *
+ * @param mixed $defaultValue The default value to fix, which can be a string, boolean, or null.
+ * @param string $type The data type of the column (e.g., ENUM, CHAR, TEXT, INT, FLOAT, DOUBLE).
+ *
+ * @return mixed The fixed default value formatted appropriately for SQL insertion.
*/
public function fixDefaultValue($defaultValue, $type)
{
@@ -186,107 +192,18 @@ public function fixDefaultValue($defaultValue, $type)
}
/**
- * Dumps data from various sources into SQL INSERT statements.
+ * Dumps a single record into an SQL INSERT statement.
*
- * This method processes data from PicoPageData, MagicObject, or an array of MagicObject instances
- * and generates SQL INSERT statements. It supports batching of records and allows for a callback
- * function to handle the generated SQL statements.
+ * This method takes a data record and constructs an SQL INSERT statement
+ * for the specified table. It maps the values of the record to the corresponding
+ * columns based on the provided column definitions.
*
- * @param array $columns Array of columns for the target table.
- * @param string $picoTableName Name of the target table.
- * @param MagicObject|PicoPageData|array $data Data to be dumped. Can be a PicoPageData instance,
- * a MagicObject instance, or an array of MagicObject instances.
- * @param int $maxRecord Maximum number of records to process in a single query (default is 100).
- * @param callable|null $callbackFunction Optional callback function to process the generated SQL
- * statements. The function should accept a single string parameter
- * representing the SQL statement.
- * @return string|null SQL INSERT statements or null if no data was processed.
- */
- public function dumpData($columns, $picoTableName, $data, $maxRecord = 100, $callbackFunction = null) //NOSONAR
- {
- // Check if $data is an instance of PicoPageData
- if($data instanceof PicoPageData)
- {
- // Handle case where fetching data is not required
- if($data->getFindOption() & MagicObject::FIND_OPTION_NO_FETCH_DATA && $maxRecord > 0 && isset($callbackFunction) && is_callable($callbackFunction))
- {
- $records = array();
- $stmt = $data->getPDOStatement();
- // Fetch records in batches
- while($data = $stmt->fetch(PDO::FETCH_ASSOC, PDO::FETCH_ORI_NEXT))
- {
- // Ensure data has all required columns
- $data = $this->processDataMapping($data, $columns);
- if(count($records) < $maxRecord)
- {
- $records[] = $data;
- }
- else
- {
- if(isset($callbackFunction) && is_callable($callbackFunction))
- {
- // Call the callback function with the generated SQL
- $sql = $this->insert($picoTableName, $records);
- call_user_func($callbackFunction, $sql);
- }
- // Reset the records buffer
- $records = array();
- }
- }
- // Handle any remaining records
- if(!empty($records) && isset($callbackFunction) && is_callable($callbackFunction))
- {
- $sql = $this->insert($picoTableName, $records);
- call_user_func($callbackFunction, $sql);
- }
- }
- else if(isset($data->getResult()[0]))
- {
- // If data is available, dump records directly
- return $this->dumpRecords($columns, $picoTableName, $data->getResult());
- }
- }
- else if($data instanceof MagicObject)
- {
- // Handle a single MagicObject instance
- return $this->dumpRecords($columns, $picoTableName, array($data));
- }
- else if(is_array($data) && isset($data[0]) && $data[0] instanceof MagicObject)
- {
- // Handle an array of MagicObject instances
- return $this->dumpRecords($columns, $picoTableName, $data);
- }
- return null; // Return null if no valid data was processed
- }
-
- /**
- * Constructs an SQL INSERT statement for a single record.
+ * @param array $columns An associative array where keys are column names and values are column details.
+ * @param string $picoTableName The name of the table where the record will be inserted.
+ * @param MagicObject $record The data record to be inserted, which provides a method to retrieve values.
*
- * This method takes a data record and maps it to the corresponding columns of the target table,
- * generating an SQL INSERT statement. It uses the PicoDatabaseQueryBuilder to build the query.
- *
- * @param array $columns Associative array mapping column names to their definitions in the target table.
- * @param string $picoTableName Name of the target table where the record will be inserted.
- * @param MagicObject $record The data record to be dumped into the SQL statement.
* @return string The generated SQL INSERT statement.
- */
- public function dumpRecords($columns, $picoTableName, $data)
- {
- $result = "";
- foreach($data as $record)
- {
- $result .= $this->dumpRecord($columns, $picoTableName, $record).";\r\n";
- }
- return $result;
- }
-
- /**
- * Dumps a single record into an SQL insert statement.
- *
- * @param array $columns Columns of the target table.
- * @param string $picoTableName Table name.
- * @param MagicObject $record Data record.
- * @return string SQL insert statement.
+ * @throws Exception If the record cannot be processed or if there are no values to insert.
*/
public function dumpRecord($columns, $picoTableName, $record)
{
@@ -296,7 +213,7 @@ public function dumpRecord($columns, $picoTableName, $record)
{
if(isset($columns[$key]))
{
- $rec[$columns[$key][self::KEY_NAME]] = $val;
+ $rec[$columns[$key][parent::KEY_NAME]] = $val;
}
}
$queryBuilder = new PicoDatabaseQueryBuilder(PicoDatabaseType::DATABASE_TYPE_MYSQL);
@@ -381,284 +298,6 @@ public function autoConfigureImportData($config)
return $config;
}
- /**
- * Automatically configures import data settings from one database to another.
- *
- * This method checks if the target table exists in the existing tables. If it does not, it creates
- * a new `SecretObject` for the table, determining whether the table is present in the source
- * database and configuring the mapping accordingly.
- *
- * @param PicoDatabase $databaseSource The source database connection.
- * @param PicoDatabase $databaseTarget The target database connection.
- * @param array $tables The current array of table configurations.
- * @param array $sourceTables List of source table names.
- * @param string $target The name of the target table to be configured.
- * @param array $existingTables List of existing tables in the target database.
- * @return array Updated array of table configurations with the new table info added if applicable.
- */
- public function updateConfigTable($databaseSource, $databaseTarget, $tables, $sourceTables, $target, $existingTables)
- {
- if(!in_array($target, $existingTables))
- {
- $tableInfo = new SecretObject();
- if(in_array($target, $sourceTables))
- {
- // ada di database sumber
- $tableInfo->setTarget($target);
- $tableInfo->setSource($target);
- $map = $this->createMapTemplate($databaseSource, $databaseTarget, $target);
- if(isset($map) && !empty($map))
- {
- $tableInfo->setMap($map);
- }
- }
- else
- {
- // tidak ada di database sumber
- $tableInfo->setTarget($target);
- $tableInfo->setSource("???");
- }
- $tables[] = $tableInfo;
- }
- return $tables;
- }
-
- /**
- * Creates a mapping template between source and target database tables.
- *
- * This method generates a mapping array that indicates which columns
- * in the target table do not exist in the source table, providing a template
- * for further processing or data transformation.
- *
- * @param PicoDatabase $databaseSource The source database connection.
- * @param PicoDatabase $databaseTarget The target database connection.
- * @param string $target The name of the target table.
- * @return string[] An array of mapping strings indicating missing columns in the source.
- */
- public function createMapTemplate($databaseSource, $databaseTarget, $target)
- {
- $targetColumns = array_keys($this->showColumns($databaseTarget, $target));
- $sourceColumns = array_keys($this->showColumns($databaseSource, $target));
- $map = array();
- foreach($targetColumns as $column)
- {
- if(!in_array($column, $sourceColumns))
- {
- $map[] = "$column : ???";
- }
- }
- return $map;
- }
-
- /**
- * Imports data from the source database to the target database.
- *
- * This method connects to the source and target databases, executes any pre-import scripts,
- * transfers data from the source tables to the target tables, and executes any post-import scripts.
- *
- * @param SecretObject $config Configuration object containing database and table details.
- * @param callable $callbackFunction Callback function to execute SQL scripts.
- * @return bool Returns true on successful import, false on failure.
- */
- public function importData($config, $callbackFunction)
- {
- $databaseConfigSource = $config->getDatabaseSource();
- $databaseConfigTarget = $config->getDatabaseTarget();
-
- $databaseSource = new PicoDatabase($databaseConfigSource);
- $databaseTarget = new PicoDatabase($databaseConfigTarget);
- try
- {
- $databaseSource->connect();
- $databaseTarget->connect();
- $tables = $config->getTable();
- $maxRecord = $config->getMaximumRecord();
-
- // query pre import data
- foreach($tables as $tableInfo)
- {
- $tableNameTarget = $tableInfo->getTarget();
- $tableNameSource = $tableInfo->getSource();
- $preImportScript = $tableInfo->getPreImportScript();
- if($this->isNotEmpty($preImportScript))
- {
- foreach($preImportScript as $sql)
- {
- call_user_func($callbackFunction, $sql, $tableNameSource, $tableNameTarget);
- }
- }
- }
-
- // import data
- foreach($tables as $tableInfo)
- {
- $tableNameTarget = $tableInfo->getTarget();
- $tableNameSource = $tableInfo->getSource();
- $this->importDataTable($databaseSource, $databaseTarget, $tableNameSource, $tableNameTarget, $tableInfo, $maxRecord, $callbackFunction);
- }
-
- // query post import data
- foreach($tables as $tableInfo)
- {
- $tableNameTarget = $tableInfo->getTarget();
- $tableNameSource = $tableInfo->getSource();
- $postImportScript = $tableInfo->getPostImportScript();
- if($this->isNotEmpty($postImportScript))
- {
- foreach($postImportScript as $sql)
- {
- call_user_func($callbackFunction, $sql, $tableNameSource, $tableNameTarget);
- }
- }
- }
- }
- catch(Exception $e)
- {
- error_log($e->getMessage());
- return false;
- }
- return true;
- }
-
- /**
- * Checks if the provided array is not empty.
- *
- * This method verifies that the input is an array and contains at least one element.
- *
- * @param array $array The array to be checked.
- * @return bool True if the array is not empty; otherwise, false.
- */
- public function isNotEmpty($array)
- {
- return $array != null && is_array($array) && !empty($array);
- }
-
- /**
- * Imports data from a source database table to a target database table.
- *
- * This method fetches records from the specified source table and processes them
- * according to the provided mapping and column information. It uses a callback
- * function to handle the generated SQL insert statements in batches, up to a
- * specified maximum record count.
- *
- * @param PicoDatabase $databaseSource The source database from which to import data.
- * @param PicoDatabase $databaseTarget The target database where data will be inserted.
- * @param string $tableNameSource The name of the source table.
- * @param string $tableNameTarget The name of the target table.
- * @param SecretObject $tableInfo Information about the table, including mapping and constraints.
- * @param int $maxRecord The maximum number of records to process in a single batch.
- * @param callable $callbackFunction A callback function to handle the generated SQL statements.
- * @return bool True on success, false on failure.
- */
- public function importDataTable($databaseSource, $databaseTarget, $tableNameSource, $tableNameTarget, $tableInfo, $maxRecord, $callbackFunction)
- {
- $maxRecord = $this->getMaxRecord($tableInfo, $maxRecord);
- try
- {
- $columns = $this->showColumns($databaseTarget, $tableNameTarget);
- $queryBuilderSource = new PicoDatabaseQueryBuilder($databaseSource);
- $sourceTable = $tableInfo->getSource();
- $queryBuilderSource->newQuery()
- ->select("*")
- ->from($sourceTable);
- $stmt = $databaseSource->query($queryBuilderSource);
- $records = array();
- while($data = $stmt->fetch(PDO::FETCH_ASSOC, PDO::FETCH_ORI_NEXT))
- {
- $data = $this->processDataMapping($data, $columns, $tableInfo->getMap());
- if(count($records) < $maxRecord)
- {
- $records[] = $data;
- }
- else
- {
- if(isset($callbackFunction) && is_callable($callbackFunction))
- {
- $sql = $this->insert($tableNameTarget, $records);
- call_user_func($callbackFunction, $sql, $tableNameSource, $tableNameTarget);
- }
- // reset buffer
- $records = array();
- }
- }
- if(!empty($records) && isset($callbackFunction) && is_callable($callbackFunction))
- {
- $sql = $this->insert($tableNameTarget, $records);
- call_user_func($callbackFunction, $sql, $tableNameSource, $tableNameTarget);
- }
- }
- catch(Exception $e)
- {
- error_log($e->getMessage());
- return false;
- }
- return true;
- }
-
- /**
- * Retrieves the maximum record limit for a query.
- *
- * This method checks the specified table information for a maximum record value
- * and ensures that the returned value is at least 1. If the table's maximum
- * record limit is defined, it overrides the provided maximum record.
- *
- * @param SecretObject $tableInfo The table information containing maximum record settings.
- * @param int $maxRecord The maximum record limit per query specified by the user.
- * @return int The effective maximum record limit to be used in queries.
- */
- public function getMaxRecord($tableInfo, $maxRecord)
- {
- // Check if the table information specifies a maximum record limit
- if ($tableInfo->getMaximumRecord() !== null) {
- $maxRecord = $tableInfo->getMaximumRecord(); // Override with table's maximum record
- }
-
- // Ensure the maximum record is at least 1
- if ($maxRecord < 1) {
- $maxRecord = 1;
- }
-
- return $maxRecord; // Return the final maximum record value
- }
-
- /**
- * Processes data mapping according to specified column types and mappings.
- *
- * This method updates the input data by mapping source fields to target fields
- * based on the provided mappings, then filters and fixes the data types
- * according to the column definitions.
- *
- * @param mixed[] $data The input data to be processed.
- * @param string[] $columns An associative array mapping column names to their types.
- * @param string[]|null $maps Optional array of mapping definitions in the format 'target:source'.
- * @return mixed[] The updated data array with fixed types and mappings applied.
- */
- public function processDataMapping($data, $columns, $maps = null)
- {
- // Check if mappings are provided and are in array format
- if(isset($maps) && is_array($maps))
- {
- foreach($maps as $map)
- {
- // Split the mapping into target and source
- $arr = explode(':', $map, 2);
- $target = trim($arr[0]);
- $source = trim($arr[1]);
- // Map the source value to the target key
- if (isset($data[$source])) {
- $data[$target] = $data[$source];
- unset($data[$source]); // Remove the source key
- }
- }
- }
- // Filter the data to include only keys present in columns
- $data = array_intersect_key($data, array_flip(array_keys($columns)));
-
- // Fix data types based on column definitions
- $data = $this->fixImportData($data, $columns);
- return $data; // Return the processed data
- }
-
/**
* Fixes imported data based on specified column types.
*
@@ -700,228 +339,5 @@ public function fixImportData($data, $columns)
return $data;
}
- /**
- * Fixes data for safe use in SQL queries.
- *
- * This method processes a given value and formats it as a string suitable for SQL.
- * It handles strings, booleans, nulls, and other data types appropriately.
- *
- * @param mixed $value The value to be processed.
- * @return string The formatted string representation of the value.
- */
- public function fixData($value)
- {
- // Initialize the return variable
- $ret = null;
-
- // Process string values
- if (is_string($value))
- {
- $ret = "'" . addslashes($value) . "'"; // Escape single quotes for SQL
- }
- else if(is_bool($value))
- {
- $ret = $value === true ? 'true' : 'false'; // Convert boolean to string
- }
- else if ($value === null)
- {
- // Handle null values
- $ret = "null"; // Return SQL null representation
- }
- else
- {
- // Handle other types (e.g., integers, floats)
- $ret = $value; // Use the value as-is
- }
- return $ret; // Return the processed value
- }
-
- /**
- * Fixes boolean data in the provided array.
- *
- * This method updates the specified key in the input array to ensure
- * that its value is a boolean. If the value is null or an empty string,
- * it sets the key to null. Otherwise, it converts the value to a boolean
- * based on the condition that if it equals 1, it is set to true; otherwise, false.
- *
- * @param mixed[] $data The input array containing data.
- * @param string $name The key in the array to update.
- * @param mixed $value The value to be processed.
- * @return mixed[] The updated array with the fixed boolean data.
- */
- public function fixBooleanData($data, $name, $value)
- {
- // Check if the value is null or an empty string
- if($value === null || $value === '')
- {
- $data[$name] = null; // Set to null if the value is not valid
- }
- else
- {
- // Convert the value to a boolean (true if 1, false otherwise)
- $data[$name] = $data[$name] == 1 ? true : false;
- }
- return $data; // Return the updated array
- }
-
- /**
- * Fixes integer data in the provided array.
- *
- * This method updates the specified key in the input array to ensure
- * that its value is an integer. If the value is null or an empty string,
- * it sets the key to null. Otherwise, it converts the value to an integer.
- *
- * @param mixed[] $data The input array containing data.
- * @param string $name The key in the array to update.
- * @param mixed $value The value to be processed.
- * @return mixed[] The updated array with the fixed integer data.
- */
- public function fixIntegerData($data, $name, $value)
- {
- // Check if the value is null or an empty string
- if($value === null || $value === '')
- {
- $data[$name] = null; // Set to null if value is not valid
- }
- else
- {
- // Convert the value to an integer
- $data[$name] = intval($data[$name]);
- }
- return $data; // Return the updated array
- }
-
- /**
- * Fixes float data in the provided array.
- *
- * This method updates the specified key in the input array to ensure
- * that its value is a float. If the value is null or an empty string,
- * it sets the key to null. Otherwise, it converts the value to a float.
- *
- * @param mixed[] $data The input array containing data.
- * @param string $name The key in the array to update.
- * @param mixed $value The value to be processed.
- * @return mixed[] The updated array with the fixed float data.
- */
- public function fixFloatData($data, $name, $value)
- {
- // Check if the value is null or an empty string
- if($value === null || $value === '')
- {
- $data[$name] = null; // Set to null if value is not valid
- }
- else
- {
- // Convert the value to a float
- $data[$name] = floatval($data[$name]);
- }
- return $data; // Return the updated array
- }
-
- /**
- * Creates an SQL INSERT query for multiple records.
- *
- * This method generates an INSERT statement for a specified table and prepares the values
- * for binding in a batch operation. It supports multiple records and ensures proper
- * formatting of values.
- *
- * @param string $tableName Name of the table where data will be inserted.
- * @param array $data An array of associative arrays, where each associative array
- * represents a record to be inserted.
- * @return string The generated SQL INSERT statement with placeholders for values.
- */
- public function insert($tableName, $data)
- {
- // Collect all unique columns from the data records
- $columns = array();
- foreach ($data as $record) {
- $columns = array_merge($columns, array_keys($record));
- }
- $columns = array_unique($columns);
-
- // Create placeholders for the prepared statement
- $placeholdersArr = array_fill(0, count($columns), '?');
- $placeholders = '(' . implode(', ', $placeholdersArr) . ')';
-
- // Build the INSERT query
- $query = "INSERT INTO $tableName (" . implode(', ', $columns) . ") \r\nVALUES \r\n".
- implode(",\r\n", array_fill(0, count($data), $placeholders));
-
- // Prepare values for binding
- $values = array();
- foreach ($data as $record) {
- foreach ($columns as $column) {
- // Use null if the value is not set
- $values[] = isset($record[$column]) && $record[$column] !== null ? $record[$column] : null;
- }
- }
-
- // Format each value for safe SQL insertion
- $formattedElements = array_map(function($element){
- return $this->fixData($element);
- }, $values);
-
- // Replace placeholders with formatted values
- return vsprintf(str_replace('?', '%s', $query), $formattedElements);
- }
-
- /**
- * Converts a MariaDB CREATE TABLE query to a PostgreSQL compatible query.
- *
- * This function takes a SQL CREATE TABLE statement written for MariaDB
- * and transforms it into a format compatible with PostgreSQL. It handles
- * common data types and syntax differences between the two databases.
- *
- * @param string $mariadbQuery The MariaDB CREATE TABLE query to be converted.
- * @return string The converted PostgreSQL CREATE TABLE query.
- */
- public function convertMariaDbToPostgreSql($mariadbQuery) {
- // Remove comments
- $query = preg_replace('/--.*?\n|\/\*.*?\*\//s', '', $mariadbQuery);
-
- // Replace MariaDB data types with PostgreSQL data types
- $replacements = [
- 'int' => 'INTEGER',
- 'tinyint(1)' => 'BOOLEAN', // MariaDB TINYINT(1) as BOOLEAN
- 'tinyint' => 'SMALLINT',
- 'smallint' => 'SMALLINT',
- 'mediumint' => 'INTEGER', // No direct equivalent, use INTEGER
- 'bigint' => 'BIGINT',
- 'float' => 'REAL',
- 'double' => 'DOUBLE PRECISION',
- 'decimal' => 'NUMERIC', // Decimal types
- 'date' => 'DATE',
- 'time' => 'TIME',
- 'datetime' => 'TIMESTAMP', // Use TIMESTAMP for datetime
- 'timestamp' => 'TIMESTAMP',
- 'varchar' => 'VARCHAR', // Variable-length string
- 'text' => 'TEXT',
- 'blob' => 'BYTEA', // Binary data
- 'mediumtext' => 'TEXT', // No direct equivalent
- 'longtext' => 'TEXT', // No direct equivalent
- 'json' => 'JSONB', // Use JSONB for better performance in PostgreSQL
- // Add more type conversions as needed
- ];
-
- $query = str_ireplace(array_keys($replacements), array_values($replacements), $query);
-
- // Handle AUTO_INCREMENT
- $query = preg_replace('/AUTO_INCREMENT=\d+/', '', $query);
- $query = preg_replace('/AUTO_INCREMENT/', '', $query);
-
- // Handle default values for strings and booleans
- $query = preg_replace('/DEFAULT \'(.*?)\'/', 'DEFAULT \'\1\'', $query);
-
- // Handle "ENGINE=InnoDB" or other ENGINE specifications
- $query = preg_replace('/ENGINE=\w+/', '', $query);
-
- // Remove unnecessary commas
- $query = preg_replace('/,\s*$/', '', $query);
-
- // Trim whitespace
- $query = trim($query);
-
- return $query;
- }
-
+
}
\ No newline at end of file
diff --git a/src/Util/Database/PicoDatabaseUtilPostgreSql.php b/src/Util/Database/PicoDatabaseUtilPostgreSql.php
index 78e7d24..e282fa6 100644
--- a/src/Util/Database/PicoDatabaseUtilPostgreSql.php
+++ b/src/Util/Database/PicoDatabaseUtilPostgreSql.php
@@ -6,7 +6,6 @@
use MagicObject\Database\PicoDatabase;
use MagicObject\Database\PicoDatabaseQueryBuilder;
use MagicObject\Database\PicoDatabaseType;
-use MagicObject\Database\PicoPageData;
use MagicObject\Database\PicoTableInfo;
use MagicObject\MagicObject;
use MagicObject\SecretObject;
@@ -15,33 +14,51 @@
/**
* Class PicoDatabaseUtilPostgreSql
*
- * Utility class for managing PostgreSQL database operations in the framework.
- * This class provides methods for retrieving table structures, generating SQL
- * statements for creating tables, dumping data into SQL insert statements,
- * and importing data from one database to another.
+ * This class extends the PicoDatabaseUtilBase and implements the PicoDatabaseUtilInterface specifically
+ * for PostgreSQL database operations. It provides specialized utility methods tailored to leverage PostgreSQL's
+ * features and syntax while ensuring compatibility with the general database utility interface.
*
- * Key Functionalities:
- * - Retrieve and display column information for tables.
- * - Generate SQL statements to create tables based on existing structures.
- * - Dump data from various sources into SQL insert statements.
- * - Facilitate the import of data between source and target databases, including
- * handling pre and post-import scripts.
- * - Ensure data integrity by fixing types during the import process.
+ * Key functionalities include:
+ *
+ * - **Retrieve and display column information for tables:** Methods to fetch detailed column data,
+ * including types and constraints, from PostgreSQL tables.
+ * - **Generate SQL statements to create tables based on existing structures:** Automated generation
+ * of CREATE TABLE statements to replicate existing table schemas.
+ * - **Dump data from various sources into SQL insert statements:** Convert data from different formats
+ * into valid SQL INSERT statements for efficient data insertion.
+ * - **Facilitate the import of data between source and target databases:** Streamlined processes for
+ * transferring data, including handling pre and post-import scripts to ensure smooth operations.
+ * - **Ensure data integrity by fixing types during the import process:** Validation and correction of
+ * data types to match PostgreSQL's requirements, enhancing data quality during imports.
+ *
+ * This class is designed for developers who are working with PostgreSQL databases and need a robust set of tools
+ * to manage database operations efficiently. By adhering to the PicoDatabaseUtilInterface, it provides
+ * a consistent API for database utilities while taking advantage of PostgreSQL-specific features.
+ *
+ * Usage:
+ * To use this class, instantiate it with a PostgreSQL database connection and utilize its methods to perform
+ * various database tasks, ensuring efficient data management and manipulation.
*
* @author Kamshory
* @package MagicObject\Util\Database
* @link https://github.com/Planetbiru/MagicObject
*/
-class PicoDatabaseUtilPostgreSql //NOSONAR
+class PicoDatabaseUtilPostgreSql extends PicoDatabaseUtilBase implements PicoDatabaseUtilInterface //NOSONAR
{
- const KEY_NAME = "name";
/**
- * Retrieves a list of columns for a specified table.
+ * Retrieves a list of columns for a specified table in the database.
+ *
+ * This method queries the information schema to obtain details about the columns
+ * of the specified table, including their names, data types, nullability,
+ * and default values.
*
- * @param PicoDatabase $database Database connection.
- * @param string $picoTableName Table name.
- * @return array An array of column details.
+ * @param PicoDatabase $database The database connection instance.
+ * @param string $picoTableName The name of the table to retrieve column information from.
+ * @return array An array of associative arrays containing details about each column,
+ * where each associative array includes 'column_name', 'data_type',
+ * 'is_nullable', and 'column_default'.
+ * @throws Exception If the database connection fails or the query cannot be executed.
*/
public function getColumnList($database, $picoTableName)
{
@@ -56,39 +73,22 @@ public function getColumnList($database, $picoTableName)
return $database->fetchAll($sql);
}
- /**
- * Gets the auto-increment keys from the provided table information.
- *
- * @param PicoTableInfo $tableInfo Table information.
- * @return array An array of auto-increment key names.
- */
- public function getAutoIncrementKey($tableInfo)
- {
- $autoIncrement = $tableInfo->getAutoIncrementKeys();
- $autoIncrementKeys = array();
- if(is_array($autoIncrement) && !empty($autoIncrement))
- {
- foreach($autoIncrement as $col)
- {
- if($col["strategy"] == 'GenerationType.IDENTITY')
- {
- $autoIncrementKeys[] = $col["name"];
- }
- }
- }
- return $autoIncrementKeys;
- }
-
/**
* Dumps the structure of a table as a SQL statement.
*
- * @param PicoTableInfo $tableInfo Table information.
- * @param string $picoTableName Table name.
- * @param bool $createIfNotExists Whether to add "IF NOT EXISTS" in the create statement.
- * @param bool $dropIfExists Whether to add "DROP TABLE IF EXISTS" before the create statement.
- * @return string SQL statement to create the table.
+ * This method generates a SQL CREATE TABLE statement based on the provided table information,
+ * including the option to include or exclude specific clauses such as "IF NOT EXISTS" and
+ * "DROP TABLE IF EXISTS". It also handles the definition of primary keys if present.
+ *
+ * @param PicoTableInfo $tableInfo The information about the table, including column details and primary keys.
+ * @param string $picoTableName The name of the table for which the structure is being generated.
+ * @param bool $createIfNotExists Whether to add "IF NOT EXISTS" in the CREATE statement (default is false).
+ * @param bool $dropIfExists Whether to add "DROP TABLE IF EXISTS" before the CREATE statement (default is false).
+ * @param string|null $engine The storage engine to use for the table (optional, default is null).
+ * @param string|null $charset The character set to use for the table (optional, default is null).
+ * @return string The SQL statement to create the table, including column definitions and primary keys.
*/
- public function dumpStructure($tableInfo, $picoTableName, $createIfNotExists = false, $dropIfExists = false)
+ public function dumpStructure($tableInfo, $picoTableName, $createIfNotExists = false, $dropIfExists = false, $engine = null, $charset = null)
{
$query = [];
if ($dropIfExists) {
@@ -127,14 +127,23 @@ public function dumpStructure($tableInfo, $picoTableName, $createIfNotExists = f
/**
* Creates a column definition for a SQL statement.
*
- * @param array $column Column details.
- * @return string SQL column definition.
+ * This method constructs a SQL column definition based on the provided column details,
+ * including the column name, data type, nullability, and default value. The resulting
+ * definition is formatted for use in a CREATE TABLE statement.
+ *
+ * @param array $column An associative array containing details about the column:
+ * - string name: The name of the column.
+ * - string type: The data type of the column (e.g., VARCHAR, INT).
+ * - bool|string nullable: Indicates if the column allows NULL values (true or 'true' for NULL; otherwise, NOT NULL).
+ * - mixed default_value: The default value for the column (optional).
+ *
+ * @return string The SQL column definition formatted as a string, suitable for inclusion in a CREATE TABLE statement.
*/
public function createColumn($column)
{
$col = [];
$col[] = "\t";
- $col[] = "\"" . $column[self::KEY_NAME] . "\"";
+ $col[] = "\"" . $column[parent::KEY_NAME] . "\"";
$col[] = $column['type'];
if (isset($column['nullable']) && strtolower(trim($column['nullable'])) == 'true') {
@@ -155,9 +164,15 @@ public function createColumn($column)
/**
* Fixes the default value for SQL insertion based on its type.
*
- * @param string $defaultValue Default value to fix.
- * @param string $type Data type of the column.
- * @return string Fixed default value.
+ * This method processes the given default value according to the specified data type,
+ * ensuring that it is correctly formatted for SQL insertion. For string-like types,
+ * the value is enclosed in single quotes, while boolean and null values are returned
+ * as is.
+ *
+ * @param mixed $defaultValue The default value to fix, which can be a string, boolean, or null.
+ * @param string $type The data type of the column (e.g., ENUM, CHAR, TEXT, INT, FLOAT, DOUBLE).
+ *
+ * @return mixed The fixed default value formatted appropriately for SQL insertion.
*/
public function fixDefaultValue($defaultValue, $type)
{
@@ -173,103 +188,18 @@ public function fixDefaultValue($defaultValue, $type)
}
/**
- * Dumps data from various sources into SQL INSERT statements.
+ * Dumps a single record into an SQL INSERT statement.
*
- * This method processes data from PicoPageData, MagicObject, or an array of MagicObject instances
- * and generates SQL INSERT statements. It supports batching of records and allows for a callback
- * function to handle the generated SQL statements.
+ * This method takes a data record and constructs an SQL INSERT statement
+ * for the specified table. It maps the values of the record to the corresponding
+ * columns based on the provided column definitions.
*
- * @param array $columns Array of columns for the target table.
- * @param string $picoTableName Name of the target table.
- * @param MagicObject|PicoPageData|array $data Data to be dumped. Can be a PicoPageData instance,
- * a MagicObject instance, or an array of MagicObject instances.
- * @param int $maxRecord Maximum number of records to process in a single query (default is 100).
- * @param callable|null $callbackFunction Optional callback function to process the generated SQL
- * statements. The function should accept a single string parameter
- * representing the SQL statement.
- * @return string|null SQL INSERT statements or null if no data was processed.
- */
- public function dumpData($columns, $picoTableName, $data, $maxRecord = 100, $callbackFunction = null) //NOSONAR
- {
- // Check if $data is an instance of PicoPageData
- if($data instanceof PicoPageData)
- {
- // Handle case where fetching data is not required
- if($data->getFindOption() & MagicObject::FIND_OPTION_NO_FETCH_DATA && $maxRecord > 0 && isset($callbackFunction) && is_callable($callbackFunction))
- {
- $records = array();
- $stmt = $data->getPDOStatement();
- // Fetch records in batches
- while($data = $stmt->fetch(PDO::FETCH_ASSOC, PDO::FETCH_ORI_NEXT))
- {
- // Ensure data has all required columns
- $data = $this->processDataMapping($data, $columns);
- if(count($records) < $maxRecord)
- {
- $records[] = $data;
- }
- else
- {
- if(isset($callbackFunction) && is_callable($callbackFunction))
- {
- // Call the callback function with the generated SQL
- $sql = $this->insert($picoTableName, $records);
- call_user_func($callbackFunction, $sql);
- }
- // Reset the records buffer
- $records = array();
- }
- }
- // Handle any remaining records
- if(!empty($records) && isset($callbackFunction) && is_callable($callbackFunction))
- {
- $sql = $this->insert($picoTableName, $records);
- call_user_func($callbackFunction, $sql);
- }
- }
- else if(isset($data->getResult()[0]))
- {
- // If data is available, dump records directly
- return $this->dumpRecords($columns, $picoTableName, $data->getResult());
- }
- }
- else if($data instanceof MagicObject)
- {
- // Handle a single MagicObject instance
- return $this->dumpRecords($columns, $picoTableName, array($data));
- }
- else if(is_array($data) && isset($data[0]) && $data[0] instanceof MagicObject)
- {
- // Handle an array of MagicObject instances
- return $this->dumpRecords($columns, $picoTableName, $data);
- }
- return null; // Return null if no valid data was processed
- }
-
- /**
- * Dumps multiple records into SQL insert statements.
+ * @param array $columns An associative array where keys are column names and values are column details.
+ * @param string $picoTableName The name of the table where the record will be inserted.
+ * @param MagicObject $record The data record to be inserted, which provides a method to retrieve values.
*
- * @param array $columns Columns of the target table.
- * @param string $picoTableName Table name.
- * @param MagicObject[] $data Data records.
- * @return string SQL insert statements.
- */
- public function dumpRecords($columns, $picoTableName, $data)
- {
- $result = "";
- foreach ($data as $record) {
- $result .= $this->dumpRecord($columns, $picoTableName, $record) . ";\r\n";
- }
- return $result;
- }
-
- /**
- * Dumps a single record into an SQL insert statement.
- *
- * @param array $columns Columns of the target table.
- * @param string $picoTableName Table name.
- * @param MagicObject $record Data record.
- * @return string SQL insert statement.
+ * @return string The generated SQL INSERT statement.
+ * @throws Exception If the record cannot be processed or if there are no values to insert.
*/
public function dumpRecord($columns, $picoTableName, $record)
{
@@ -277,7 +207,7 @@ public function dumpRecord($columns, $picoTableName, $record)
$rec = [];
foreach ($value as $key => $val) {
if (isset($columns[$key])) {
- $rec[$columns[$key][self::KEY_NAME]] = $val;
+ $rec[$columns[$key][parent::KEY_NAME]] = $val;
}
}
@@ -292,11 +222,15 @@ public function dumpRecord($columns, $picoTableName, $record)
}
/**
- * Shows the columns of a specified table.
+ * Retrieves the columns of a specified table from the database.
+ *
+ * This method executes a SQL query to show the columns of the given table and returns
+ * an associative array where the keys are column names and the values are their respective types.
*
- * @param PicoDatabase $database Database connection.
- * @param string $tableName Table name.
- * @return string[] An associative array of column names and their types.
+ * @param PicoDatabase $database Database connection object.
+ * @param string $tableName Name of the table whose columns are to be retrieved.
+ * @return array An associative array mapping column names to their types.
+ * @throws Exception If the query fails or the table does not exist.
*/
public function showColumns($database, $tableName)
{
@@ -318,10 +252,14 @@ public function showColumns($database, $tableName)
}
/**
- * Autoconfigure import data
+ * Automatically configures the import data settings based on the source and target databases.
+ *
+ * This method connects to the source and target databases, retrieves the list of existing
+ * tables, and updates the configuration for each target table by checking its presence in the
+ * source database. It handles exceptions and logs any errors encountered during the process.
*
- * @param SecretObject $config Configuration
- * @return SecretObject
+ * @param SecretObject $config The configuration object containing database and table information.
+ * @return SecretObject The updated configuration object with modified table settings.
*/
public function autoConfigureImportData($config)
{
@@ -367,266 +305,15 @@ public function autoConfigureImportData($config)
}
/**
- * Automatically configures import data settings from one database to another.
+ * Fixes imported data based on specified column types.
*
- * @param PicoDatabase $source Source database connection.
- * @param PicoDatabase $target Target database connection.
- * @param string $sourceTable Source table name.
- * @param string $targetTable Target table name.
- * @param array $options Additional options for import configuration.
- * @return array Configured options for import.
- */
- public function updateConfigTable($databaseSource, $databaseTarget, $tables, $sourceTables, $target, $existingTables)
- {
- if(!in_array($target, $existingTables))
- {
- $tableInfo = new SecretObject();
- if(in_array($target, $sourceTables))
- {
- // ada di database sumber
- $tableInfo->setTarget($target);
- $tableInfo->setSource($target);
- $map = $this->createMapTemplate($databaseSource, $databaseTarget, $target);
- if(isset($map) && !empty($map))
- {
- $tableInfo->setMap($map);
- }
- }
- else
- {
- // tidak ada di database sumber
- $tableInfo->setTarget($target);
- $tableInfo->setSource("???");
- }
- $tables[] = $tableInfo;
- }
- return $tables;
- }
-
- /**
- * Create map template
- *
- * @param PicoDatabase $databaseSource Source database
- * @param PicoDatabase $databaseTarget Target database
- * @param string $target Target table
- * @return string[]
- */
- public function createMapTemplate($databaseSource, $databaseTarget, $target)
- {
- $targetColumns = array_keys($this->showColumns($databaseTarget, $target));
- $sourceColumns = array_keys($this->showColumns($databaseSource, $target));
- $map = array();
- foreach($targetColumns as $column)
- {
- if(!in_array($column, $sourceColumns))
- {
- $map[] = "$column : ???";
- }
- }
- return $map;
- }
-
- /**
- * Imports data from the source database to the target database.
- *
- * @param PicoDatabase $source Source database connection.
- * @param PicoDatabase $target Target database connection.
- * @param string $sourceTable Source table name.
- * @param string $targetTable Target table name.
- * @param array $options Options for import operation.
- * @return void
- */
- public function importData($config, $callbackFunction)
- {
- $databaseConfigSource = $config->getDatabaseSource();
- $databaseConfigTarget = $config->getDatabaseTarget();
-
- $databaseSource = new PicoDatabase($databaseConfigSource);
- $databaseTarget = new PicoDatabase($databaseConfigTarget);
- try
- {
- $databaseSource->connect();
- $databaseTarget->connect();
- $tables = $config->getTable();
- $maxRecord = $config->getMaximumRecord();
-
- // query pre import data
- foreach($tables as $tableInfo)
- {
- $tableNameTarget = $tableInfo->getTarget();
- $tableNameSource = $tableInfo->getSource();
- $preImportScript = $tableInfo->getPreImportScript();
- if($this->isNotEmpty($preImportScript))
- {
- foreach($preImportScript as $sql)
- {
- call_user_func($callbackFunction, $sql, $tableNameSource, $tableNameTarget);
- }
- }
- }
-
- // import data
- foreach($tables as $tableInfo)
- {
- $tableNameTarget = $tableInfo->getTarget();
- $tableNameSource = $tableInfo->getSource();
- $this->importDataTable($databaseSource, $databaseTarget, $tableNameSource, $tableNameTarget, $tableInfo, $maxRecord, $callbackFunction);
- }
-
- // query post import data
- foreach($tables as $tableInfo)
- {
- $tableNameTarget = $tableInfo->getTarget();
- $tableNameSource = $tableInfo->getSource();
- $postImportScript = $tableInfo->getPostImportScript();
- if($this->isNotEmpty($postImportScript))
- {
- foreach($postImportScript as $sql)
- {
- call_user_func($callbackFunction, $sql, $tableNameSource, $tableNameTarget);
- }
- }
- }
- }
- catch(Exception $e)
- {
- error_log($e->getMessage());
- return false;
- }
- return true;
- }
-
- /**
- * Check if array is not empty
- *
- * @param array $array Array to be checked
- * @return bool
- */
- public function isNotEmpty($array)
- {
- return $array != null && is_array($array) && !empty($array);
- }
-
- /**
- * Import table
- *
- * @param PicoDatabase $databaseSource Source database
- * @param PicoDatabase $databaseTarget Target database
- * @param string $tableName Table name
- * @param SecretObject $tableInfo Table information
- * @param int $maxRecord Maximum record per query
- * @param callable $callbackFunction Callback function
- * @return bool
- */
- public function importDataTable($databaseSource, $databaseTarget, $tableNameSource, $tableNameTarget, $tableInfo, $maxRecord, $callbackFunction)
- {
- $maxRecord = $this->getMaxRecord($tableInfo, $maxRecord);
- try
- {
- $columns = $this->showColumns($databaseTarget, $tableNameTarget);
- $queryBuilderSource = new PicoDatabaseQueryBuilder($databaseSource);
- $sourceTable = $tableInfo->getSource();
- $queryBuilderSource->newQuery()
- ->select("*")
- ->from($sourceTable);
- $stmt = $databaseSource->query($queryBuilderSource);
- $records = array();
- while($data = $stmt->fetch(PDO::FETCH_ASSOC, PDO::FETCH_ORI_NEXT))
- {
- $data = $this->processDataMapping($data, $columns, $tableInfo->getMap());
- if(count($records) < $maxRecord)
- {
- $records[] = $data;
- }
- else
- {
- if(isset($callbackFunction) && is_callable($callbackFunction))
- {
- $sql = $this->insert($tableNameTarget, $records);
- call_user_func($callbackFunction, $sql, $tableNameSource, $tableNameTarget);
- }
- // reset buffer
- $records = array();
- }
- }
- if(!empty($records) && isset($callbackFunction) && is_callable($callbackFunction))
- {
- $sql = $this->insert($tableNameTarget, $records);
- call_user_func($callbackFunction, $sql, $tableNameSource, $tableNameTarget);
- }
- }
- catch(Exception $e)
- {
- error_log($e->getMessage());
- return false;
- }
- return true;
- }
-
- /**
- * Get maximum record
- *
- * @param SecretObject $tableInfo Table information
- * @param int $maxRecord Maximum record per query
- * @return int
- */
- public function getMaxRecord($tableInfo, $maxRecord)
- {
- if($tableInfo->getMaximumRecord() != null)
- {
- $maxRecord = $tableInfo->getMaximumRecord();
- }
- if($maxRecord < 1)
- {
- $maxRecord = 1;
- }
- return $maxRecord;
- }
-
- /**
- * Processes data mapping according to specified column types and mappings.
- *
- * This method updates the input data by mapping source fields to target fields
- * based on the provided mappings, then filters and fixes the data types
- * according to the column definitions.
+ * This method processes the input data array and adjusts the values
+ * according to the expected types defined in the columns array. It
+ * supports boolean, integer, and float types.
*
* @param mixed[] $data The input data to be processed.
* @param string[] $columns An associative array mapping column names to their types.
- * @param string[]|null $maps Optional array of mapping definitions in the format 'target:source'.
- * @return mixed[] The updated data array with fixed types and mappings applied.
- */
- public function processDataMapping($data, $columns, $maps = null)
- {
- // Check if mappings are provided and are in array format
- if(isset($maps) && is_array($maps))
- {
- foreach($maps as $map)
- {
- // Split the mapping into target and source
- $arr = explode(':', $map, 2);
- $target = trim($arr[0]);
- $source = trim($arr[1]);
- // Map the source value to the target key
- if (isset($data[$source])) {
- $data[$target] = $data[$source];
- unset($data[$source]); // Remove the source key
- }
- }
- }
- // Filter the data to include only keys present in columns
- $data = array_intersect_key($data, array_flip(array_keys($columns)));
-
- // Fix data types based on column definitions
- $data = $this->fixImportData($data, $columns);
- return $data; // Return the processed data
- }
-
- /**
- * Fix import data
- *
- * @param mixed[] $data Data
- * @param string[] $columns Columns
- * @return mixed[]
+ * @return mixed[] The updated data array with fixed types.
*/
public function fixImportData($data, $columns)
{
@@ -652,195 +339,4 @@ public function fixImportData($data, $columns)
return $data;
}
- /**
- * Fix data
- *
- * @param mixed $value Value
- * @return string
- */
- public function fixData($value)
- {
- $ret = null;
- if (is_string($value))
- {
- $ret = "'" . addslashes($value) . "'";
- }
- else if(is_bool($value))
- {
- $ret = $value === true ? 'true' : 'false';
- }
- else if ($value === null)
- {
- $ret = "null";
- }
- else
- {
- $ret = $value;
- }
- return $ret;
- }
-
- /**
- * Fix boolean data
- *
- * @param mixed[] $data Data
- * @param string $name Name
- * @param mixed $value Value
- * @return mixed[]
- */
- public function fixBooleanData($data, $name, $value)
- {
- if($value === null || $value === '')
- {
- $data[$name] = null;
- }
- else
- {
- $data[$name] = $data[$name] == 1 ? true : false;
- }
- return $data;
- }
-
- /**
- * Fix integer data
- *
- * @param mixed[] $data Data
- * @param string $name Name
- * @param mixed $value Value
- * @return mixed[]
- */
- public function fixIntegerData($data, $name, $value)
- {
- if($value === null || $value === '')
- {
- $data[$name] = null;
- }
- else
- {
- $data[$name] = intval($data[$name]);
- }
- return $data;
- }
-
- /**
- * Fix float data
- *
- * @param mixed[] $data Data
- * @param string $name Name
- * @param mixed $value Value
- * @return mixed[]
- */
- public function fixFloatData($data, $name, $value)
- {
- if($value === null || $value === '')
- {
- $data[$name] = null;
- }
- else
- {
- $data[$name] = floatval($data[$name]);
- }
- return $data;
- }
-
- /**
- * Create query insert with multiple record
- *
- * @param string $tableName Table name
- * @param array $data Data
- * @return string
- */
- public function insert($tableName, $data)
- {
- // Kumpulkan semua kolom
- $columns = array();
- foreach ($data as $record) {
- $columns = array_merge($columns, array_keys($record));
- }
- $columns = array_unique($columns);
-
- // Buat placeholder untuk prepared statement
- $placeholdersArr = array_fill(0, count($columns), '?');
- $placeholders = '(' . implode(', ', $placeholdersArr) . ')';
-
- // Buat query INSERT
- $query = "INSERT INTO $tableName (" . implode(', ', $columns) . ") \r\nVALUES \r\n".
- implode(",\r\n", array_fill(0, count($data), $placeholders));
-
- // Siapkan nilai untuk bind
- $values = array();
- foreach ($data as $record) {
- foreach ($columns as $column) {
- $values[] = isset($record[$column]) && $record[$column] !== null ? $record[$column] : null;
- }
- }
-
- // Fungsi untuk menambahkan single quote jika elemen adalah string
-
- // Format elemen array
- $formattedElements = array_map(function($element){
- return $this->fixData($element);
- }, $values);
-
- // Ganti tanda tanya dengan elemen array yang telah diformat
- return vsprintf(str_replace('?', '%s', $query), $formattedElements);
- }
-
-
- /**
- * Converts a PostgreSQL CREATE TABLE query to a MySQL compatible query.
- *
- * This function takes a SQL CREATE TABLE statement written for PostgreSQL
- * and transforms it into a format compatible with MySQL. It handles common
- * data types and syntax differences between the two databases.
- *
- * @param string $postgresqlQuery The PostgreSQL CREATE TABLE query to be converted.
- * @return string The converted MySQL CREATE TABLE query.
- */
- public function convertPostgreSqlToMySql($postgresqlQuery) {
- // Remove comments
- $query = preg_replace('/--.*?\n|\/\*.*?\*\//s', '', $postgresqlQuery);
-
- // Replace PostgreSQL data types with MySQL data types
- $replacements = [
- 'bigserial' => 'BIGINT AUTO_INCREMENT',
- 'serial' => 'INT AUTO_INCREMENT',
- 'character varying' => 'VARCHAR', // Added handling for character varying
- 'text' => 'TEXT',
- 'varchar' => 'VARCHAR',
- 'bigint' => 'BIGINT',
- 'int' => 'INT',
- 'integer' => 'INT',
- 'smallint' => 'SMALLINT',
- 'real' => 'FLOAT', // Added handling for real
- 'double precision' => 'DOUBLE', // Added handling for double precision
- 'boolean' => 'TINYINT(1)',
- 'timestamp' => 'DATETIME',
- 'date' => 'DATE',
- 'time' => 'TIME',
- 'json' => 'JSON',
- 'bytea' => 'BLOB', // Added handling for bytea
- // Add more type conversions as needed
- ];
-
- $query = str_ireplace(array_keys($replacements), array_values($replacements), $query);
-
- // Replace DEFAULT on columns with strings to NULL in MySQL
- $query = preg_replace('/DEFAULT (\'[^\']*\')/', 'DEFAULT $1', $query);
-
- // Replace SERIAL with INT AUTO_INCREMENT
- $query = preg_replace('/\bSERIAL\b/', 'INT AUTO_INCREMENT', $query);
-
- // Modify "IF NOT EXISTS" for MySQL
- $query = preg_replace('/CREATE TABLE IF NOT EXISTS/', 'CREATE TABLE IF NOT EXISTS', $query);
-
- // Remove UNIQUE constraints if necessary (optional)
- $query = preg_replace('/UNIQUE\s*\(.*?\),?\s*/i', '', $query);
-
- // Remove 'USING BTREE' if present
- $query = preg_replace('/USING BTREE/', '', $query);
-
- return $query;
- }
-
}
diff --git a/tests/dto.php b/tests/dto.php
new file mode 100644
index 0000000..44c24b3
--- /dev/null
+++ b/tests/dto.php
@@ -0,0 +1,709 @@
+name")
+ * @var string
+ */
+ protected $producerName;
+
+ /**
+ * City
+ *
+ * @JsonProperty("domisili")
+ * @Source("producer->city->namaKota")
+ * @var string
+ */
+ protected $kotaDomisili;
+}
+
+class ProducerDto extends MagicDto
+{
+ /**
+ * Producer ID
+ *
+ * @Source("producerId")
+ * @JsonProperty("id_producer")
+ * @var string
+ */
+ protected $pid;
+
+ /**
+ * Name
+ *
+ * @JsonProperty("jenenge")
+ * @var string
+ */
+ protected $name;
+}
+$city = new MagicObject();
+$album = new EntityAlbum();
+$album->setProducer(new Producer());
+
+$album->setAlbumId("1234");
+$album->setName("Album Pertama");
+$album->getProducer()->setProducerId("5678");
+$album->getProducer()->setName("Kamshory");
+$album->getProducer()->setCity($city);
+$album->getProducer()->getCity()->setNamaKota("Jakarta");
+
+
+$albumDto = new AlbumDto($album);
+
+echo $albumDto;
\ No newline at end of file
diff --git a/tests/dump.php b/tests/dump.php
new file mode 100644
index 0000000..f8cc8d5
--- /dev/null
+++ b/tests/dump.php
@@ -0,0 +1,34 @@
+loadYamlFile(dirname(dirname(__DIR__))."/test.yml", false, true, true);
+$database = new PicoDatabase($databaseCredential->getDatabase(), function(){}, function($sql){
+ //echo $sql;
+});
+$database->connect();
+
+$song = new Song(null, $database);
+/*
+$speficication = null
+$pageable = null
+$sortable = null
+$passive = true
+$subqueryMap = null
+$findOption = MagicObject::FIND_OPTION_NO_COUNT_DATA | MagicObject::FIND_OPTION_NO_FETCH_DATA
+*/
+$pageData = $song->findAll(null, null, null, true, null, MagicObject::FIND_OPTION_NO_COUNT_DATA | MagicObject::FIND_OPTION_NO_FETCH_DATA);
+$dumpForSong = new PicoDatabaseDump();
+
+$dumpForSong->dumpData($pageData, PicoDatabaseType::DATABASE_TYPE_MYSQL, new Song(), $maxRecord, function($sql){
+ $fp = fopen("dump.sql", "a");
+ fputs($fp, $sql);
+ fclose($fp);
+});
diff --git a/tests/import.php b/tests/import.php
index 4aad373..4fa80b2 100644
--- a/tests/import.php
+++ b/tests/import.php
@@ -10,7 +10,8 @@
$fp = fopen(__DIR__.'/db.sql', 'w');
fclose($fp);
-$sql = PicoDatabaseUtilMySql::importData($config, function($sql, $source, $target){
+$tool = new PicoDatabaseUtilMySql();
+$sql = $tool->importData($config, function($sql, $source, $target){
$fp = fopen(__DIR__.'/db.sql', 'a');
fwrite($fp, $sql.";\r\n\r\n");
fclose($fp);
diff --git a/tutorial.md b/tutorial.md
index fadb26f..d7c25c0 100644
--- a/tutorial.md
+++ b/tutorial.md
@@ -1524,6 +1524,421 @@ catch(Exception $e)
}
```
+
+## MagicDto
+
+### Introduction to DTOs
+
+A Data Transfer Object (DTO) is a design pattern used to transfer data between software application subsystems or layers. DTOs encapsulate data, reducing the number of method calls needed to retrieve or send information. JSON (JavaScript Object Notation) has become the standard for data serialization due to its simplicity and ease of integration with various programming languages.
+
+The properties defined in MagicDto adhere strictly to the specifications set forth by the developer, ensuring a well-defined structure. This means that users are prohibited from adding any input or output that falls outside the established DTO framework. As a result, the integrity of the data is maintained, and the application remains predictable and reliable, as any deviation from the predefined structure is not permitted. This strict adherence to the DTO structure promotes clarity and consistency, facilitating better communication between different layers of the application while reducing the risk of errors or unintended behavior.
+
+### The Need for MagicDto
+
+In modern applications, especially those that interact with third-party services, maintaining consistent data formats can be challenging. Different systems may use varying naming conventions, such as camel case (`myProperty`) and snake case (`my_property`). Additionally, inconsistencies can occur with uppercase and lowercase letters, leading to potential mismatches when exchanging data.
+
+**MagicDto** addresses these issues by allowing developers to create DTOs that seamlessly translate property names between different naming conventions. This ensures that data is properly formatted for both internal and external use, enhancing interoperability and reducing errors.
+
+### Features of MagicDto
+
+1. **Flexible Naming Strategies**:
+
+ - MagicDto supports both camel case and snake case naming strategies. This flexibility is particularly useful when integrating with diverse APIs or legacy systems that may employ different conventions.
+
+2. **Automatic Property Mapping**:
+
+ - Users can define DTOs that automatically map properties from their internal representation to the expected format of third-party services. This reduces boilerplate code and simplifies maintenance.
+
+3. **Annotations for Clarity**:
+
+ - The MagicDto class utilizes PHP annotations to clarify the purpose of each property. These annotations enhance code readability and provide useful metadata for serialization.
+
+### Class Structure
+
+The `MagicDto` class is designed with properties that have protected access levels, ensuring encapsulation while still allowing derived classes to access these properties. Each property is annotated with `@var`, which specifies its data type. This structured approach enhances type safety and improves code quality.
+
+#### Key Annotations
+
+1. **@Source**
+
+ The `@Source` annotation indicates the source property that maps to a specific field in the incoming data. If this annotation is omitted, MagicDto will default to using the property name that matches the class property name. This allows for flexibility in cases where the external API may use different naming conventions.
+
+```php
+/**
+ * @Source("album_name")
+ * @var string
+ */
+protected $title;
+```
+
+2. **@JsonProperty**
+
+ The `@JsonProperty` annotation specifies the output property name when data is serialized to JSON. If this annotation is not provided, MagicDto will serialize the property using its class property name. This ensures that data sent to third-party applications adheres to their expected format.
+
+```php
+/**
+ * @JsonProperty("album_title")
+ * @var string
+ */
+protected $title;
+```
+
+We can put it together
+
+```php
+/**
+ * @Source("album_name")
+ * @JsonProperty("album_title")
+ * @var string
+ */
+protected $title;
+```
+
+In this example, `@Source("album_name")` indicates that the incoming data will use `album_name`, while `@JsonProperty("album_title")` specifies that when the data is serialized, it will be output as `album_title`.
+
+To facilitate bidirectional communication, we need two different DTOs. The `@Source` annotation in the first DTO corresponds to the `@JsonProperty` annotation in the second DTO, while the `@JsonProperty` in the first DTO maps to the `@Source` in the second DTO.
+
+**Example:**
+
+DTO on the Input Side
+
+```php
+class AlbumDtoInput extends MagicDto
+{
+ /**
+ * @Source("album_id")
+ * @JsonProperty("albumId")
+ * @var string
+ */
+ protected $id;
+
+ /**
+ * @Source("album_name")
+ * @JsonProperty("albumTitle")
+ * @var string
+ */
+ protected $title;
+
+ /**
+ * @Source("date_release")
+ * @JsonProperty("releaseDate")
+ * @var string
+ */
+ protected $release;
+
+ /**
+ * @Source("song")
+ * @JsonProperty("numberOfSong")
+ * @var string
+ */
+ protected $songs;
+}
+```
+
+DTO on the Output Side
+
+```php
+class AlbumDtoOutput extends MagicDto
+{
+ /**
+ * @Source("albumId")
+ * @JsonProperty("album_id")
+ * @var string
+ */
+ protected $id;
+
+ /**
+ * @Source("albumTitle")
+ * @JsonProperty("album_name")
+ * @var string
+ */
+ protected $title;
+
+ /**
+ * @Source("releaseDate")
+ * @JsonProperty("date_release")
+ * @var string
+ */
+ protected $release;
+
+ /**
+ * @Source("numberOfSong")
+ * @JsonProperty("song")
+ * @var string
+ */
+ protected $songs;
+}
+```
+
+**Description**
+
+In this example, we have two DTO classes: AlbumDtoInput and AlbumDtoOutput. The AlbumDtoInput class is designed to receive data from external sources, using the @Source annotation to specify the incoming property names and the @JsonProperty annotation to define the corresponding properties in the internal representation.
+
+Conversely, the AlbumDtoOutput class is structured for sending data outwards. Here, the @Source annotation reflects the internal property names, while the @JsonProperty annotation defines the expected property names when the data is serialized for external use. This bidirectional mapping ensures that data flows seamlessly between internal and external systems.
+
+The `@Source` annotation allows a Data Transfer Object (DTO) to inherit properties from an underlying object, enabling seamless data integration across related entities.
+
+### Cross Object Mapping
+
+
+#### Cross Object Mapping Explanation
+
+1. **Concept Clarification**:
+
+ - Cross Object Mapping refers to the ability to access and utilize properties from related objects in a hierarchical structure. In your case, the `SongDto` pulls in the agency name associated with the artist of a song.
+2. **DTO Definition**:
+
+ - A DTO is a simple object that carries data between processes. In this context, `SongDto` aggregates data from the `Song`, `Artist`, and `Agency` models without duplicating properties unnecessarily.
+
+For example, we want to directly include properties from the agency within the SongDto.
+
+- **Song**
+ - **Artist**
+ - **Agency**
+
+When creating a DTO for a `Song`, the user can incorporate properties from the associated `Agency` into the `SongDto`. This is particularly useful for aggregating data from related models without needing to replicate information.
+
+#### Code Implementation
+
+**Song**
+
+```php
+class Song extends MagicObject {
+ /**
+ * Song ID
+ *
+ * @Column(name="song_id")
+ * @var string
+ */
+ protected $songId;
+
+ /**
+ * Name
+ *
+ * @Column(name="name")
+ * @var string
+ */
+ protected $name;
+
+ /**
+ * Artist
+ *
+ * @JoinColumn(name="artist_id")
+ * @var Artist
+ */
+ protected $artist;
+
+ // Additional properties and methods for the Song can be defined here.
+}
+```
+
+**Artist**
+
+```php
+class Artist extends MagicObject {
+ /**
+ * Artist ID
+ *
+ * @Column(name="artist_id")
+ * @var string
+ */
+ protected $artistId;
+
+ /**
+ * Name
+ *
+ * @Column(name="name")
+ * @var string
+ */
+ protected $name;
+
+ /**
+ * Agency
+ *
+ * @JoinColumn(name="agency_id")
+ * @var Agency
+ */
+ protected $agency;
+
+ // Additional properties and methods for the Artist can be defined here.
+}
+```
+
+**Agency**
+
+```php
+class Agency extends MagicObject {
+ /**
+ * Agency ID
+ *
+ * @Column(name="agency_id")
+ * @var string
+ */
+ protected $agencyId;
+
+ /**
+ * Name
+ *
+ * @Column(name="name")
+ * @var string
+ */
+ protected $name;
+
+ // Additional properties and methods for the Agency can be defined here.
+}
+```
+
+**SongDto**
+
+```php
+class SongDto extends MagicDto
+{
+ /**
+ * Song ID
+ *
+ * @Source("songId")
+ * @JsonProperty(name="song_id")
+ * @var string
+ */
+ protected $songId;
+
+ /**
+ * Title
+ *
+ * @Source("title")
+ * @JsonProperty("title")
+ * @var string
+ */
+ protected $title;
+
+ /**
+ * Artist
+ *
+ * @Source("artist")
+ * @JsonProperty("artist")
+ * @var ArtistDto
+ */
+ protected $artist;
+
+ /**
+ * The name of the agency associated with the artist.
+ *
+ * This property is sourced from the agency related to the artist of the song.
+ *
+ * @Source("artist->agency->name")
+ * @JsonProperty("agency_name")
+ * @var string
+ */
+ protected $agencyName;
+
+ // Additional properties and methods for the SongDto can be defined here.
+}
+```
+
+**ArtistDto**
+
+```php
+class ArtistDto extends MagicDto
+{
+ /**
+ * Artist ID
+ *
+ * @Source("artistId")
+ * @JsonProperty(name="artist_id")
+ * @var string
+ */
+ protected $artistId;
+
+ /**
+ * Name
+ *
+ * @Source("name")
+ * @JsonProperty("name")
+ * @var string
+ */
+ protected $name;
+
+ /**
+ * Agency
+ *
+ * @Source("agency")
+ * @JsonProperty("agency")
+ * @var AgencyDto
+ */
+ protected $agency;
+
+ /**
+ * The name of the agency associated with the artist.
+ *
+ * This property is sourced from the agency related to the artist of the song.
+ *
+ * @Source("artist->agency->name")
+ * @JsonProperty("agency_name")
+ * @var string
+ */
+ protected $agencyName;
+
+ // Additional properties and methods for the SongDto can be defined here.
+}
+```
+
+**AgencyDto**
+
+```php
+class AgencyDto extends MagicDto
+{
+ /**
+ * Agency ID
+ *
+ * @Source("agencyId")
+ * @JsonProperty(name="agency_id")
+ * @var string
+ */
+ protected $agencyId;
+
+ /**
+ * Name
+ *
+ * @Source("name")
+ * @JsonProperty("name")
+ * @var string
+ */
+ protected $name;
+}
+
+```
+
+**Usage**
+
+```php
+$song = new Song(null, $database);
+$song->find("1234");
+$songDto = new SongDto($song);
+
+header("Content-type: application/json");
+echo $songDto;
+```
+
+#### Explanation
+
+- **@Source**: This annotation specifies the path to the property within the nested object structure. In this case, `artist->agency->name` indicates that the `agencyName` will pull data from the `name` property of the `Agency` object linked to the `Artist`.
+
+- **@JsonProperty**: This annotation maps the `agencyName` property to a different key in the JSON representation of the DTO. Here, it will be serialized as `agency_name`.
+
+- **protected $agencyName**: This declares the `agencyName` property with protected visibility, ensuring that it can only be accessed within the class itself and by subclasses.
+
+This approach enhances data encapsulation and promotes cleaner code by allowing DTOs to automatically gather necessary data from related entities.
+
+### Benefits of Using MagicDto
+
+- **Reduced Complexity**: By automating property translation, MagicDto minimizes the need for manual mapping code, reducing complexity and potential errors.
+- **Improved Maintainability**: With clearly defined annotations and a structured approach, developers can easily understand and maintain the DTOs, even as systems evolve.
+- **Enhanced Interoperability**: MagicDto ensures that data exchanged between different systems is consistent and correctly formatted, leading to smoother integrations and fewer runtime issues.
+
+### Conclusion
+
+MagicDto is a powerful solution for managing data transfer in applications that need to communicate with external systems. By leveraging flexible naming strategies and clear annotations, it simplifies the process of creating and maintaining DTOs, ensuring seamless data exchange. Whether you’re building a new application or integrating with legacy systems, MagicDto can help you navigate the complexities of data serialization and improve overall application reliability.
+
## Input POST/GET/COOKIE/REQUEST/SERVER
In PHP, handling user input can be done through various superglobals, such as $_POST, $_GET, $_COOKIE, $_REQUEST, and $_SERVER. Each of these superglobals serves a specific purpose for gathering data from different types of requests.
@@ -10078,6 +10493,13 @@ $dumpForSong = new PicoDatabaseDump();
echo $dumpForSong->dumpStructure($song, PicoDatabaseType::DATABASE_TYPE_MYSQL, true, true);
```
+**Parameters:**
+
+- **$song:** An instance of the Song class, representing the table structure you want to dump.
+- **PicoDatabaseType::DATABASE_TYPE_MYSQL:** The type of database you are targeting (e.g., MySQL, PostgreSQL).
+- **true (createIfNotExists):** Whether to include the "IF NOT EXISTS" clause in the CREATE statement.
+- **true (dropIfExists):** Whether to include the "DROP TABLE IF EXISTS" statement before creating the table.
+
### Dump Data
We can dump data by connecting to real database. Don't forget to define the target database type. If we will dump multiple table, we must use dedicated instance of `PicoDatabaseDump`.
@@ -10089,6 +10511,39 @@ $dumpForSong = new PicoDatabaseDump();
echo $dumpForSong->dumpData($pageData, PicoDatabaseType::DATABASE_TYPE_MYSQL);
```
+**Important Note:**
+
+When exporting a table with a large amount of data, the above method may not be suitable, as it can consume a significant amount of memory while trying to accommodate all the data before writing it to a file.
+
+To efficiently handle large datasets, you can use the following approach:
+
+```php
+$song = new Song(null, $database);
+/*
+$speficication = null
+$pageable = null
+$sortable = null
+$passive = true
+$subqueryMap = null
+$findOption = MagicObject::FIND_OPTION_NO_COUNT_DATA | MagicObject::FIND_OPTION_NO_FETCH_DATA
+*/
+$pageData = $song->findAll(null, null, null, true, null, MagicObject::FIND_OPTION_NO_COUNT_DATA | MagicObject::FIND_OPTION_NO_FETCH_DATA);
+$dumpForSong = new PicoDatabaseDump();
+
+$dumpForSong->dumpData($pageData, PicoDatabaseType::DATABASE_TYPE_MYSQL, new Song(), $maxRecord, function($sql){
+ $fp = fopen("dump.sql", "a");
+ fputs($fp, $sql);
+ fclose($fp);
+});
+```
+
+**Explanation:**
+
+- **findAll Parameters:** This allows you to customize your query. The options provided (e.g., FIND_OPTION_NO_COUNT_DATA, FIND_OPTION_NO_FETCH_DATA) ensure that only the necessary data is fetched, thus reducing memory consumption.
+- **Callback Function:** The anonymous function passed as a parameter to dumpData handles the SQL output, appending it to dump.sql in an efficient manner, thereby avoiding excessive memory usage.
+
+By following these guidelines, you can effectively manage both the structure and data dumping processes while optimizing for performance and resource utilization.
+
### Summary
- **Structure Dumping**: Use `dumpStructure` to get the schema of the database without a real connection.