X

Break New Ground

Recent Posts

DevOps

Using GitLFS for Git Binary File Management with Visual Builder Studio

As a best practice your Git repository shouldn't include large binary files - since you usually don't actually manage the content of these files in a similar way to source code files. But sometime these files are part of your code base, and you would like to track them along with the rest of your code. GitLFS - an open source project - helps in doing this in an efficient way. The latest version of Oracle Visual Builder Studio added support for using GitLFS. Git Large File Storage (LFS) replaces large files such as audio samples, videos, datasets, and graphics with text pointers inside Git, while storing the file contents on a remote server. Visual Builder Studio will store the files in your Oracle Cloud Object Storage. In the demo below you can see how easy it is to use GitLFS with your free Git repositories in Visual Builder Studio. After you download and install GitLFS on your client, you can add it to a specific Git repo by issuing a "git lfs install" in the directory into which you cloned your repository. Then indicate which files to track using commands such as "git lfs track *.bin". Add the new .gitattributes file to your git transaction, and that's it. From now on, when you add files of the specific type to your git repo - they will be stored outside of your git repo. Here is a quick demo -  For more info see our doc.  

As a best practice your Git repository shouldn't include large binary files - since you usually don't actually manage the content of these files in a similar way to source code files. But...

DevOps

Analyzing Your Code Dependencies for Vulnerabilities with Visual Builder Studio

Most of the projects developed today rely on open source libraries - it is likely that your project does too. While using these libraries accelerates development by relying on proven solution for common needs, it also introduces a potential risk factor. Over time people discover vulnerabilities in many of these libraries. While most of the issues will get fixed in newer versions of the libraries, it is up to the consumer of the library to make sure they are up-to-date on the latest version. It is a hard task to constantly track the libraries you are using and to keep them up to date. For projects that are already in production, and have not kept their libraries updated, this is an even bigger risk opening organization to potential malicious attacks leveraging these known vulnerabilities. Automating a review process of the libraries used in your code against databases of known vulnerability can help you stay one-step ahead of attackers. This is exactly what the Dependency Vulnerability Analyzer in Visual Builder Studio helps you achieve. In the demo below you can see how simple it is to automate the process of reviewing the libraries your code uses, get a report about any known vulnerability in those libraries, and even how fixing your code can be streamlined. The demo specifically uses a project that is managing the libraries used through NPM - and scans those dependencies from the package.json file against several repositories of known vulnerabilities. You can also conduct similar checks for Java projects that leverage Maven and pom files to manage dependencies. Check out the demo, and start implementing these security check for your project today!    

Most of the projects developed today rely on open source libraries - it is likely that your project does too. While using these libraries accelerates development by relying on proven solution for...

Open Source

Getting Started With The Oracle Cloud Infrastructure Python SDK

In a recent blog post I illustrated how to use the OCI Command Line Interface (CLI) in shell scripts. While the OCI CLI is comprehensive and powerful, it may not be the best solution when you need to handle a lot of data in shell scripts. In such cases using a programming language such as Python and the Oracle Cloud Infrastructure Python SDK makes more sense. Data manipulation is much easier, and the API is —as expected— more complex. In an attempt to demystify the use of the OCI Python SDK, I have re-written and improved the sample oci-provision.sh shell script in Python. This sample project is named oci-compute and is published on GitHub. This blog post highlights the key concepts of the OCI Python SDK, and together with the oci-compute sample code it should help you to get started easily. About oci-compute The oci-compute tool does everything oci-provision.sh does; better, faster and with some additional capabilities: List available Platform, Custom and Marketplace images Create Compute Instances from a Platform, Custom or Marketplace image A cloud-init file can be specified to run custom scripts during instance configuration List, start, stop and terminate Compute Instances Command line syntax and parameters naming are similar to the OCI CLI tool. See the project README for more information on usage and configuration. I am using this tool on a daily basis to easily manage OCI Compute instances from the command line. OCI Python SDK installation At the time of this writing, the SDK supports Python version 3.5 or 3.6 and can be easily installed using pip, preferably in a Python virtual environment. Installation and required dependencies are described in detail in the documentation. oci-compute installation The oci-compute utility is distributed as a Python package. The setup.py file lists the SDK as dependency; installing the tool will automatically pull the SDK if not already installed. See the README file for detailed installation steps, but in short it is as simple as creating a virtual environment and running: $ pip3 install . The package is split in two main parts: cli.py: handles the command line parsing using the Click package. It defines all the commands, sub-commands and their parameters; instantiate the OciCompute class and invoke its methods. oci_compute.py: defines the OciCompute class which interacts with the OCI SDK. This is the most interesting part of this project. OCI SDK Key concepts This section describes the key concepts used by the OCI SDK. Configuration The first step for using the OCI SDK is to create a configuration dictionary (Python dict). While you can build it manually, you will typically use the oci.config.from_file API call to load it from a configuration file. The default configuration file is ~/.oci/config. It is worth noticing that the OCI CLI uses the same configuration file and provides a command to create it: $ oci setup config For oci-compute, the configuration file is loaded during the class initialization: self._config = oci.config.from_file(config_file, profile) API Service Clients The OCI API is organized in Services, and for each Service you will have to instantiate a Service Client. For example, our oci-compute package uses the following Services: Compute Service (part of Core Services): to manage the Compute Services (provision and manage compute hosts). Virtual Network Service (part of Core Services): to manage the Networking Components (virtual cloud network, Subnet, …) Identity Service: to manage users, groups, compartments, and policies. Marketplace Service: to manage applications in Oracle Cloud Infrastructure Marketplace We instantiate the Service Clients in the class initialization: # Instantiate clients self._compute_client = oci.core.ComputeClient(self._config) self._identity_client = oci.identity.IdentityClient(self._config) self._virtual_network_client = oci.core.VirtualNetworkClient(self._config) self._marketplace_client = oci.marketplace.MarketplaceClient(self._config) Models Models allows you to create objects needed by the API calls. Example: to use an image from the Marketplace, we need to subscribe to the Application Catalog. This is done with the ComputeClient create_app_catalog_subscription method. This method needs an CreateAppCatalogSubscriptionDetails object as parameter. We will use the corresponding model to create such object: oci.core.models.CreateAppCatalogSubscriptionDetails. In oci-compute: app_catalog_subscription_detail = oci.core.models.CreateAppCatalogSubscriptionDetails( compartment_id=compartment_id, listing_id=app_catalog_listing_agreements.listing_id, listing_resource_version=app_catalog_listing_agreements.listing_resource_version, oracle_terms_of_use_link=app_catalog_listing_agreements.oracle_terms_of_use_link, eula_link=app_catalog_listing_agreements.eula_link, signature=app_catalog_listing_agreements.signature, time_retrieved=app_catalog_listing_agreements.time_retrieved ) self._compute_client.create_app_catalog_subscription(app_catalog_subscription_detail).data Pagination All list operations are paginated; that is: they will return a single page of data and you will need to call the method again to get additional pages. The pagination module allows you, amongst other, to retrieve all data in a single API call. Example: to list the available images in a compartment we could do: response = self._compute_client.list_images(compartment_id) which will only return the first page of data. To get get all images at once we will do instead: response = oci.pagination.list_call_get_all_results(self._compute_client.list_images, compartment_id) The first parameter to list_call_get_all_results is the paginated list method, subsequent parameters are the ones of the list method itself. Waiters and Composite operations To wait for an operation to complete (e.g.: wait until an instance is started), you can use the wait_until function. Alternatively, there are convenience classes in the SDK which will perform an action on a resource and wait for it to enter a particular state: the CompositeOperation classes. Example: start an instance and wait until it is started. The following code snippet shows how to start an instance and wait until it is up and running: compute_client_composite_operations = oci.core.ComputeClientCompositeOperations(self._compute_client) compute_client_composite_operations.instance_action_and_wait_for_state( instance_id=instance_id, action='START', wait_for_states=[oci.core.models.Instance.LIFECYCLE_STATE_RUNNING]) Error handling A complete list of exceptions raised by the SDK is available in the exception handling section of the documentation. In short, if your API calls are valid (correct parameters, …) the main exception you should care about is the ServiceError one which is raised when a service returns an error response; that is: a non-2xx HTTP status. For the sake of simplicity and clarity in the sample code, oci-compute does not capture most exceptions. Service Errors will result in a Python stack traceback. A simple piece of code where we have to consider the Service Error exception is illustrated here: for vnic_attachment in vnic_attachments: try: vnic = self._virtual_network_client.get_vnic(vnic_attachment.vnic_id).data except oci.exceptions.ServiceError: vnic = None if vnic and vnic.is_primary: break Putting it all together The oci-compute sample code should be self explanatory, but let’s walk through what happens when e.g. oci-compute provision platform --operating-system "Oracle Linux" --operating-system-version 7.8 --display-name ol78 is invoked. First of all, the CLI parser will instantiate an OciCompute object. This is done once at the top level, for any oci-compute command: ctx.obj['oci'] = OciCompute(config_file=config_file, profile=profile, verbose=verbose The OciCompute class initialization will: Load the OCI configuration from file Instantiate the Service Clients The Click package will then invoke provision_platform function which in turn will call the OciCompute.provision_platform method. We use the oci.core.ComputeClient.list_images to retrieve the most recent Platform Image matching the given Operating System and its version: images = self._compute_client.list_images( compartment_id, operating_system=operating_system, operating_system_version=operating_system_version, shape=shape, sort_by='TIMECREATED', sort_order='DESC').data if not images: self._echo_error("No image found") return None image = images[0] We then call OciCompute._provision_image for the actual provisioning. This method uses all of the key concepts explained earlier. Pagination is used to retrieve the Availability Domains using the Identity Client list_availability_domains method: availability_domains = oci.pagination.list_call_get_all_results( self._identity_client.list_availability_domains, compartment_id ).data VCN and subnet are retrieved using the Virtual Network Client (list_vcns and list_subnets methods) Metadata is populated with the SSH public key and a cloud-init file if provided: # Metadata with the ssh keys and the cloud-init file metadata = {} with open(ssh_authorized_keys_file) as ssh_authorized_keys: metadata['ssh_authorized_keys'] = ssh_authorized_keys.read() if cloud_init_file: metadata['user_data'] = oci.util.file_content_as_launch_instance_user_data(cloud_init_file) Models are used to create an instance launch details (oci.core.models.InstanceSourceViaImageDetails, oci.core.models.CreateVnicDetails and oci.core.models.LaunchInstanceDetails methods): instance_source_via_image_details = oci.core.models.InstanceSourceViaImageDetails(image_id=image.id) create_vnic_details = oci.core.models.CreateVnicDetails(subnet_id=subnet.id) launch_instance_details = oci.core.models.LaunchInstanceDetails( display_name=display_name, compartment_id=compartment_id, availability_domain=availability_domain.name, shape=shape, metadata=metadata, source_details=instance_source_via_image_details, create_vnic_details=create_vnic_details) Last step is to use the launch_instance_and_wait_for_state Composite Operation to actually provision the instance and wait until it is available: compute_client_composite_operations = oci.core.ComputeClientCompositeOperations(self._compute_client) response = compute_client_composite_operations.launch_instance_and_wait_for_state( launch_instance_details, wait_for_states=[oci.core.models.Instance.LIFECYCLE_STATE_RUNNING], waiter_kwargs={'wait_callback': self._wait_callback}) We use the optional waiter callback to display a simple progress indicator oci-compute demo Short demo of the oci-compute tool: Conclusion In this post, I’ve shown how to use oci-compute to easily provision and manage your OCI Compute Instances from the command line as well as how to create your own Python scripts using the Oracle Cloud Infrastructure Python SDK.

In a recent blog post I illustrated how to use the OCI Command Line Interface (CLI) in shell scripts. While the OCI CLI is comprehensive and powerful, it may not be the best solution when you need to...

Cloud

New Features In Oracle Visual Builder - March 2020

The Visual Builder team is happy to announce the rollout of a new version of our service with many new features and enhancements. In this blog entry we highlight the key new additions, but you are certain to discover many other enhancements as you go through your daily development process as well. Action Chain Test Automation We all know that following a test driven development approach can help detect coding issues sooner, making it easier to produce a fix. So how come not all developers follow this methodology? Maybe because writing tests for your code is not fun. With the new Action Chains Test Automation features Visual Builder makes test creation easy and painless. Visual Builder looks at your action chain, and automatically suggest helpful information for defining tests. All you have to do is provide examples of values you expect in variables, and typical results from your REST calls, and VB runs through your action chains and offer suggestions for test results you might want to add to your test case. You'll also be able to see what percentage of your code is covered by your test, and be able to run individual or complete test suites easily. Application Profiles When Visual Builder applications access data from other sources, it is quite common to have different backend environments used for the development, test, and production stages. With the new application profiles feature in VB, you can define combinations of different servers and security settings to be used in each stage of your application. These profiles will be picked up when you move your application from dev to stage to publish and applied properly. You can also use the same profiles when you automate your application's CI/CD through the VB grunt lifecycle commands. Further simplifying this switching is the introduction of two new artifacts - backends and servers. Backends - are types of servers you connect to such as Oracle Cloud Apps, Oracle Integration Cloud, and Oracle Process Cloud. The other artifact is called a server - which is constructed from a combination of URL, headers, authentication, and connection settings for a specific server. This allows you to easily switch the server being used as a specific backend in your application when using a specific application profile. Open API 3.0 Support Open API 3.0 is becoming the de-facto industry standard for describing REST based services through metadata. Up until now VB supported Swagger - the predecessor to Open API. With this release Visual Builder adds support to import Open API 3.0 meta-data to help help you consume REST services easily. Improved REST Service Connection Options We've redesigned our REST service definition dialogs to streamline the process of adding REST services to your app. We've also made it easier to understand how to connect to different services in terms of security settings, CORS support, and usage of the Visual Builder Proxy.  In addition, we added support for connecting to resources on Oracle Compute Infrastructure (OCI) using the API signature approach. We also made it easy to specify headers and assign them to specific servers. You'll notice UI improvements all over the service connection dialogs, as well ass the ability to switch to code view to see the service's metadata directly. Easier Component ID manipulation In the new release we exposed the ID attribute of UI components in the Property Inspector - so you can see or modify it quickly. This is quite handy when you later want to refer to the component from action chains, either through a call component method action, or from your JavaScript code when you try to access the UI component. By the way, if you decide to change the ID after you already referred to it from action chains, VB is smart enough to refactor the ID across those places too. Find in Files While on the topic of refactoring across the app, a new "Find in Files" tab allows you to find all the references to a name in the app, so you can refactor all of them more easily.  New Mobile Capabilities To help you further leverage the devices you are running your apps on, the Open URL action allows you to easily access files and open them with dedicated mobile apps.  Two new $application level variables ($application.deployment.appType & $application.deployment.pwa) distinct between applications that are web vs mobile and whether they are packaged as PWA. This can help you better tailor your app behavior while reusing code in the app. Mobile app also benefit from the new Application Profile features mentioned above. JET 8.1 Support We've updated Visual Builder to pick up the latest version of Oracle JET, including new UI components, and, of course, any bug fixes and enhancement that were included in the releases since JET 7.1. You'll also notice new UI components in the Component Palette including treeView and heatMap visualizations. Business Object Resource Editors When you build large data models made up of many inter-related business objects, the service definitions for these business objects can become bloated. This is because, by default, Visual Builder generates a complete set of accessor REST APIs at every level of the relationship between objects.  We've introduced a new Business Object Resource Editors which allow you to tailor exactly which relationships are exposed by REST endpoints and which aren't, thereby maximizing the performance of your apps both at runtime and when using the designer. Wait There's More The features we listed above are by no means a complete list of new features. You should visit our "What's New" document to learn about other features and to be able to directly navigate for the new documentation on the features we added. Let us know what you think by commenting on the blog or on the Oracle Visual Builder Community.  

The Visual Builder team is happy to announce the rollout of a new version of our service with many new features and enhancements. In this blog entry we highlight the key new additions, but you...

Database

Connect PHP 7 to Oracle Database using packages from Oracle Linux Yum Server

Note: This post was updated to include the latest available release of PHP as well as simplified installation intstructions for Oracle Instant Client introduced starting with 19c.   We recently added PHP 7.4 to our repos on Oracle Linux yum server. These repos include also include the PHP OCI8 extenstion to connect your PHP applications to Oracle database. In this post I describe the steps to install PHP 7.4, PHP OCI8 and Oracle Instant Client on Oracle Linux to connect PHP to Oracle Database. For this blog post, I used a free Autonomous Database included in Oracle Cloud Free Tier. Install Oracle Instant Client Oracle Instant Client RPMs are available on Oracle Linux yum server also. To access them, install the oracle-release-el7 package first to setup the appropriate repositories:   $ sudo yum -y install oracle-release-el7 $ sudo yum -y install oracle-instantclient19.5-basic If you want to be able to use SQL*Plus (this can come in handy for some sanity checks), install the SQL*Plus RPM also: $ sudo yum -y install oracle-instantclient19.5-sqlplus Create a Schema and Install the HR Sample Objects (Optional) You can use any schema you already have in your database. I’m going to use the HR schema from the Oracle Database Sample Schemas on github.com If you already have a schema with database objects to work with, you can skip this step. $ yum -y install git $ git clone https://github.com/oracle/db-sample-schemas.git $ cd db-sample-schemas/human_resources As SYSTEM (or ADMIN, if you are using Autonomous Database), create a user PHPTEST SQL> grant connect, resource, create view to phptest identified by <YOUR DATABASE PASSWORD>; SQL> alter user PHPTEST quota 5m on USERS; If you are using Autonomous Database like I am, change the tablespace above to DATA: SQL> alter user phptest quota 5m on DATA; As the PHPTEST user, run the scripts hr_cre.sql and hr_popul.sql to create and populate the HR database objects SQL> connect phptest/<YOUR DATABASE PASSWORD>@<YOUR CONNECT STRING> SQL> @hr_cre.sql SQL> @hr_popul.sql Install PHP and PHP OCI8 To install PHP 7.4, make sure you have the latest oracle-php-release-el7 package installed first. $ sudo yum install -y oracle-php-release-el7 Next, install PHP and the PHP OCI8 extenstion corresponding to the Oracle Instant Client installed earlier: $ sudo yum -y install php php-oci8-19c Running the following php code snippet should verify that we can connect PHP to the database and bring back data. Make sure you replace the schema and connect string as appropriate. Create a file emp.php based on the code above. Run it! $ php emp.php This should produce the following: King Kochhar De Haan Hunold Ernst Austin Pataballa Lorentz Greenberg Faviet Chen Sciarra ...  

Note: This post was updated to include the latest available release of PHP as well as simplified installation intstructions for Oracle Instant Client introduced starting with 19c.   We recently added...