There is a lack of integrations to other products, particularly application development life cycle management ADLM or other test management solutions. Borland Borland has full support for the development life cycle, including tools for assessing code quality and integration between requirements and test management. The products should be considered by those requiring broad technology coverage and desiring consistent incremental innovation.
Strengths Borland provides very strong test management and support for risk-based test planning and continuous stream of innovation, including mobile and agile. The vendor provides strong integration within its testing suite to external integrated development environments IDEs , and its own life cycle management tools for traceability and automation. Support for mobile load and performance testing is built into Silk Performer. Cautions Borland lacks support for testing service-oriented architecture SOA and service virtualization.
The vendor continues to invest in the former Compuware tools, but users should consider eventually using migration tools to switch to Silk. While Borland has made improvements to cross-product integration, there is still needed improvement between test management, test automation and performance testing.
The vendor is expanding its focus on the testing space with go-to-market investments, as well adding a number of OEM relationships and expanding its graphical user interface GUI testing capabilities.
Strengths CA offers market-leading service virtualization solutions with broad protocol and technology support. The vendor has taken a strong position in tools to enable DevOps practices, including driving continuous testing. CA offers new support for mobile testing, cloud-based performance testing and Selenium-based UI test automation. Customers note that integration within the suite is weak, but has improved in recent releases.
Strengths HP is the market share leader in performance testing and test management, making investments in expanding its capabilities to incorporate cloud and mobile, and connecting DevOps features, including integration into continuous integration tools and life cycle virtualization solutions. HP has a broad network of partners, both for service and technology providers, including even its competitors.
Customer references cite flexibility, extensibility and maturity of the tool suite, as well as alignment to core test delivery. Cautions HP lacks support for developer unit testing, although it offers API testing and integration with continuous integration tools; in general, it has become more of a fast follower than an innovator, leveraging partners to fill gaps until the market settles.
The vendor carries a price premium, with an improving, but complex, pricing model. References consistently mention the high costs of licensing, support and maintenance, although this is being mitigated with the recent introductions of lower-cost license and SaaS offerings. Gartner has seen a consistent pattern among competitors gaining market share directly from former HP customers, indicating a weakening trend in its traditionally dominant market leadership.
It is well-positioned for large-scale enterprise testing organizations, combining the tool portfolio and IBM Global Business Services to build effective practices and a strong partner network. The products have made strong strides during the past few years in becoming more consistent and improving ease of use. Strengths IBM has one of the broadest portfolios in the testing tool market, going beyond the core of test management and test automation to include static analysis, unit testing, service virtualization and test data management.
IBM is also a market leader in ADLM tools, providing companies a consolidated set of tools across requirements, development, and quality management and execution. Users note that collaboration facilities across various participants in the software delivery process are very strong.
Cautions While IBM has greatly improved the consistency and integration facilities of its tools, customers still note that ease of adoption, integration to third-party products and ease of upgrades are a challenge, and IBM should communicate its directions more clearly.
Because of the broad set of scenarios supported, some users note that the pace of innovation can be slower than desired and product support experiences can be mixed. The ability to support multiple environments and reuse testing assets across different test types and landscapes has been noted by users as needing improvement.
Its tools will primarily appeal to enterprises utilizing Visual Studio for development with technically oriented test teams. Strengths Because of the core integration to development teams and TFS, Microsoft provides solid support for the integration of testing into the build process and for governing code quality processes. Well-documented and well-supported tools and training are provided through products, services, the Microsoft Developer Network, and a third-party ecosystem of publishers and service producers.
Microsoft has been extending its development and testing stack with support for open source, enabling support for a wide variety of technologies running on its Azure cloud offering, and also supporting cross-browser testing. Cautions A strong focus on TFS and the Microsoft stack means that its testing suite is less valuable for outside the Microsoft sphere; moreover, there is little support for package application testing.
TFS offers limited support for mobile. Currently, it relies on integration with Perfecto Mobile to support native device testing via coded UI tests. A developer-centric focus has resulted in the lack of a script-free approach to functional automation. This has the effect of relegating subject matter experts SMEs to manual and exploratory test roles. However, support for exploratory testing is exemplary. The vendor offers frameworks to drive acceleration of testing for its packaged applications, as well as for Oracle development frameworks.
Functional test scripts can be consumed by load and performance test drivers. Oracle has a strong partner network and global presence. Cautions The vendor lacks a cohesive application life cycle management ALM platform or integration to other life cycle products, and internal integration and coordination is often lacking e. Recognition outside of the Oracle system management market has been limited. Customers note that the included reporting features are suited for simple reports only.
Original Software Original Software provides a strong solution for functional test automation and a process that enables users to progress easily from manual to automated testing and tools that do not require scripting, thus reducing costs to create and maintain automation. The tools are well-suited to business-focused test teams.
Customers consistently report that Original Software offers ease and high productivity via a code-free approach and process to migrate from manual to automated testing. The vendor offers test data management that extracts and masks data, as well as rollback database testing and support for API-level SOA testing. The vendor is strongest in North America and the U. While it continues to strengthen its support on the agile front, it lacks service virtualization.
Parasoft Parasoft has a strong message around development productivity that comes from a shift-left approach to driving quality through the entire development process.
However, this approach tends to position it more as a complement to other QA vendors, rather than as a direct competitor. As such, its technologies are often good tuck-ins, primarily focused on the developer to round out an overall approach to software quality. Strengths Parasoft has a strong focus on overall quality and the early detection of defects to drive a software defect prevention approach; toward this end, it has a major focus on both continuous integration and agile application development AD.
Strong dashboarding functionality enables everyone connected with a project to gauge progress and quality levels, and tools tie in to common continuous integration systems, creating a good fit for agile teams. Tools are not oriented around basic acceptance tests or SME-level users.
Its focus on defect prevention, rather than traditional testing areas, positions Parasoft as a complement to a larger test strategy, but generally not as the core anchor of a QA tool portfolio. SmartBear Consider SmartBear if you are looking for a broad set of tools that covers developer-oriented code quality, including code and document review, in addition to core testing. The tools are a good fit for technically adept organizations that want well-priced tools that weave naturally from development to operations.
Strengths SmartBear provides solid support for migrating assets from HP solutions into its products.
It offers an attractive pricing model, with price points that address different markets. SmartBear has a full life cycle view of software quality, including application performance management, as well as third-party integration and a strong developer community program. SmartBear has grown through acquisitions, leading to tools with different UI models and levels of integration. Soasta Soasta has consistently provided innovation for modern applications with cloud-delivered load testing, and extending to mobile application testing.
Consider Soasta for testing modern browser applications, including mobile. Strengths Soasta provides ease of automation with its codeless, full object-level testing without modification of the device, thus reducing maintenance and providing more accurate gesture capture and playback.
The vendor has been proven in extremely large-scale performance testing and has strong services to aid in the effective testing of high-capacity sites. The combination of real-time analytics with the codeless automation environment and real-user monitoring RUM provides a good foundation for driving improved test effectiveness. Cautions Soasta is focused on specific market segments — mobile for iOS and Android and Web performance testing — and acts as a best of breed for specific functions, rather than a comprehensive quality suite.
The vendor is missing test management functionality, and users report that some overall administrative functions are weak; it relies on integration and partnerships for these.
Telerik Telerik provides a set of testing tools that supports modern UI frameworks, agile testing techniques, and the testing of iOS, Android and Windows Phone 8 apps.
It is well-suited to independent software vendor ISV - and product-development-focused teams. NET or JavaScript as the scripting language. The vendor offers a low-cost product line with a subscription-based licensing model. There are frequent updates, adding new features. Tests can be recorded either on the device or using an emulator. The Telerik AppFeedback module offers the ability to gather and directly integrate customer feedback into the mobile development process.
Cautions Although Telerik goes beyond its integration with the Microsoft technology stack for Web testing, the larger market often fails to see beyond this partnership, keeping the vendor off the shortlists of many potential non-Microsoft-centric customers. TestPlant TestPlant has grown rapidly over the past several years — from a foundation as the only vendor focused on the Apple platform to a full cross-platform test suite provider offering tools that are easy to use at nonpremium prices.
The vendor has delivered a consistent stream of innovation and delivers a full complement of functionality, including industry-specific solutions. Strengths TestPlant offers a complete solution for functional, performance and test management, focused on ease of use. Functional tests can be reused in the eggPlant Performance testing tool, which can also consume Selenium tests and includes broad protocol support.
TestPlant has shown a strong ability to execute the delivery of new products to market most recently, manual testing and mobile device management at a consistent pace, both through internal development and acquisitions.
The Test management solution provides lab management and integration to continuous integration. Users also note that functionality varies across OS types. The vendor has limited partnerships and integrations to the rest of the AD life cycle, although it does provide an integration facility. The scripting language, while straightforward, is unique to TestPlant.
Tricentis Tricentis delivers innovative tools that have proven strong results in driving automation success. The vendor has grown its portfolio through internal innovation and partnerships, and has greatly improved its global field operations during the past year. It should be considered by enterprises that have struggled to make test automation work and those seeking to support agile continuous automation practices.
Strengths Tricentis offers market-leading, model-based test planning and automation. The vendor provides market-leading test case design and planning, with strong analytics and risk coverage model. Tricentis offers integrated synthetic test data management, service testing and support for testing analytics applications.
It is making strong efforts to support agile testing with integration to continuous integration, and support for behavior-driven development and API testing.
We use cookies to deliver the best possible experience on our website. To learn more, visit our Privacy Policy. By continuing to use this site, or closing this box, you consent to our use of cookies. By Thomas Murphy January 31, 2 Comments.
The testing tool market continues to be dynamic with many new entrants each year. Most of these are new solutions for a specific area: functional automation, device testing, bug tracking but there are also new players entering the market each year going after more comprehensive solutions.
Software testing continues to be hard, costly, and embattled to explain why it is both of these. Gartner Research.
Already have a Gartner Account? Log In. Purchase this Document To purchase this document, you will need to register or sign in above Buy Now. Become a client Learn how to access this content as a Gartner client.
0コメント