In this study, we evaluate several trading strategies based on the textual content of corporate disclosures using machine learning. To obtain a conservative estimate of profitability, we require orders to be placed for the close price of the current trading day and stocks must exhibit high liquidity to ensure proper order execution. Our evaluation based on 354,992 form 8-K filings and 10,204 ad hoc announcements shows that the proposed trading strategies yield up to 7.81% and 9.34% out-of-sample annualized return. More importantly, we find that the prevalent approach in the literature of estimating the stock market reaction of a disclosure based on the closing price of the past trading and omitting liquidity filters substantially overestimates profitability. We also provide useful insights for practitioners by describing feature importance to shed light onto how the machine learning models arrive at decisions.