Source-free open-set domain adaptation (SF-OSDA) aims to maintain the model’s discriminative ability on known classes while recognizing and distinguishing unknown class samples in the target domain when source domain data is inaccessible. Although existing SF-OSDA methods have introduced mechanisms to model and identify unknown classes, their exploration remains confined to a single visual feature space and thus fails to effectively leverage the deep semantic information inherent in unknown classes. As a result, these methods struggle to reliably model unknown classes with complex semantics or large cross-domain variations, which in turn limits their generalization capability in complex real-world scenarios. To address this issue, this study proposes a source-free open-set domain adaptation method based on attribute prompting and prototype alignment, which leverages the strong semantic alignment capability of the multimodal vision-language model CLIP. On the one hand, by constructing fine-grained class attribute prompts, the zero-shot recognition ability of the CLIP model is enhanced, allowing the semantic features of samples from different classes in the target domain to be captured more accurately. On the other hand, to achieve structured alignment between the visual space and the semantic space, as well as precise recognition of known classes and effective distinction of unknown classes, a joint optimization strategy based on prototype alignment is proposed. By jointly leveraging predictions from the source model and CLIP, semantic prototypes for known classes and clustering-based prototypes for unknown classes are constructed, and pseudo-labels are dynamically updated based on the similarity between samples and prototypes. In addition, a prototype contrastive loss is employed to reinforce intra-class cohesion and inter-class separation. Experimental results show that the proposed method outperforms existing SF-OSDA methods on multiple standard datasets (Office-31, Office-Home, VisDA-2017), demonstrating its effectiveness and generalization ability in open-set source-free scenarios.